Lazy iteration vs. array chaining on 500k rows – benchmark results

I built a TypeScript iterator library (iterflow) and wanted to measure the actual heap difference between lazy and eager pipelines. This is the benchmark writeup.

The pipelines

Eager - standard array chaining:

const data = Array.from(generateRows(500_000));

const results = data .filter(r => r.active && r.value > threshold) .map(r => ({ id: r.id, score: r.value * 1.5 })) .slice(0, 10_000);

Each step produces a new intermediate array. .filter() allocates one, .map() allocates another, .slice() then discards most of both.

Lazy - same pipeline via iterflow:

import { iter } from '@mathscapes/iterflow';

const results = iter(generateRows(500_000)) .filter(r => r.active && r.value > threshold) .map(r => ({ id: r.id, score: r.value * 1.5 })) .take(10_000) .toArray(); generateRows is a generator, yields one row at a time. Nothing is materialized until .toArray() pulls values through the chain. No intermediate arrays.

Results

Dataset: 500,000 rows Pipeline: filter(active && value > 5000) → map(score) → take(10,000)

native array (.filter → .map → .slice) 15.4 MB (min 15.2 MB, max 16.2 MB) iterflow (.filter → .map → .take) 5.8 MB (min 5.8 MB, max 5.8 MB)

Methodology

- Metric: heapUsed delta before and after the pipeline, not total process memory - Both pipelines start from the same generator source — the delta measures pipeline allocations only, not source data --expose-gc with explicit gc() calls forced between every run - One warm-up run discarded before measurement - Median of 5 runs reported - The native array run materializes the full 500k dataset into data before the pipeline runs. That allocation is not included in the delta - both approaches are measured on the same footing.

A few notes on the library

- iter() is a wrapper around ES2015 generators and the iterator protocol - no magic, just a fluent API so the call site looks identical to array chaining - .sum() and .mean() are typed to Iterflow<number> only - calling them on a non-numeric iterator is a compile error - Has some streaming statistical operations (.streamingMean(), .ewma(), .windowedMin()) for running aggregations without a separate accumulator - Zero runtime dependencies

https://www.npmjs.com/package/@mathscapes/iterflow