That's a neat little implementation. For bonus points, try adding these features:
1. More non-commutative operations, like Ruby's each_cons, each_slice, chunk (different than lodash's chunk). I especially enjoy implementing these both iteratively and recursively.
2. Infinite sequences (like Python's itertools.count). You may find that implementing the other operations becomes easier when you treat the sequences themselves as being lazy and potentially infinite. Do you still need to make the distinction between commutative and non-commutative operations?
3. Operations that combine multiple sequences, like iterools.zip, itertools.product, and merge (assuming sorted sequences).
4. Operations that return multiple sequences (say, a sequence of sequences). Can you implement the inverse of itertools.zip (perhaps call it unzip)?
My first thought was much like your second point! If we have lazy operations on collections, we’re 90% of the way to `seq` & similar lazy structures.
Edit: in fact, probably 99%. You just need to define the data structures as an initial operation that materializes values for subsequent operations on demand.
Yeah, years ago I got deep into this stuff and managed to implement a lot of it for fun, thinking I was doing something original, before discovering it’s not original (in my case, discovering Seq in OCaml). I think treating everything as lazy and potential infinite makes the implementation of operations much simpler than this article’s implementation.
I think for most usage, you’re right. There are places where it probably has more overhead than you’d want in a tight loop, and you’d be better off with the “bad” imperative version. E.g. “If a tree falls in the woods, does it make a sound?
If a pure function mutates some local data in order to produce an immutable return value, is that ok?”
I’m coming from a perspective where I now work primarily in JS/TS, prefer FP by default and with it were more accessible in the ecosystem, and even prefer `reduce` over a loop nine times out of ten. But there’s still a bail point where performance is key or “this feels incredibly awkward in this environment”.
Oh yeah, for sure there could be performance implications to lazy sequences, but ideally the compiler or interpreter could figure out some nice optimizations.
I also especially like the use of a Symbol to halt iteration, and think it should be part of both the core language and libraries providing similar APIs (again likening to Clojure, similar to `reduced`).
Good article detailing a particularly high-value part of Lodash.
I tend to be highly critical of the library overall:
- It takes most of its inspiration from functional programming, but sometimes surprises with unexpected mutation.
- Surprises of that kind are especially likely to be missed, because the API is quite large, and because (as the article author mentions) it provides a lot of shorthand functionality/magic behavior that one might not expect… especially combined.
- FP patterns like composition and currying are unnecessarily ergonomically difficult, because argument order is wrong throughout most of the API; the lodash/fp module mostly addresses this, but it’s not in wide use by a lot of projects heavily dependent on Lodash.
- Where composition is accounted for, it tends to exacerbate surprises caused by magic behavior.
All of that said, the laziness of chain is one bright spot that (afaik) is pretty faithful to FP expectations, notwithstanding those other issues. And while I’m not personally a fan of “fluent” APIs and chaining syntax, with laziness it’s not hard to squint and see it as roughly analogous to a threading macro/pipe operator.
IMO there are better-overall FP libraries in the ecosystem, but they tend to feel very unidiomatic. E.g. dropping in Haskell-like semantics and names that are likely unfamiliar to JS/TS devs who would nonetheless be familiar with similar equivalents in the ecosystem.
These days I mostly treat Lodash as YAGNI, and very seldom install or import it unless there’s a strong team preference. But I think, given its prevalence, I’d like to see a fresh “Lodash done right” which:
- Exports functions rather than a namespace (ESM and tree-shaking friendly, encourages use and composition of plain functions)
- Adopts either the native-equivalent name (if available) or the familiar Lodash name for functions (with convenience aliases if desired, but ideally isolated in some compat module)
- Places arguments in an order one would expect coming from the equivalent implementation in a FP language (collection first, operator/parameter after) to encourage composition
- Eliminates all implicit magic (missing arguments are treated as a mistake; null is a value)
- Provides/expects lightweight data types for explicit magic (eg `path('foo.bar')` is better and safer than 'foo.bar')
- Embraces evolution of the language/platform, deferring to native implementations as they’re available/introduced
- Engaged with the standardization process to coordinate and advocate for the principles and benefits of FP functionality
… and all of that’s a lot to ask. Certainly beyond my bandwidth alone. But I’d be happy to collaborate if other folks want to see it happen :)
I tried to nudge my last team over to Ramda and… nope, team wasn’t interested. I think there’s a good opportunity for a library that has Ramada’s principles but appeals to Lodash users.
Assuming we want `canDrink` to exist in those objects for later use, reversing the steps here would mean duplicating the logic and the work of computing it across the map() and the filter(), instead of keeping those concerns separate and re-using values:
but map also creates a new array, and within map you are not mutating the items of the old array but creating brand new ones no?
[...].map(p => ({...p, canDrink}))
^ the original items of the array are not mutated in this map callback. you could mutate them if you really wanted, i.e.
I don't see how lodash could possibly filter a list of 30 by a criteria without visiting each item at least once to take a look at it. the only way to make it more efficient would be to have lodash collect the desired transformations (filter/map) and then run a single loop over the items, mapping only those items that meet the filter criteria. But that is not much more efficient than the [].filter().map() that you could do with native methods.
It doesn't have to visit all 30 items, just enough items to take 5 from, could be 8, could be 10 etc. The lazy approach _is_ to collect all transformations and apply them in the end.
is not only unoptimized in the sense that it does O(n) age increments, but also does O(n) object clones (which are far more expensive). It also does twice as many array allocation as the naive procedural approach:
const ps = [];
for (const p of persons) {
if (p.age > 29) ps.push(p);
if (ps.length === 5) break;
}
If one wants to argue about readability, consider that the lazy approach requires understanding the semantics of both `chain` and `take` (in addition to all the mentioned downsides about extra code), whereas pretty much everything in the procedural approach can be found in a beginners JS course (or most other mainstream languages). And for an advanced developer's eye, the cost of the snippet is explicit.
Takeaway: sometimes, the seemingly overly simplistic solution is the best choice, and conversely an overly complex solution is a product of a semi-irrational bias, rather than objective analysis.
Comparisons of lodash to native methods are interesting but they always miss the real selling points of lodash in my opinion, which are the richer data structure manipulations like groupBy, partition, intersection, difference, union, etc.
This is very true, and what I mostly use lodash for these days. I also like the string utilities such as capitalize, and the utilities for dealing / getting random values.
24 comments
[ 3.2 ms ] story [ 45.8 ms ] thread1. More non-commutative operations, like Ruby's each_cons, each_slice, chunk (different than lodash's chunk). I especially enjoy implementing these both iteratively and recursively.
2. Infinite sequences (like Python's itertools.count). You may find that implementing the other operations becomes easier when you treat the sequences themselves as being lazy and potentially infinite. Do you still need to make the distinction between commutative and non-commutative operations?
3. Operations that combine multiple sequences, like iterools.zip, itertools.product, and merge (assuming sorted sequences).
4. Operations that return multiple sequences (say, a sequence of sequences). Can you implement the inverse of itertools.zip (perhaps call it unzip)?
Edit: in fact, probably 99%. You just need to define the data structures as an initial operation that materializes values for subsequent operations on demand.
I’m coming from a perspective where I now work primarily in JS/TS, prefer FP by default and with it were more accessible in the ecosystem, and even prefer `reduce` over a loop nine times out of ten. But there’s still a bail point where performance is key or “this feels incredibly awkward in this environment”.
I tend to be highly critical of the library overall:
- It takes most of its inspiration from functional programming, but sometimes surprises with unexpected mutation.
- Surprises of that kind are especially likely to be missed, because the API is quite large, and because (as the article author mentions) it provides a lot of shorthand functionality/magic behavior that one might not expect… especially combined.
- FP patterns like composition and currying are unnecessarily ergonomically difficult, because argument order is wrong throughout most of the API; the lodash/fp module mostly addresses this, but it’s not in wide use by a lot of projects heavily dependent on Lodash.
- Where composition is accounted for, it tends to exacerbate surprises caused by magic behavior.
All of that said, the laziness of chain is one bright spot that (afaik) is pretty faithful to FP expectations, notwithstanding those other issues. And while I’m not personally a fan of “fluent” APIs and chaining syntax, with laziness it’s not hard to squint and see it as roughly analogous to a threading macro/pipe operator.
IMO there are better-overall FP libraries in the ecosystem, but they tend to feel very unidiomatic. E.g. dropping in Haskell-like semantics and names that are likely unfamiliar to JS/TS devs who would nonetheless be familiar with similar equivalents in the ecosystem.
These days I mostly treat Lodash as YAGNI, and very seldom install or import it unless there’s a strong team preference. But I think, given its prevalence, I’d like to see a fresh “Lodash done right” which:
- Exports functions rather than a namespace (ESM and tree-shaking friendly, encourages use and composition of plain functions)
- Adopts either the native-equivalent name (if available) or the familiar Lodash name for functions (with convenience aliases if desired, but ideally isolated in some compat module)
- Places arguments in an order one would expect coming from the equivalent implementation in a FP language (collection first, operator/parameter after) to encourage composition
- Eliminates all implicit magic (missing arguments are treated as a mistake; null is a value)
- Provides/expects lightweight data types for explicit magic (eg `path('foo.bar')` is better and safer than 'foo.bar')
- Embraces evolution of the language/platform, deferring to native implementations as they’re available/introduced
- Engaged with the standardization process to coordinate and advocate for the principles and benefits of FP functionality
… and all of that’s a lot to ask. Certainly beyond my bandwidth alone. But I’d be happy to collaborate if other folks want to see it happen :)
persons .map(p => ({ ...p, age: p.age + 1 })) .filter(p => p.age >= 30) .slice(0, 5);
^ But this is intentionally(?) unoptimized. You would instead put the filter and slice first, i.e.
person.filter(p => p.age >= 29) .slice(0,5) .map(p => ({...p, age: p.age + 1})
^ the original items of the array are not mutated in this map callback. you could mutate them if you really wanted, i.e.
[...].map(p => {p.canDrink = p.age > 20; return p})
but that is a straight up hack.
Consider that the initial idiom
is not only unoptimized in the sense that it does O(n) age increments, but also does O(n) object clones (which are far more expensive). It also does twice as many array allocation as the naive procedural approach: If one wants to argue about readability, consider that the lazy approach requires understanding the semantics of both `chain` and `take` (in addition to all the mentioned downsides about extra code), whereas pretty much everything in the procedural approach can be found in a beginners JS course (or most other mainstream languages). And for an advanced developer's eye, the cost of the snippet is explicit.Takeaway: sometimes, the seemingly overly simplistic solution is the best choice, and conversely an overly complex solution is a product of a semi-irrational bias, rather than objective analysis.