Agree, i thought the name is awful as I expected to see something related to I/O and buffering of content. It looks like a handy list object and seeing "stream" everywhere was just blinding.
However, right at the bottom the etymology of the name is made clear:
> The name 'stream' is used in Scheme, a LISP dialect that supports these features.
So the name is from the FP community. Given the cultural issues, a name change seems unlikely...
I didn't know that series of lectures, so thanks a hundred times. (Just spend the evening watching the first two hours and bought Harvey's book as well.)
Don't you think that a data stream is a special case of this kind f stream that just happens to have a bunch f idiosyncratic names in it's API? It's not like these streams are a completely different idea.
Streams and other lazy infinite data structures can be very convenient, but this isn't really obvious at first glance. I recommend you check out some of the FP literature. A good place to start reading about the virtues of streams and lazy evaluation is "Why functional programming matters" (available at http://www.cs.utexas.edu/~shmat/courses/cs345/whyfp.pdf).
It's a good concept and all, the issue i've had with it for my own use is that the native Array functions are so fast in JavaScript (and function invocation so comparatively slow) that you really need a very specific use case for them to be worthwhile over simply performing your operations on the array directly like Underscore.js.
As someone who conjured this exact thing up in another language after reading SICP, I have to suggest that the first two commenters here missed the point: this is a very basic but powerful structure that implements a lazy stream and is nearly a monad (many standard options are written for you but could be generalized).
Want it to apply to data I/O? Stick your reader into the HEAD function, store the pointer to the next byte in TAIL, and implement functions for reading X number of bytes and converting them into their new types.
Yes, you can do math with it. You can also implement process/procedure into it, if you consider yourself popping lambdas off the stream, and which lambda is Next can be guided by choices in the current lambda.
Mostly, you can use this device to hold any collection of things and operate upon them in a streamy-syntax kind of way. For those who are familiar with jQuery, it is a bit of a reimplementation of a lot of jQuery's basic array-data handling with extra helpers like "range" tossed in and a stronger focus on the laziness. Where jQuery's prime target is work with the DOM, this stream package's target is the more theoretical version of some of jQuery's features, specifically those that iterate over the contained arrays. You could wrap jQuery in this and get an amazing hybrid.
Don't sell it short; it really is another useful and powerful way to approach data management. Check out how you construct your own stream with this package, and think beyond the basic numeric approaches.
I didn't get very far with it though - the combined issues of JavaScript's lack of tail-call optimisation and its verbose lambda syntax made building streams to painful.
This looks like an interesting approach, though you'll never get around JS's heavy lambdas (unless you use CoffeeScript or something).
"A monad for enumerating sets: like the list monad, but impervious to infinite descent.
A depth-first search of a data structure can fail to give a full traversal if it has an infinitely deep path. Likewise, a breadth-first search of a data structure can fall short if it has an infinitely branching node. Omega addresses this problem by using a "diagonal" traversal that gracefully dissolves such data."
I just want to quote something you said for the sheer awesome implications of it. "You can also implement process/procedure into it, if you consider yourself popping lambdas off the stream, and which lambda is Next can be guided by choices in the current lambda." This. Is. Powerful.
There is also "LINQ for Javascript", which is fully integrated in with JQuery and VS as well. http://linqjs.codeplex.com/
Streams seems to be a proper subset of the things you can do with linqjs. Sometimes MSFT can do cool things too, it just doesn't always get picked up :)
Thank you for mentioning this—I had no idea it existed! Do you use lazy enumerables in your projects?
I can think of a few fun uses (such using them as creative replacements for looping constructs), but I'm struggling to find a problem that is most easily solvable with these.
Yes, I use them liberally in C# projects. In fact, I prefer to use them over the standard looping constructs because of the pipe-like characteristics. It's very useful.
buddydvd, linq looks quite nice. Thanks for mentioning it, I had no idea it existed. I added a link to it at the 'Tribute' section of stream.js in case someone wants to check it out or prefers its syntax instead :)
It'd be great if the websites for JavaScript libraries like this one would actually include the library in the page, so that while I'm reading about it, I can open up the JavaScript debugging console and actually try out the library as I'm reading about it.
Thanks, that's a great idea! I made it load up the library and added a note in the tutorial that one can use their browser JS console to fiddle with it :)
Here's another example you might want to include, if you're going for the whole Haskell angle. Not as elegant, but illustrates the concept:
var fibs = new Stream(1, function() {
return new Stream(1, function() {
return fibs.zip(function(a, b) { return a + b; }, fibs.tail());
});
});
fibs.take(10).print();
Maybe I don't get it, but what is the practical use-case for this? Infinite arrays of numbers - maybe useful for math or statistical applications, but are people using javascript for this over things like MATLAB, R, etc?
I can't think of anywhere to use this in the context of building applications and I certainly don't feel like it "completely changed the way I think about programming" at all.
Edit: So it looks like you can put more than just numbers into the streams -- still what does this do that Underscore.js doesn't?
Sounds like generators in python, but implemented in javascript. The head element and tail function was a interesting way to view them.
Generators a great way to do processing on a lot of data, in a very list-y way, without having to read all the data into memory first. It allows you to start seeing results straight away.
Just curious -- and be honest -- the snippet at the end they ask you to decipher (and they say "Most programmers find it hard to understand unless they have a functional programming background, so don't feel bad if you don't get it immediately") did you have trouble with it?
For me, I thought I had the answer after maybe 30 seconds of reading it, but decided to "check my work" by walking through it again, and only then, a moment later, did I realize what it was actually doing with that recursive call.
I'm just curious about their "most programmers without a functional background" remark. So: did ya get it?
Yes, but it would have been significantly more baffling if the word "sieve" didn't hint at its behavior.
Still, I'm not sure that a functional background would have helped, or maybe my background is more functional than I think. I know some Python, and I read a few articles about functional programming a couple of years ago, but I don't think I could give you a clear definition of what functional programming is.
The name "sieve" gave it away for me before I'd even looked at the code, but I decided to make sure I was right. Then i looked at it and my mind was kind of blown but how neat and concise it was. This definitely makes me want to take a look at functional programming (just ordered "The Little Schemer")
This comment thread reminds me a bit of "The Emperor's new clothes". I think the code for sieve (even though I understood what it did) is way too unnecessarily confusing and I would not want to see it in a project I'm on.
However, I do appreciate the usefulness of lazy evaluation. Still, I think this documentation could benefit by not making outlandish claims ("it will change the way you think") and just saying the author has implemented lazy evaluation in javascript (and maybe link to the wikipedia article on lazy evaluation and this stack overflow to explain why it's useful: http://stackoverflow.com/questions/2151226/what-are-the-adva...)
If your project team has people comfortable with functional programming - there's absolutely nothing wrong with that code.
As far as changing the way you think, it certainly came as a shock for me when I encountered it years ago in SICP and I'm still enjoying the benefits of lazy sequences in Clojure and ClojureScript.
am i the only one here who finds the whole "omfg, this is so amazing!!1" tone of this article mildly offensive? i mean, it's not like he has a self-levitating hamster or anything.
I felt roughly the same way about it that I would feel about the frolicking of a self-levitating hamster: pretty silly, but not actually offensive.
Lazy lists are pretty neat, and if you make them a convenient part of a programming language and its software ecosystem, they can be useful. This JavaScript version, though, looks more like a clever toy at the moment. Maybe that will change eventually.
skrebbel, thank you for your feedback. I'm sorry it felt that way to you. In no way do I think my readers are less smart than me. I tried in the article to keep a tone that addresses my past self, before I learned about infinite lists in Haskell or streams in Scheme. In that sense, it was a revelation to me, which made me a better programmer. So I'm hoping that it may have introduced at least some of the people reading it to this different paradigm, which, to me, was enlightening.
In his defense, the first time I really understood Sequences after reading The Joy of Clojure, I thought it was awesome too.
Now here's where I really blow your brain: a Sequence is some stuff, in a particular order. What about the keyUp event? If I were to listen to that event, it might give me ['T','e','s','t'].
That too, is some stuff, in order. This means, that Events are Sequences. Which means, that all the cool stuff that can be done to Sequences, can be done to an Event too, which is exactly what the Reactive Extensions for JS from Microsoft does (disclaimer: I'm writing a book on this).
Haskell of course has this too, it's called the Continuation Monad.
Also see the paper "A Concurrent Window System" by Pike (89) which talks about representing keyboard and mouse events as items on a non-finite potentially-blocking channel. And Newsqueak almost certainly isn't the first.
Embracing functional programming in Javascript is trending. All these dollars, underscores and streams will make for an interesting benchmark against ClojureScript as it matures.
In ClojureScript, streams or lazy lists are an integral language feature. Standalone javascript libraries can not provide a functional framework of the same calibre. You would naturally end up wanting to, e.g., run an underscore.js chain on a stream and that would require more work.
Um, 'new data structure'? You have to be kidding. This is implemented in practically every single functional programming language. Also, this was already done in JavaScript: https://github.com/Gozala/streamer and looks surprisingly similar to your library. As you can see, streams in JavaScript are nothing new.
I've been playing around with this in my current project, yet I'm not convinced this is always such a great model. It requires a lot of mental work to remember what types you are operating on at each state in the pipeline. That being said, I'm a big fan of LINQ, and I'd like to see this model applied more frequently to DB queries.
The advantage of "streams" is that it can be used to elegantly define infinite sequences recursively.
For example, you can define powers of two as:
var powersOf2 = new Stream(1, function() {
return powersOf2.scale(2);
});
(untested) which means that the sequence of powers of 2 begins with 1 follows with the the same sequence scaled by 2!
The implementation in the linked article however seems to be flawed, since it doesn't store the node's tail after computing it. This means that to get each element of the sequence you have to compute all the elements before it (even though you already did in order to get to that element).
This works not only for numbers but for other infinite sequences such as expressions in a language. I recommend the book Higher Order Perl if you're interested in this.
89 comments
[ 2.9 ms ] story [ 172 ms ] threadHowever, right at the bottom the etymology of the name is made clear:
> The name 'stream' is used in Scheme, a LISP dialect that supports these features.
So the name is from the FP community. Given the cultural issues, a name change seems unlikely...
Not terribly useful imho and certainly doesn't live up to the hyped intro.
Want it to apply to data I/O? Stick your reader into the HEAD function, store the pointer to the next byte in TAIL, and implement functions for reading X number of bytes and converting them into their new types.
Yes, you can do math with it. You can also implement process/procedure into it, if you consider yourself popping lambdas off the stream, and which lambda is Next can be guided by choices in the current lambda.
Mostly, you can use this device to hold any collection of things and operate upon them in a streamy-syntax kind of way. For those who are familiar with jQuery, it is a bit of a reimplementation of a lot of jQuery's basic array-data handling with extra helpers like "range" tossed in and a stronger focus on the laziness. Where jQuery's prime target is work with the DOM, this stream package's target is the more theoretical version of some of jQuery's features, specifically those that iterate over the contained arrays. You could wrap jQuery in this and get an amazing hybrid.
Don't sell it short; it really is another useful and powerful way to approach data management. Check out how you construct your own stream with this package, and think beyond the basic numeric approaches.
Me too! http://jsfiddle.net/skilldrick/vT2mC/
I didn't get very far with it though - the combined issues of JavaScript's lack of tail-call optimisation and its verbose lambda syntax made building streams to painful.
This looks like an interesting approach, though you'll never get around JS's heavy lambdas (unless you use CoffeeScript or something).
http://weblog.raganwald.com/2007/02/haskell-ruby-and-infinit...
"A monad for enumerating sets: like the list monad, but impervious to infinite descent.
A depth-first search of a data structure can fail to give a full traversal if it has an infinitely deep path. Likewise, a breadth-first search of a data structure can fall short if it has an infinitely branching node. Omega addresses this problem by using a "diagonal" traversal that gracefully dissolves such data."
http://neue.cc/reference.htm
Output:Streams seems to be a proper subset of the things you can do with linqjs. Sometimes MSFT can do cool things too, it just doesn't always get picked up :)
I can think of a few fun uses (such using them as creative replacements for looping constructs), but I'm struggling to find a problem that is most easily solvable with these.
Here's another example you might want to include, if you're going for the whole Haskell angle. Not as elegant, but illustrates the concept:
I can't think of anywhere to use this in the context of building applications and I certainly don't feel like it "completely changed the way I think about programming" at all.
Edit: So it looks like you can put more than just numbers into the streams -- still what does this do that Underscore.js doesn't?
Generators a great way to do processing on a lot of data, in a very list-y way, without having to read all the data into memory first. It allows you to start seeing results straight away.
http://danielnouri.org/docs/SuperColliderHelp/Streams-Patter...
http://danielnouri.org/docs/SuperColliderHelp/Streams-Patter...
SuperCollider is a fun language to study. You end up with compositional structures like the following (notice 'inf', which indicates an infinite loop)
However, I use Javascript functions as base instead of a Stream constructor.
For me, I thought I had the answer after maybe 30 seconds of reading it, but decided to "check my work" by walking through it again, and only then, a moment later, did I realize what it was actually doing with that recursive call.
I'm just curious about their "most programmers without a functional background" remark. So: did ya get it?
Still, I'm not sure that a functional background would have helped, or maybe my background is more functional than I think. I know some Python, and I read a few articles about functional programming a couple of years ago, but I don't think I could give you a clear definition of what functional programming is.
ClojureScript of course takes it even further.
However, I do appreciate the usefulness of lazy evaluation. Still, I think this documentation could benefit by not making outlandish claims ("it will change the way you think") and just saying the author has implemented lazy evaluation in javascript (and maybe link to the wikipedia article on lazy evaluation and this stack overflow to explain why it's useful: http://stackoverflow.com/questions/2151226/what-are-the-adva...)
As far as changing the way you think, it certainly came as a shock for me when I encountered it years ago in SICP and I'm still enjoying the benefits of lazy sequences in Clojure and ClojureScript.
Lazy lists are pretty neat, and if you make them a convenient part of a programming language and its software ecosystem, they can be useful. This JavaScript version, though, looks more like a clever toy at the moment. Maybe that will change eventually.
I can explain classes in 10 minutes and have the same effect. Assuming, of course, you've only coded FORTRAN.
In fact, you picked out the part that I mostly found mildly offensive. He's treating his readers like "less smart than him".
Now here's where I really blow your brain: a Sequence is some stuff, in a particular order. What about the keyUp event? If I were to listen to that event, it might give me ['T','e','s','t'].
That too, is some stuff, in order. This means, that Events are Sequences. Which means, that all the cool stuff that can be done to Sequences, can be done to an Event too, which is exactly what the Reactive Extensions for JS from Microsoft does (disclaimer: I'm writing a book on this).
Haskell of course has this too, it's called the Continuation Monad.
That's hilarious! I'm seriously fighting my social-urge to upvote...
In ClojureScript, streams or lazy lists are an integral language feature. Standalone javascript libraries can not provide a functional framework of the same calibre. You would naturally end up wanting to, e.g., run an underscore.js chain on a stream and that would require more work.
I've been playing around with this in my current project, yet I'm not convinced this is always such a great model. It requires a lot of mental work to remember what types you are operating on at each state in the pipeline. That being said, I'm a big fan of LINQ, and I'd like to see this model applied more frequently to DB queries.
For example, you can define powers of two as:
(untested) which means that the sequence of powers of 2 begins with 1 follows with the the same sequence scaled by 2!The implementation in the linked article however seems to be flawed, since it doesn't store the node's tail after computing it. This means that to get each element of the sequence you have to compute all the elements before it (even though you already did in order to get to that element).
This works not only for numbers but for other infinite sequences such as expressions in a language. I recommend the book Higher Order Perl if you're interested in this.