For those that don't know TJ Holowaychuk, he's the guy behind express and commander.js (and a lot more) which are two of the most popular node.js packages.
Mocha is the only JavaScript testing framework that works on the server and the browser and is simply beautiful. Superagent is likewise a simple and obvious API that works identically in both environments.
TJ has done quite a bit to advance the state of the art in JavaScript development, it will be interesting to see what he does with Go.
"First, lets admit the facts: as of today, it is possible to produce a competitive Node application whose entire stack will comfortably rest on various frameworks, tools and libraries developed exclusively by TJ. If you are doing things with Node, you will not escape TJ. He is omnipresent."
I've been using his libraries since I started out with node, and there isn't a single project that I'm working on today that doesn't have at least 2 or 3 of his libraries as dependencies.
He has contributed a lot to the nodejs ecosystem (around 600 packages! [1]), and it saddens me to see him go. The Go guys should be very excited to have him on board!
Yeah, it's absolutely crazy how much that dude put out there over the last years, totally impossible to do JS web dev work without using at least a few of his libraries like Express, Jade, Mocha, Co. and then one day it's like "alright done with that." If I could be even 1/2 that productive...
Or maybe Google is behind this. They created an icon within an emerging community where all the cool kids go, in the hope they will follow once he publicly converts to golangism
Couldn't agree more with this post. Node has always had a problem with favoring performance over robustness, coupled with an overly zealous, polarized community that exists in a bubble. This resulted in synchronous libraries like fibers being shot down without proper consideration.
At StartHQ (https://starthq.com) we've abstracted away all async code with a sync layer using fibers and that has served us well over the past year, but I can totally see how trying to mix the two would have been a recipe for disaster.
> Node has always had a problem with favoring performance over robustness
I don't disagree with the gist of what you're saying, and I think you're correct. But the premise of Node being high-performance is perhaps more reputation than actual fact. I think it gets that reputation because it is higher-performance than some more popular options, but in the big scheme of things, it's but a medium-performance platform [1].
Agreed. I benchmarked Node/V8 vs. the JVM and while Node has lower memory usage, Java is as performant while being more reliable, with less variance in HTTP response times, due to the better garbage collector.
There is no guarantee that a custom fork with fibers will be maintained and patched in the long term. Generators are a safer, forwards-compatible choice for doing async work. That's why there is little support in the community for adopting fibers.
The big issue with generators is making sense of stack traces while debugging. And due to the nature of how generators work we'll probably need an fairly creative solution here. I'm fine with generators today, but it would be better if await sugar landed sooner.
I stand corrected. However, the point still holds.
- It is outside the JS specification
- Needs to be updated for newer versions of node. Last commit says "Support for Node 0.11.10". Also, it needs to keep up specifically for every platform.
- Will never work in the browser, unlike yield which you can get working on current browsers using transpilers. Increasingly important, as we see more code sharing between client and server (isomorphic JS).
The JS spec says nothing about threads or coroutines, just like the C spec says nothing about them either. That doesn't mean that they are incompatible with the language. In fact, real multi-threaded JS code runs just fine on the JVM for example.
It works fine with the latest Node for me. Fibers relies on libcoro which has only had minor updates over the past four years.
I'm not sure where you're seeing this code sharing. In practice the only code that is shared is utility methods that don't do any I/O, so whether you use fibers or not makes no difference.
In fact, Meteor, being the framework that is pushing the boundaries of code sharing between client and server, uses fibers on the server.
I'm still curious about a js engine that supports coroutines better (fiber's stacks are pretty huge), it would be interesting to start fresh without all the legacy baggage that node has now.
One could fork fibers and have smaller stacks and even make their size dynamic, but actually the number of active fibers per process has been fairly low in all our use cases so stack size was never a concern. The more interesting problem is that of running multiple processes in parallel so you can saturate the increasing number of cores, while retaining low jitter and having a sensible end user API.
My co-founder alexlamsl and I are actually working on something akin to Scala actors for JS that is portable between the browser via Web Workers and the server using Node and Common Node (https://github.com/olegp/common-node). We'll blog about it at https://starthq.com/blog/ in the near future.
That actually can be easily fixed on top of V8 - as V8 already sprinkles stack overflow checks you can make actual fiber stack smaller and expand it on demand.
visionmedia is his screen name, not the company where he works. He worked at a company called LearnBoost (https://www.learnboost.com/blog/welcome-aaron-and-tj/), which I can only assume is some sort of mob-style front for laundering JavaScript code. (Look at their employee list and GitHub repo)
Now, I've heard he works for Segement.io (in SOMA), but does so remotely.
Maybe Golang is great for developing frameworks, but as someone who build websites i prefer Node because Go's typing makes development slower and more expensive.
Dynamic typing was very cool when I started out with Node, too. It lets you get things wired up very quickly and come up with a very functional app with almost no friction. But in my experience, once the app reaches a point where I no longer remember every single line of code that was written in it, I start to miss what a type system could tell you.
Hmm, do I pass a string in the first parameter, or was it the second? What does that funciton return? Am I really getting a number in this parameter? Of course, you can get around that confusion by adding more unit tests and parameter validation but with strong typing I'd get all that for free.
Maybe try Typescript, it has optional typing and is compatible with normal Javascript. It really solves those problems you mention and will give you compile errors if you send parameters of the wrong type to functions.
While I don't disagree with you that that's inconvenient and totally solved by strongly typed languages (I love having my arguments filled in when writing a function call in Go), this usually isn't a big deal in literally any of the production Node systems I've worked on (some of which are particularly complex).
At my shop, we emphasize making obvious (if somewhat verbose at times) names for functions, so that it's pretty easy to intuit what arguments it takes. Also a naked function (e.g. "foo(bar)") will always be defined in the same (<100 line) file, so it's not hard to find it, or in a module (which is a Cmd+P away).
All this to say, while I agree with you that strong type systems mostly eliminate this pain, it's not exactly "painful" in Javascript either.
I totally see your point. I think what's tripping me up is that I'm new to node/JavaScript and coming from a strongly typed world, I'm still writing my apps with that framework in mind. Working with dynamically typed code requires you to think about your application differently and I'm just starting to understand the patterns.
I understand that. There certainly is a learning curve, and I don't pretend that all the Node stuff I've written since day 0 has been awesome - I've churned out my fair share of nasty, unmaintainable garbage because I abused the freedom that Javascript gave me.
One tip: rubber ducking is really valuable. If it takes more than ~30 seconds to explain whats going on in ~10 lines of code, then you probably should do some abstracting.
I actually prefer stronger typing and Typescript solves that for me for Javascript+Node.js
The bigger reason I see for sticking with Node.js for web development is being able to use Javascript for both server and client code.
Newer Javascript frameworks and templating libraries such as dust.js, Ractive.js, React.js can render both on server side and client side so on the initial requst the server can send a fully rendered page for SEO and performance, and then client side Javascript can dynamically update the page using the same code/templates.
We're a pretty heavy Go production shop, but today I decided that we would move our dumb rails codebase (we were really only using the asset pipeline) to nodejs+mincer (mainly to get rid ruby).
Hope the departure of an incredibly js developer doesn't impact the community too badly.
I guess the important lesson from him here (which he mentions) is to get out of your comfort zone and learn new stuff, as to be able to tackle problems with a wider view of things and solve them with the most adequate tools.
I was suprised to see the comment that tooling for Node wasn't great; I thought the node tooling (npm, gulp, grunt, etc.) was significantly superior to the primitive (ie. relatively low level) tooling go provides.
...but I see that's a very specific comment:
the tooling Go provides for profiling and debugging is great...
Ah, that makes more sense. This is definitely an area where node applications are really troublesome to work with (live debugging).
I was surprised as well, until I saw that. The npm/gulp/bower/less/node stack is pretty awesome for full-stack web development. All in one language, all open source, easy-breezy package management. If Go had that sort of complete stack (and for God's sake a package manager to rival npm), I'd probably be pushing Go as hard as possible at work.
As it stands, there's way too much friction and it would cost a LOT of money to invest in Go. There's a good chance that the ecosystem may not evolve to Node's level for years - and worst of all, if you want to build competitive web apps, you still can't avoid Javascript.
> We could achieve similar things in Node with generators, but in my opinion generators will only ever get us half way there
I wish more people realized this. Generators seem to be billed (ironically by TJ amongst others with his Koa project) as the solution to callback hell but, having tried Go, I got the same felling that actors are much easier to reason about and debug (although concurrency is still hard).
On the client side, web workers allow parallelism but the API is inconvenient to use: you need to have a file for each web worker whereas Go's `go` routines are akin to a function call. In addition to this standardized conundrum, you have browsers discrepancies with the APIs available inside a worker varying between vendors [1].
On node, you have the cluster API, which allows managing several processes so it's even further from light threads. On the bright side, most (all?) APIs have both a sync and async version.
As a result, there's nothing you can use for multithreading in a module without coordinating with the whole project. I think JavaScript needs language support for light threads, not APIs for multiprocessing.
Nodejs could have workers,it's not really a javascript.Node maintainers just didnt implement that stuff.
I personnally dont like generators for concurrency.It's the wrong solution to the problem.For now I stick with promises,and I hope JS gets async/await keywords quickly.
Agreed, but what I meant is that the API (running a file in a worker) is inconvenient, first because there is no closure. Go allows the go routine to read variables in scope and has a race detector to tell you when your accesses are unsafe. There's also the issue of managing workers: is a web worker an OS thread or a light thread [1]? how many can you spawn? which worker should you send work to? These problems are solved by Go. I think it would be very hard to obtain the same reliably and performance with today's web workers. Other languages solve this with a language construct, not an API to run another file.
> I hope JS gets async/await keywords quickly
It would be a step up. But most of the time you want to express a linear list of tasks. Javascript should support this by having a synchronous variant of its APIs and having light threads.
My hope is more that some language having such features will compile to asm.js
This post really resonates with me. I'm not as prolific and don't have the ability to breeze into a new language on a whim, but seeing so many other languages gaining momentum nowadays really makes me feel restless. All the best of luck to him, and I certainly hope I'm not stuck in the same rut the next ten years.
Server side dev folks have always enjoyed choice of language/system when it comes to building. You can simply choose the thing that works best for you, that you're more comfortable with, is most performant for your app, etc. On the client side, we have ... javascript. While JS is pretty fast these days everywhere and we can cross compile with emscripten and all, I'm almost tempted to say that the true web standard for programmability on both client-side and server side ought to be some kind of a byte-code .. a concept the Java guys had right way back.
If you consider that we have PNaCl compiling llvm-bitcode to native within the browser, Emscripten cross compiling llvm-bitcode to asm.js (which Firefox's engine converts to native), Safari's engine compiling js itself to llvm-bitcode, I'm thinking why not make llvm-bitcode an open web standard? That way, any language that compiles to llvm-bitcode would be equally usable on the server side and on the client side.
The biggest potential downside to that would probably be educational. I learn a lot by looking at source code and the web was always open in that way, though with minified JS these days that's not so possible any more. However, it might make it easier to distribute source code in any language, since a module can then compile and run such code right in the browser.
edit: These thoughts came up when I was thinking about how TJ won't be able to take his work with him as he moves to Go. Well, I suppose he wants to do new kind of work for which Go is better suited, but is then all of the prior work just a brief view along the journey?
> Additionally we are now past Rails wave, Node.js wave and into Go wave.
And the narrative is similar as well: starting a new project with the shiny new thing is cool but maintaining it is boring and awful and to quote the OP "I need maintainers!".
Part of the problem is the language of course. Maintaining even a medium sized codebase written in a dynamic language is a challenge. But the other part is the mantra that "coding is easy" and "everyone should do it". And now there's an entire generation of developers whose idea of software development is "writing an app in Node in a weekend". Which of course doesn't require years of maintenance, complicated tooling or even adherence to common best practice.
Reminds me of Zed Shaw leaving Ruby... Honestly, Node.js has done a lot, but, come on, it's still JavaScript in the core. The selling point that you can run the same code both on the client and on the server doesn't really hold up to the reality check. I'm not so excited about Go, I honestly find Rust better. Dart also seems promising - especially having dart2js and Google backing it, too, but it doesn't seem to be picking hype as fast as Ruby and then Node.js did. To me, having a powerful package manager seems to be the common theme and they key to success. I still can't believe with all the ego, Python is still relying on a joke like pip. Although I agree it would be challenging, I think the world will be a better place if there's a cross-language package manager, with language-specific plugins. At the end of the day, maybe 75% of the logic is shared and is around downloading things, installing symlinks, checking dependencies, and so on. Package that with a cross-language pyenv/rbenv/nodeenv, etc., and you would have a killer.
> I'm not so excited about Go, I honestly find Rust better.
Same here, but Go has productive libraries that can help you hit the ground running. Just with the packaged standard libraries, you can do things like have a useful webserver going. I prefer Rust though and might stick with JS until I exhaust it .. or until Rust matures enough to put into production and then jump fully on to Rust. Rust compiles to llvm-bitcode, unlike Go, so that's a win for me.
Yes, until a few days ago, I didn't know that it supports OS X as well, so, I installed it, but didn't have time to look deeper into it. Thanks for reminding me, I'm definitely gonna look into it!
... or Rust, given that the language should have matured by then.
But other than that, I'm also waiting for people to eventually discard Go and move on to one of those languages...
He did give erlang a try and tweeted about it 4 years back. I think judging from some of this other comments on disqus(I can't find them at the moment) he did not like it very much.
He is one of the topmost contributors to the node.js ecosystem if not the most. He has given a plenty of good libraries. Besides the reasons he gave to leave node.js, I feel that he was under-appreciated and over-worked many times. With components he tried to create a toolchain, wrote express and koa for server, jade for templating, mocha for debugging etc.
He tried to do a lot many things single-handedly. Which is why he needs a lot of maintainers. I don't blame him for overloading and stressing out. That is the way how it goes in node.js. Working in node.js is like battling a multi-headed beast. The core javascript keeps evolving, node.js isn't even in version 1.0, API changes a lot. There are lot of blanks to fill and too much time goes into boilerplate stuff and managing existing code.
It is just not rewarding and fulfilling to contribute to node.js. On top of that he faced a lot of friction from other contributors. He had some really difficult times, like express-connect conflict, bower-components conflict. Having handled conflicts too many, some people also consider him as being rude and too self-centered. Besides he is young and is probably one of the first things he has done he is serious about.
Many times before I asked to myself how does this guy do it. In a community where to contribute even a little one has to put in a lot of effort, his contributions are gigantic. I hope he continues to work, although not at too many projects . His experiences will definitely be handy.
On top of that I remember the first nodeup podcasts and on twitter some key member from the community would bitch about express all the time. Ranging from express is bloated to middleware is a shitty abstraction to you don't need a framework in Node.js. It was really weird because express was (is?) one of the most used framework in the Node.js community and here you have people that other people follow and listen to tell you that it's crap.
Haha yup, I wont lie I have been an asshole at times, mostly because you sort of have to defend yourself when you're surrounded by people who attack your work, but after a while I realized it wasn't worth it so I just sort of detached (no longer on IRC, the mailing list etc). Defending your work isn't worth becoming rude yourself. Communities always form cliques that's just how it is.
The core node community was very unreceptive to change since the very beginning, to be honest Ryan and Isaac were some of the few humble people in the bunch. I've always been a "threat" to "the node way" of doing things, whatever that means. The elitism gets a little old.
94 comments
[ 4.0 ms ] story [ 196 ms ] threadTJ has done quite a bit to advance the state of the art in JavaScript development, it will be interesting to see what he does with Go.
http://www.quora.com/TJ-Holowaychuk-1/How-is-TJ-Holowaychuk-...
He has contributed a lot to the nodejs ecosystem (around 600 packages! [1]), and it saddens me to see him go. The Go guys should be very excited to have him on board!
[1] https://www.npmjs.org/~tjholowaychuk
Also listed on his GitHub - https://github.com/visionmedia
At StartHQ (https://starthq.com) we've abstracted away all async code with a sync layer using fibers and that has served us well over the past year, but I can totally see how trying to mix the two would have been a recipe for disaster.
Here is a video of a talk we gave on this: http://www.youtube.com/watch?v=pmyDJnEza6A and here are the slides: http://www.slideshare.net/olegp/start-hq-server-side
I don't disagree with the gist of what you're saying, and I think you're correct. But the premise of Node being high-performance is perhaps more reputation than actual fact. I think it gets that reputation because it is higher-performance than some more popular options, but in the big scheme of things, it's but a medium-performance platform [1].
[1] http://www.techempower.com/benchmarks/#section=data-r9&hw=i7...
http://www.slideshare.net/olegp/server-side-javascript-going... - there are some graphs from slide 46 onwards.
The big issue with generators is making sense of stack traces while debugging. And due to the nature of how generators work we'll probably need an fairly creative solution here. I'm fine with generators today, but it would be better if await sugar landed sooner.
- It is outside the JS specification
- Needs to be updated for newer versions of node. Last commit says "Support for Node 0.11.10". Also, it needs to keep up specifically for every platform.
- Will never work in the browser, unlike yield which you can get working on current browsers using transpilers. Increasingly important, as we see more code sharing between client and server (isomorphic JS).
If you ask me, fibers are quite risky.
It works fine with the latest Node for me. Fibers relies on libcoro which has only had minor updates over the past four years.
I'm not sure where you're seeing this code sharing. In practice the only code that is shared is utility methods that don't do any I/O, so whether you use fibers or not makes no difference.
In fact, Meteor, being the framework that is pushing the boundaries of code sharing between client and server, uses fibers on the server.
My co-founder alexlamsl and I are actually working on something akin to Scala actors for JS that is portable between the browser via Web Workers and the server using Node and Common Node (https://github.com/olegp/common-node). We'll blog about it at https://starthq.com/blog/ in the near future.
Now, I've heard he works for Segement.io (in SOMA), but does so remotely.
Hmm, do I pass a string in the first parameter, or was it the second? What does that funciton return? Am I really getting a number in this parameter? Of course, you can get around that confusion by adding more unit tests and parameter validation but with strong typing I'd get all that for free.
[0]: http://esolangs.org/wiki/Funciton
At my shop, we emphasize making obvious (if somewhat verbose at times) names for functions, so that it's pretty easy to intuit what arguments it takes. Also a naked function (e.g. "foo(bar)") will always be defined in the same (<100 line) file, so it's not hard to find it, or in a module (which is a Cmd+P away).
All this to say, while I agree with you that strong type systems mostly eliminate this pain, it's not exactly "painful" in Javascript either.
One tip: rubber ducking is really valuable. If it takes more than ~30 seconds to explain whats going on in ~10 lines of code, then you probably should do some abstracting.
The bigger reason I see for sticking with Node.js for web development is being able to use Javascript for both server and client code.
Newer Javascript frameworks and templating libraries such as dust.js, Ractive.js, React.js can render both on server side and client side so on the initial requst the server can send a fully rendered page for SEO and performance, and then client side Javascript can dynamically update the page using the same code/templates.
really means
"I am ready to trade speed of development for having more bugs"
Hope the departure of an incredibly js developer doesn't impact the community too badly.
...but I see that's a very specific comment:
Ah, that makes more sense. This is definitely an area where node applications are really troublesome to work with (live debugging).As it stands, there's way too much friction and it would cost a LOT of money to invest in Go. There's a good chance that the ecosystem may not evolve to Node's level for years - and worst of all, if you want to build competitive web apps, you still can't avoid Javascript.
running it today tells us that he's still number one:
I wish more people realized this. Generators seem to be billed (ironically by TJ amongst others with his Koa project) as the solution to callback hell but, having tried Go, I got the same felling that actors are much easier to reason about and debug (although concurrency is still hard).
On the client side, web workers allow parallelism but the API is inconvenient to use: you need to have a file for each web worker whereas Go's `go` routines are akin to a function call. In addition to this standardized conundrum, you have browsers discrepancies with the APIs available inside a worker varying between vendors [1].
On node, you have the cluster API, which allows managing several processes so it's even further from light threads. On the bright side, most (all?) APIs have both a sync and async version.
As a result, there's nothing you can use for multithreading in a module without coordinating with the whole project. I think JavaScript needs language support for light threads, not APIs for multiprocessing.
[1]: https://developer.mozilla.org/en-US/docs/Web/API/Worker/Func...
I personnally dont like generators for concurrency.It's the wrong solution to the problem.For now I stick with promises,and I hope JS gets async/await keywords quickly.
Agreed, but what I meant is that the API (running a file in a worker) is inconvenient, first because there is no closure. Go allows the go routine to read variables in scope and has a race detector to tell you when your accesses are unsafe. There's also the issue of managing workers: is a web worker an OS thread or a light thread [1]? how many can you spawn? which worker should you send work to? These problems are solved by Go. I think it would be very hard to obtain the same reliably and performance with today's web workers. Other languages solve this with a language construct, not an API to run another file.
> I hope JS gets async/await keywords quickly
It would be a step up. But most of the time you want to express a linear list of tasks. Javascript should support this by having a synchronous variant of its APIs and having light threads.
My hope is more that some language having such features will compile to asm.js
[1]: it's either a thread, a process or an "equivalent construct" http://dev.w3.org/html5/workers/#processing-model
If you consider that we have PNaCl compiling llvm-bitcode to native within the browser, Emscripten cross compiling llvm-bitcode to asm.js (which Firefox's engine converts to native), Safari's engine compiling js itself to llvm-bitcode, I'm thinking why not make llvm-bitcode an open web standard? That way, any language that compiles to llvm-bitcode would be equally usable on the server side and on the client side.
The biggest potential downside to that would probably be educational. I learn a lot by looking at source code and the web was always open in that way, though with minified JS these days that's not so possible any more. However, it might make it easier to distribute source code in any language, since a module can then compile and run such code right in the browser.
edit: These thoughts came up when I was thinking about how TJ won't be able to take his work with him as he moves to Go. Well, I suppose he wants to do new kind of work for which Go is better suited, but is then all of the prior work just a brief view along the journey?
Additionally we are now past Rails wave, Node.js wave and into Go wave.
And the narrative is similar as well: starting a new project with the shiny new thing is cool but maintaining it is boring and awful and to quote the OP "I need maintainers!".
Part of the problem is the language of course. Maintaining even a medium sized codebase written in a dynamic language is a challenge. But the other part is the mantra that "coding is easy" and "everyone should do it". And now there's an entire generation of developers whose idea of software development is "writing an app in Node in a weekend". Which of course doesn't require years of maintenance, complicated tooling or even adherence to common best practice.
Same here, but Go has productive libraries that can help you hit the ground running. Just with the packaged standard libraries, you can do things like have a useful webserver going. I prefer Rust though and might stick with JS until I exhaust it .. or until Rust matures enough to put into production and then jump fully on to Rust. Rust compiles to llvm-bitcode, unlike Go, so that's a win for me.
edit: I stand corrected - https://github.com/go-llvm/llgo
We use it at work for both packaging and deployment and it's awesome!
https://twitter.com/tjholowaychuk/status/16988926506
He tried to do a lot many things single-handedly. Which is why he needs a lot of maintainers. I don't blame him for overloading and stressing out. That is the way how it goes in node.js. Working in node.js is like battling a multi-headed beast. The core javascript keeps evolving, node.js isn't even in version 1.0, API changes a lot. There are lot of blanks to fill and too much time goes into boilerplate stuff and managing existing code.
It is just not rewarding and fulfilling to contribute to node.js. On top of that he faced a lot of friction from other contributors. He had some really difficult times, like express-connect conflict, bower-components conflict. Having handled conflicts too many, some people also consider him as being rude and too self-centered. Besides he is young and is probably one of the first things he has done he is serious about.
Many times before I asked to myself how does this guy do it. In a community where to contribute even a little one has to put in a lot of effort, his contributions are gigantic. I hope he continues to work, although not at too many projects . His experiences will definitely be handy.
The core node community was very unreceptive to change since the very beginning, to be honest Ryan and Isaac were some of the few humble people in the bunch. I've always been a "threat" to "the node way" of doing things, whatever that means. The elitism gets a little old.