This looks promising. Trying to figure out how to benchmark this to test the "insanely fast" claim though. Any ideas? Maybe even add a tiny infographic on there?
Absolutely. Such a claim needs a serious backup in the domain of messaging, especially when the competition includes fast and reliable solutions such as Apollo or RabbitMQ.
So, not to be dense, but could you give an example of an application that would make good use of this library?
It feels very websocketish, but with some more infrastructure around it. Would people use this instead of something like socket.io, or are the 2 solving different problems?
I think it's more like Hook.io than Socket.io. One use could be server monitoring or deployment. You can also check out the Hook.io hook libraries to see how that has been used: https://github.com/hookio/hook.io/wiki/Hook.io-Libraries
i assume messenger works the same way, but the nature of hook is the processes are completely decoupled. essentially, you are just creating an event emitter that works cross-process. one benefit of this decoupling is it enables you to spin up and take down these processes without affecting the rest of the program.
I originally built node-monitor (https://github.com/franklovecchio/node-monitor/tree/v0.5) to monitor multiple servers; I was trying to emulate Splunk by sending all data from an environment to a couple servers, which then pushed out to websockets (live log viewing) + POSTed to CloudWatch. Ended up throwing out the socket piece because it was a pain in the ass to manage.
On a side note, you can scale what would be single-instance architecture with something like messenger (think an MQtt.js broker).
While it's nice to see projects such as this, it's somewhat of a pity that most of such solutions are specific to a single language, runtime or environment. I recently started working on something similar, but which is supposed to glue code from any language or environment together (somewhat like AMPQ etc, but on a slightly lower level, I guess...): http://dev.yorhel.nl/doc/commvis
One of the things that RabbitMQ does which this doesn't seems to be much more support for variable topography. This seems to need to know of all receivers when you create a "speaker" which in dynamic networks doesn't seem like such a good idea. Of course you can have each receiver register with some sort of registry and have speakers interrogate that, but then you're back with a single point of (logical) failure.
Discovery and dynamic usage doesn't seem to be covered in many of these projects (cool as they are) which seems to me to be a more challenging and useful extension.
It's a very good point. I plan to add automatic discovery where every speaker and listener maintains a current list of the communication topography.
I figure I'll use this space to explain "why" messenger.js even exists.
One reason I wrote messenger.js is because I find it easy to install, work with, and deploy over using RabbitMQ in node. Personally I was using zeromq with node-zeromq.
This means, I need to install zmq, install the node module for it, and anything else that it may need to work properly. In addition I have to do this on dev and production.
On top of that, I also have to write an abstraction over it similar to how the messenger.js listener works right now. To me, there seems to be too many moving parts to the puzzle just to get some simple communication going.
I fell in love with the DNode API at one point, but I have to start my servers first and clients second. It doesn't appear to be a way to capture error events and try to auto-reconnect. There is a ticket for it but it seems neglected. For me, I don't want to worry about that. I want that abstracted away. I also wanted it in TCP, not HTTP.
Of course, the benefit of RabbitMQ is in a separate language, someone probably already wrote the adapter for you to communicate with services of different languages.
I'm not trying to compete with RabbitMQ here though. Just trying to get node.js communication to work in a scalable fashion with a 1 line install that just works.
Yeah that's fair enough - I'm working with RabbitMQ in Node at the moment, and I've got a simple domain specific wrapper around it, which I find fairly easy. Of course, not everything on it is in Node, which is one reason this wouldn't work for me - but if it works for you, that's great!
I'll be keeping an eye on it anyway - you never know when something like that might solve a problem you encounter.
I will put them up. In theory it should be faster than something like DNode or anything that uses DNode (hook.io) because it doesn't have the TCP tear down from using HTTP to carry what is probably small payloads most of the time.
Right now it's as fast as Node.js will allow TCP to be, if that makes any sense.
Benchmarks will come as soon as I get some free time to sit down and do them all. I'll plan to compare with zmq+nodezmq, rabbitmq+nodeamqp, dnode, hook.io. Any others?
Eh, correct, but you still pass unnecessary headers each time as part of the payload. It would even up the speed theoretically, but it's still a request reply model that demands a response, and you really don't need that for pub/sub for example.
TCP just feels cleaner as a protocol for this kind of stuff but I would certainly concede that speed of HTTP isn't as bad as people would make it out to be.
Also, I'm considering adding createFastSpeaker and createFastListener to the API for people like game makers who don't need to guarantee delivery of certain data over UDP.
similarly, check out hook-io by nodejitsu. i actually don't see the benefit of messenger.js at all in comparison. perhaps its simplicity, if that happens to be beneficial in your scenario.
The examples and output don't seem to be synchronized (for example in the pub/sub example). But more interesting, how can the client reach servers on a different network/ip? since that is the more common case for distributed servers...
27 comments
[ 2.4 ms ] story [ 56.6 ms ] threadIt feels very websocketish, but with some more infrastructure around it. Would people use this instead of something like socket.io, or are the 2 solving different problems?
Looks pretty cool, either way.
This seems like a simpler API to me.
I'm fairly new to working with Node, so any kind of context for use-cases around some of these libraries is always a help
On a side note, you can scale what would be single-instance architecture with something like messenger (think an MQtt.js broker).
They all communicate with each other, but in very different ways.
The scraper for example uses fire and forget to send something to the data access layer.
The web app cluster needs to use the request reply model with the data access layer.
The data access layer needs to send pub / sub messages to the stream clusters holding live connections.
I also have a mobile service which does a combination of pub / sub (notify stream servers), req / rep (data access), and fire / forget (analytics).
I needed one library with a 1 line install that handled all of these cases, with the simplest API I could come up with.
I certainly need to play with that to find out more. Thanks for the mention!
Discovery and dynamic usage doesn't seem to be covered in many of these projects (cool as they are) which seems to me to be a more challenging and useful extension.
I figure I'll use this space to explain "why" messenger.js even exists.
One reason I wrote messenger.js is because I find it easy to install, work with, and deploy over using RabbitMQ in node. Personally I was using zeromq with node-zeromq.
This means, I need to install zmq, install the node module for it, and anything else that it may need to work properly. In addition I have to do this on dev and production.
On top of that, I also have to write an abstraction over it similar to how the messenger.js listener works right now. To me, there seems to be too many moving parts to the puzzle just to get some simple communication going.
I fell in love with the DNode API at one point, but I have to start my servers first and clients second. It doesn't appear to be a way to capture error events and try to auto-reconnect. There is a ticket for it but it seems neglected. For me, I don't want to worry about that. I want that abstracted away. I also wanted it in TCP, not HTTP.
Of course, the benefit of RabbitMQ is in a separate language, someone probably already wrote the adapter for you to communicate with services of different languages.
I'm not trying to compete with RabbitMQ here though. Just trying to get node.js communication to work in a scalable fashion with a 1 line install that just works.
I'll be keeping an eye on it anyway - you never know when something like that might solve a problem you encounter.
Right now it's as fast as Node.js will allow TCP to be, if that makes any sense.
Benchmarks will come as soon as I get some free time to sit down and do them all. I'll plan to compare with zmq+nodezmq, rabbitmq+nodeamqp, dnode, hook.io. Any others?
TCP just feels cleaner as a protocol for this kind of stuff but I would certainly concede that speed of HTTP isn't as bad as people would make it out to be.
Also, I'm considering adding createFastSpeaker and createFastListener to the API for people like game makers who don't need to guarantee delivery of certain data over UDP.
on other server
messenger.createListener(2222, 2000)
You'll of course have to update your IP Tables.