Same reason most infrastructure as a service companies exist.
Setting up, maintaining, and scaling web socket messaging servers isn't my core business. Accordingly I am happy to pay a reasonable fee to somebody like Pusher to have the problem go away.
If real-time messaging is core/fundamental to your business, then you probably should run your own servers. Otherwise let somebody else deal with that bit of complexity and focus on the parts of your business that matter most.
It's cheap, it's already working in my project, it has a nice test stub, automatic reconnections, flash fallback for older browsers, support is responsive and effective, even when it's my fault.
The many toy pusher clones need to worry about the 90% of a project that is the polish before pusher will go away.
I also built my own Pusher compatible clone, https://github.com/danbeaulieu/PushPlay which is built on the Play! framework. Not recommended for production deployments, yet.
There is nothing free about managing and supporting a complex infrastructure component.
When you're building out an already sophisticated application, and your core user value proposition has nothing to do with providing subscribed socket connections then it is suddenly a real joy to pay real money to a competent vendor that takes responsibility for something that is hard to get right.
Tech startups are 10% software and 90% running a business. Pusher helps me run my business.
As an addendum, my investors and board appreciate this thinking. "I can do all of this myself" is a common ego trap that precedes cowboy mentality. Running a socket stack sounds like fun until something goes wrong at 3am.
This took 5 hours of coding and it's been running 2 months without problems, where is the complexity of it??
I agree your point when you talk about really complex stuff, but something that be done in that amount time doesn't worth to be outsourced in my opinion.
Also it's good to have alternatives, that's how we make things better.
how many messages are you sending, to how many clients? likewise, how are you running a public node server for free? how many of these servers are you running? what if... etc, etc.
not dismissing your library, but sounds like a case of "i don't know what i don't know". pusher exists as a service for a reason. there's a bit more to it than pushing json around.
Pusher costs ~$49/month, my time is worth $1000/hour [1]
It took me about 5 minutes to setup Pusher and for that I get private channels, presence, someone to handle the scaling etc... as djb_hackernews already mentioned.
We outsource DNS, email, hosting infrastructure, anything I possibly can which isn't core to the product itself.
The question shouldn't be why outsource this, the question is why would it be worth our time writing and supporting it ourselves when we can be working on other things?
Maybe we'll insource this stuff when we're at a point where we hit the 'enterprise' plan on Pusher, when we have a few more developers and a dedicated sys-admin. I'll cross that bride then, in the meantime I'm happy to have a $49/m hit on my credit card and move onto the next thing!
All that aside, it's great to have options so thanks for releasing this!
Your time really isn't worth $1000/hour. If everyone starts pretending it is it'd be silly to write a single line of code yourself, why would you when you're worth $7-10,000 a day?
I still do understand your point, but this is Hacker News after all, not Outsourcing News. This thread reads a little like a couple of suits just came in here to stomp on a hacker's opensource project.
I mean replace the words 'subscribed socket connections' in peterforde's original comment with 'web server' and suddenly it looks odd to be paying $19 or $50 a month for something like this in this day and age. Where would we be if there were only IIS out there to use? Or no MySQL/Postgres?
I've not used websockets in anger yet and I know it's still kinda ropey, but this is basic programming stuff, paying someone else for it is a bit, well, raise eyebrows time. SMTP is hard because of the configuration problems and dealing with a ton of arcane and opaque problems with major email providers.
This is something that's actually fun to muck around with. In the end the solution of Pusher is only marginally less complicated than the original problem.
At the moment maybe Pusher's a good solution for a fast moving business to a problem that's not well solved in the open source stack, but it's only a matter of time till there's the equivalent of apache or nginx.
And this is perhaps the first step. So kudos to daraosn for putting it out there, also seems the perfect kind of problem to be using node.js for.
I was going to put a smiley after that first line as it was kind of tongue in cheek, but HN isn't the place for smilies!
People pay Heroku/AWS each month, I don't see any difference with outsourcing this part too. That said, we use Linode boxes because I'm ok with configuring nginx etc... and I don't feel comfortable using IndexTank or MongoHQ etc... for outsourcing our search or database components, so I certainly sympathise with the unease at outsourcing key infrastructure which is easy to manage.
Pusher is marginally more complicated, but means I can call the problem solved and move onto other things. I agree it's a very interesting and fun area so it's certainly something I want to play with more in the future, but there are also 100 other interesting things which I need to work on, so I'll tackle those first.
Definite kudos to daraosn, just because I use Pusher because it solves my problem doesn't mean I don't think it's an awesome thing to work on as an open source project.
Wait, there's a difference between concentrating on tasks that you reasonably expect to yield $1000 an hour, and spending $1000 here and now to save an hour.
If there were two services, say an email support answering service, one requiring $50 a month and 2 hours of your time, and one requiring $1000 a month and 1 hour of your time, are you really saying you'd choose option #2?
I don't think that's in the spirit of the original blog post. It stipulated the example current, risk free value of time as $150, which is probably more accurate, and that would be the figure you'd use to make decisions like this.
In other words, spend an hour to save $150 today, or spend an hour to possibly make $1000 two years from now.
It took me about 5 minutes to setup Pusher and for that I get private channels, presence, someone to handle the scaling etc... as djb_hackernews already mentioned.
We outsource DNS, email, hosting infrastructure, anything I possibly can which isn't core to the product itself.
The other side of the coin is that you now depend on dozen of external services that could go out of business one day or another.
Note that I'm not saying that what you're doing is wrong, but that everything should be pondered
That's certainly something to be considered when outsourcing key parts of infrastructure yes.
I know the Pusher guys personally, so I'm confident they'll be around for the foreseeable future and I've integrated it into our code base in such a way that moving to someone else, or using our own server would be trivial.
It's something which needs to be evaluated on a case by case basis, I'm happy that all the components we outsource are both trivial to replace/insource and solid businesses who will be here for some time.
And I need something that is going to last 2 years without problems. Sorry but non-differentiated functionality like sending emails, push, analytics etc is exactly the sort of thing to outsource.
Not yet... I shared this for two reasons, because is an open-source project and so anyone can contribute to implement what is missing. Thanks, will include your list in the README.
There seem to be a lot of people here who are wondering why you bothered doing this instead of buying/renting/paying for some other solution. I'd suggest quietly acknowledging their points, and then going back to working on cool things.
Be an engineer. Be a problem-solver. Be a thinker. Don't worry about some magical valuation of your time and how learning to roll-your-own is just cowboy development--that's bullshit the suits will feed you to try and make you reliant on their capital. Do the hard things, and you'll go far.
(oh, and look into em-websockets...some cool stuff there if you don't want to mess with a full Rails deployment)
I fail to understand how delegation is not problem solving, but I know that it is most certainly smart business. Engineers need to manage their time and their priorities, and building a socket queue is a solved problem if you let it be. Why not spend that energy on UX?
About UX: that's an issue for the designers, yeah? I imagine the wheel-reinvention should be left to your engineers.
In general, I'd agree with you delegation is a good idea, but if you look at the site for the OP, it's a hackathon site. My assumption is that daraosn is more on the dev side of the dev-biz spectrum.
It's really counterproductive and harmful to the dev ecosystem to keep pointing out that whatever we're working on, well, it already exists, and we just have to pay for it and move on. That's just my opinion, mind you, but I hope some share it.
At the end of the day, we already have operating systems. And web stacks. And programming languages. Anything you pick, anything at all you use for your business, has probably been done before and probably better. We stand on the shoulders of giants.
That said, it's important sometimes to reinvent the wheel. It's important to scratch tech itches, even if your biz guys are breathing down your neck and trying to convince you that you're wasting time and investor money.
The big problem I have, as a developer, is that there are a lot of paid services to solve problems. But, sometimes, they don't work, or they have bad APIs, or they get acquired and go away.
Moreover, every time I outsource part of my stack, I'm reinforcing to the biz guys "Hey, us devs? Yeah, we're just cogs. Find somebody else to replace my sliver of logic integrating all these off-the-shelf APIs. I'm not really that important."
How much progress in tech has been made by people reinventing the wheel?
This[1] was the most fun I've had in ages. MongoDB already exists, but this was just a blast to write and is great to use. I agree with both you and peterforde, you guys are really arguing about orthogonal issues.
Kudos for sharing this and showing how easy it is to use node.js and socket.io (for those who haven't tried out socket.io, you really should) I implemented something similar for my side project recently, but instead of using an HTTP end point, I used redis and it's in built pub/sub layer. Pushing events is then a simple matter of publishing messages on redis channels. My node.js code listens to these redis channels and pushes the events upstream to the browsers via websockets, exactly as you do.
Another thing I'd suggest you to highlight (or rather warn) is your use of eval. In the use cases you've mentioned (message board or chat room), you're essentially going to eval user input. Any attacker who knows this will most definitely try to perform XSS.
34 comments
[ 6.1 ms ] story [ 90.9 ms ] threadSetting up, maintaining, and scaling web socket messaging servers isn't my core business. Accordingly I am happy to pay a reasonable fee to somebody like Pusher to have the problem go away.
If real-time messaging is core/fundamental to your business, then you probably should run your own servers. Otherwise let somebody else deal with that bit of complexity and focus on the parts of your business that matter most.
The many toy pusher clones need to worry about the 90% of a project that is the polish before pusher will go away.
https://github.com/stevegraham/slanger
It works with most (if not all) pusher libs and is rather convenient (except the "run on your own infrastructure" part).
When you're building out an already sophisticated application, and your core user value proposition has nothing to do with providing subscribed socket connections then it is suddenly a real joy to pay real money to a competent vendor that takes responsibility for something that is hard to get right.
Tech startups are 10% software and 90% running a business. Pusher helps me run my business.
I agree your point when you talk about really complex stuff, but something that be done in that amount time doesn't worth to be outsourced in my opinion.
Also it's good to have alternatives, that's how we make things better.
not dismissing your library, but sounds like a case of "i don't know what i don't know". pusher exists as a service for a reason. there's a bit more to it than pushing json around.
It took me about 5 minutes to setup Pusher and for that I get private channels, presence, someone to handle the scaling etc... as djb_hackernews already mentioned.
We outsource DNS, email, hosting infrastructure, anything I possibly can which isn't core to the product itself.
The question shouldn't be why outsource this, the question is why would it be worth our time writing and supporting it ourselves when we can be working on other things?
Maybe we'll insource this stuff when we're at a point where we hit the 'enterprise' plan on Pusher, when we have a few more developers and a dedicated sys-admin. I'll cross that bride then, in the meantime I'm happy to have a $49/m hit on my credit card and move onto the next thing!
All that aside, it's great to have options so thanks for releasing this!
[1] - http://blog.asmartbear.com/value-time.html
I still do understand your point, but this is Hacker News after all, not Outsourcing News. This thread reads a little like a couple of suits just came in here to stomp on a hacker's opensource project.
I mean replace the words 'subscribed socket connections' in peterforde's original comment with 'web server' and suddenly it looks odd to be paying $19 or $50 a month for something like this in this day and age. Where would we be if there were only IIS out there to use? Or no MySQL/Postgres?
I've not used websockets in anger yet and I know it's still kinda ropey, but this is basic programming stuff, paying someone else for it is a bit, well, raise eyebrows time. SMTP is hard because of the configuration problems and dealing with a ton of arcane and opaque problems with major email providers.
This is something that's actually fun to muck around with. In the end the solution of Pusher is only marginally less complicated than the original problem.
At the moment maybe Pusher's a good solution for a fast moving business to a problem that's not well solved in the open source stack, but it's only a matter of time till there's the equivalent of apache or nginx.
And this is perhaps the first step. So kudos to daraosn for putting it out there, also seems the perfect kind of problem to be using node.js for.
People pay Heroku/AWS each month, I don't see any difference with outsourcing this part too. That said, we use Linode boxes because I'm ok with configuring nginx etc... and I don't feel comfortable using IndexTank or MongoHQ etc... for outsourcing our search or database components, so I certainly sympathise with the unease at outsourcing key infrastructure which is easy to manage.
Pusher is marginally more complicated, but means I can call the problem solved and move onto other things. I agree it's a very interesting and fun area so it's certainly something I want to play with more in the future, but there are also 100 other interesting things which I need to work on, so I'll tackle those first.
Definite kudos to daraosn, just because I use Pusher because it solves my problem doesn't mean I don't think it's an awesome thing to work on as an open source project.
If there were two services, say an email support answering service, one requiring $50 a month and 2 hours of your time, and one requiring $1000 a month and 1 hour of your time, are you really saying you'd choose option #2?
I don't think that's in the spirit of the original blog post. It stipulated the example current, risk free value of time as $150, which is probably more accurate, and that would be the figure you'd use to make decisions like this.
In other words, spend an hour to save $150 today, or spend an hour to possibly make $1000 two years from now.
(But I agree with the rest of your post, FWIW)
This is such a great point. Because there is a world of difference between saying your time is worth $1k/hr, and walking the talk, as the phrase goes.
The other side of the coin is that you now depend on dozen of external services that could go out of business one day or another. Note that I'm not saying that what you're doing is wrong, but that everything should be pondered
I know the Pusher guys personally, so I'm confident they'll be around for the foreseeable future and I've integrated it into our code base in such a way that moving to someone else, or using our own server would be trivial.
It's something which needs to be evaluated on a case by case basis, I'm happy that all the components we outsource are both trivial to replace/insource and solid businesses who will be here for some time.
And I need something that is going to last 2 years without problems. Sorry but non-differentiated functionality like sending emails, push, analytics etc is exactly the sort of thing to outsource.
Now.js DOES make things simpler.
To be fair, you should list the things it can't do that Pusher can before you advise people to forget it. It looks like node-puller:
* Doesn't scale
* Doesn't have authentication
* Doesn't have private channels
* Can't do presence
There seem to be a lot of people here who are wondering why you bothered doing this instead of buying/renting/paying for some other solution. I'd suggest quietly acknowledging their points, and then going back to working on cool things.
Be an engineer. Be a problem-solver. Be a thinker. Don't worry about some magical valuation of your time and how learning to roll-your-own is just cowboy development--that's bullshit the suits will feed you to try and make you reliant on their capital. Do the hard things, and you'll go far.
(oh, and look into em-websockets...some cool stuff there if you don't want to mess with a full Rails deployment)
https://vimeo.com/4031833
I fail to understand how delegation is not problem solving, but I know that it is most certainly smart business. Engineers need to manage their time and their priorities, and building a socket queue is a solved problem if you let it be. Why not spend that energy on UX?
About UX: that's an issue for the designers, yeah? I imagine the wheel-reinvention should be left to your engineers.
In general, I'd agree with you delegation is a good idea, but if you look at the site for the OP, it's a hackathon site. My assumption is that daraosn is more on the dev side of the dev-biz spectrum.
It's really counterproductive and harmful to the dev ecosystem to keep pointing out that whatever we're working on, well, it already exists, and we just have to pay for it and move on. That's just my opinion, mind you, but I hope some share it.
At the end of the day, we already have operating systems. And web stacks. And programming languages. Anything you pick, anything at all you use for your business, has probably been done before and probably better. We stand on the shoulders of giants.
That said, it's important sometimes to reinvent the wheel. It's important to scratch tech itches, even if your biz guys are breathing down your neck and trying to convince you that you're wasting time and investor money.
The big problem I have, as a developer, is that there are a lot of paid services to solve problems. But, sometimes, they don't work, or they have bad APIs, or they get acquired and go away.
Moreover, every time I outsource part of my stack, I'm reinforcing to the biz guys "Hey, us devs? Yeah, we're just cogs. Find somebody else to replace my sliver of logic integrating all these off-the-shelf APIs. I'm not really that important."
How much progress in tech has been made by people reinventing the wheel?
[1] https://github.com/stochastic-technologies/goatfish
Another thing I'd suggest you to highlight (or rather warn) is your use of eval. In the use cases you've mentioned (message board or chat room), you're essentially going to eval user input. Any attacker who knows this will most definitely try to perform XSS.