Keep in mind with more Unicorn workers per box you will need fewer dynos overall so it does not necessarily mean you will have to pay double in the long run. At 1x Dynos we could only have 2 workers going but at 2x we could squeeze five workers in so we use half of the dynos we did before.
It should save you even more than you describe, as you shouldn't need to over provision as much. The odds of two 95% percentile requests stalling your 1x dyno is pretty good. The odds of getting 5 such requests is much lower.
Same here. We were struggling to fit 2 unicorn workers on a 1x dyno but can easily get 4 on a 2x. We're paying the same amount for twice the concurrency. The 2x dynos with unicorn have pretty much solved the queuing issues for us.
Wait up... I'm a bit confused here. How is it that you were struggling with 2 unicorns on a 1x (512mb) dyno, but 4 unicorns works on a 2x (1024mb) dyno?
I'm not sure I understand the pricing. I get you got twice the amount of memory compared to a 1X dyno. But still the rest of the resources is the same. For instance they will share a single queue (which is the point, no problem here), so they don't have really any added complexity or new resources. But the price is doubled.
"2X dynos contain twice the capacity. Memory and CPU shares. It's a new option to experiment with vertical scaling: you may get better performance from one 2X dyno than two 1X's. In that case you'll get more value. As explained in the post whether or not that's the case depends on the app. If you find that scaling 1X dynos works better, you are free to keep these instead."
Yeah, I had the same thought when the discussion came around last time. I actually took the source to the simulations (thanks for sharing, rapgenius!) and did some experiments on how random routing performs if it's routing to backends that can handle various #s of concurrent requests.
I think that is the wrong conclusion. You still have the problem of having to design a system where the dynos communicate their queues to the second layer of the routing mesh.
The solution is to have dynos large enough that you can run 8+ workers and let the operating system do all the complex scheduling.
> and let the operating system do all the complex scheduling.
So Heroku could spawn workers with a $FD environment variable instead of $PORT and the "complex scheduling" done by the OS _is_ the second routing layer.
But really, they could still do a second level routing even outside a single OS as the scale of the distribution is much smaller, so having the routing mesh be aware of the worker availability seems feasible again.
The first dyno is free, but it doesn't stay booted. After a period of inactivity, Heroku shuts it down if you don't have any paid dynos. When a new request comes in, it take a few seconds for the dyno to boot back up. For users that have little traffic, the dynos get shut down and your requests are very slow when they happen.
Heroku has good reason for this. The number of test apps that people create to try the system and then never shut down is probably quite high. They don't want to be running 10,000x512MB worth of apps that aren't serving any traffic. However, that also means that if you have a low-traffic (but real app) running on their system, the dyno is likely to get shut down during the times where you have no traffic.
A 1/2 size dyno would allow people to have a cheaper dyno that wouldn't be shut down.
At one point in time, you could add the free version of NewRelic along with a scheduler task to keep your free dyno active: https://coderwall.com/p/u0x3nw I haven't tested that... so things might be different now.
As we've seen from a screenshot leak on HN, Heroku is working on a cheaper tier of service that would presumably meet whatever need requires more than one but less than two dynos. I believe the price was something like $12 or $18 per month for one of these units.
For me Heroku is only usable as free "push-test-show-delete" demo server/testbed. It's either free or 35 bucks per month, and for $35/mo one can get quite powerful VPS.
I agree that thinking about memory use steps a bit closer to the world of server administration, which doesn't thrill me. We've actually had 2X dynos in alpha for close to a year, but we never pushed forward with a public release precisely for this reason.
However, the demand from customers — especially apps on the JVM — has been overwhelming. There's no way around it: once you start doing heavy background processing (e.g. geospacial or image processing) or web concurrency with a memory-intensive language like Ruby, then you care about memory. It would be very head-in-the-sand for us to not serve this need.
I'd love to imagine that there's some design that allows us to abstract away memory, but despite a lot of attempts, we haven't seen one. Further, memory is generally the most expensive part of server-side computing resources these days, so decoupling memory use from price isn't very feasible.
One point that I haven't seen raised yet is that a reduction in the # of dynos required to operate a service would provide the benefit of being able to run certain addons that are priced on a per dyno basis more cheaply (e.g. NewRelic).
I've been told they're encouraging add-ons to move away from a per-dyno price. (Looked like New Relic is the only one right now).
The guys I talked to cited both the future (now present) "dyno is not a unit" problem, as well as reporting user dissatisfaction with non-flat-rate billing.
"Do single 2X dyno apps idle?
Yes. Idling affects single 1X and 2X dyno apps alike."
I genuinely do not understand this. I'm still paying for it. I run a node.js app for my school that gets decent traffic, but not enough to warrant adding another dyno (and moreover I only wrote it to work on one process). Yet I still deal with the idling problem.
I would gladly pay Heroku the money for a 2X dyno if it meant I would stop getting idled (which is extremely annoying and disruptive), or even just some express per-month fee to disable idling.
I feel like I'm being punished because my app is performant and doesn't need two or more dynos...
Am I crazy here, or does this really bug anyone else?
But there is insufficient demand to warrant a change?
But it would be technologically too demanding?
But we don't think its a problem to wait 6+ seconds?
But we will address this later as its not a priority?
Its great that you are aware of the issue, but it would be better if you could shed some light on why or why not this is not addressed.
But there is insufficient demand to warrant a change?
No
But it would be technologically too demanding?
No
But we don't think its a problem to wait 6+ seconds?
Actually, if you're not paying us for service, we think this is completely acceptable. The only deficiency is where users would like to pay us but still want to have only a single dyno.
But we will address this later as its not a priority?
Sort of, see below.
Its great that you are aware of the issue, but it would be better if you could shed some light on why or why not this is not addressed.
Focus comes at a premium. There are only so many product initiatives that can get complete focus at one time without sacrificing on quality. We have a lot of things in the works, but hopefully this one will get the attention it deserves soon.
Thanks for shedding some light onto your thought process about this. I can definitely understand it a bit better now.
I guess I feel differently when you say "if you're not paying us for the service", since I pay ~$50/mo in addons through Heroku, of which which I'm sure you must take a cut. So I am in fact paying you, just not in the form of dynos.
Perhaps this could be a loophole to the current system?
There's an easy solution to that. Just install the free version of New Relic add-on. In there, you'll find an option to periodically check availability. That will trigger a request to the URL you provide every 30 seconds, keeping the dyno alive at all times.
While that's true, I just really don't understand why I have to resort to sucks hacks to simply eliminate idling. I am a paying Heroku customer with a CC on file... I just want to give them money to do something properly which I could hack around legitimately.
The fact that this has clearly been brought to their attention before and they have offered no real answers (I'm literally suggesting a checkbox for $10 a month or something that simply keeps my app un-idled) frankly gets me thinking about ulterior motives. I love Heroku, and find their services very valuable, but some of their decisions make very little sense to me, like this, or more recently how they've handled the RapGenius accusations.
I totally agree that hacks to avoid idling when are willing and able to pay is lame. We've actually been working on something very similar to what you describe. Send me an email at adam at heroku dot com and I can give you early access.
You can set up New Relic, whose monitoring will ping the site frequently enough that it won't idle, or if that's not an option you can set up a cron job yourself to do the same thing.
You can also consider moving elsewhere -- I'm paying $9.50/month to webfaction to host my app. While it took about a half hour to set everything up, (In my case rails, postgres, etc) I use Capistrano to deploy just as easily/quickly as Heroku.
44 comments
[ 7.1 ms ] story [ 1178 ms ] thread"2X dynos contain twice the capacity. Memory and CPU shares. It's a new option to experiment with vertical scaling: you may get better performance from one 2X dyno than two 1X's. In that case you'll get more value. As explained in the post whether or not that's the case depends on the app. If you find that scaling 1X dynos works better, you are free to keep these instead."
Here's the writeup: https://www.appneta.com/2013/02/21/the-taming-of-the-queue-m...
(Spoilers: turns out it gets a lot better very quickly.)
The solution is to have dynos large enough that you can run 8+ workers and let the operating system do all the complex scheduling.
So Heroku could spawn workers with a $FD environment variable instead of $PORT and the "complex scheduling" done by the OS _is_ the second routing layer.
But really, they could still do a second level routing even outside a single OS as the scale of the distribution is much smaller, so having the routing mesh be aware of the worker availability seems feasible again.
Heroku has good reason for this. The number of test apps that people create to try the system and then never shut down is probably quite high. They don't want to be running 10,000x512MB worth of apps that aren't serving any traffic. However, that also means that if you have a low-traffic (but real app) running on their system, the dyno is likely to get shut down during the times where you have no traffic.
A 1/2 size dyno would allow people to have a cheaper dyno that wouldn't be shut down.
For me Heroku is only usable as free "push-test-show-delete" demo server/testbed. It's either free or 35 bucks per month, and for $35/mo one can get quite powerful VPS.
"Get up and running in minutes, and deploy instantly with git. Focus 100% on your code, and never think about servers, instances, or VMs again."
This is not the case anymore.
However, the demand from customers — especially apps on the JVM — has been overwhelming. There's no way around it: once you start doing heavy background processing (e.g. geospacial or image processing) or web concurrency with a memory-intensive language like Ruby, then you care about memory. It would be very head-in-the-sand for us to not serve this need.
I'd love to imagine that there's some design that allows us to abstract away memory, but despite a lot of attempts, we haven't seen one. Further, memory is generally the most expensive part of server-side computing resources these days, so decoupling memory use from price isn't very feasible.
The guys I talked to cited both the future (now present) "dyno is not a unit" problem, as well as reporting user dissatisfaction with non-flat-rate billing.
I genuinely do not understand this. I'm still paying for it. I run a node.js app for my school that gets decent traffic, but not enough to warrant adding another dyno (and moreover I only wrote it to work on one process). Yet I still deal with the idling problem.
I would gladly pay Heroku the money for a 2X dyno if it meant I would stop getting idled (which is extremely annoying and disruptive), or even just some express per-month fee to disable idling.
I feel like I'm being punished because my app is performant and doesn't need two or more dynos...
Am I crazy here, or does this really bug anyone else?
Its great that you are aware of the issue, but it would be better if you could shed some light on why or why not this is not addressed.
No
But it would be technologically too demanding?
No
But we don't think its a problem to wait 6+ seconds?
Actually, if you're not paying us for service, we think this is completely acceptable. The only deficiency is where users would like to pay us but still want to have only a single dyno.
But we will address this later as its not a priority?
Sort of, see below.
Its great that you are aware of the issue, but it would be better if you could shed some light on why or why not this is not addressed.
Focus comes at a premium. There are only so many product initiatives that can get complete focus at one time without sacrificing on quality. We have a lot of things in the works, but hopefully this one will get the attention it deserves soon.
I guess I feel differently when you say "if you're not paying us for the service", since I pay ~$50/mo in addons through Heroku, of which which I'm sure you must take a cut. So I am in fact paying you, just not in the form of dynos.
Perhaps this could be a loophole to the current system?
The fact that this has clearly been brought to their attention before and they have offered no real answers (I'm literally suggesting a checkbox for $10 a month or something that simply keeps my app un-idled) frankly gets me thinking about ulterior motives. I love Heroku, and find their services very valuable, but some of their decisions make very little sense to me, like this, or more recently how they've handled the RapGenius accusations.
Certainly cheaper than paying for 2 dynos.