The strikethroughs are for services I once used, but have since cancelled because I either moved to another service or no longer needed that service anymore. Early on, I paid for Code Climate, but realized it was an expensive non-necessity at the time. For services like Mailchimp, I moved my newsletter to Intercom once the spiked cost of Mailchimp didn't make sense to keep.
The decision to cancel Baremetrics is a long story, but the main reason is that I don't agree with a few tactics they've used to improve MRR, like removing the ability for users to cancel their account on their own (you now need to email them). I'd rather make a little less in order to improve the user experience and build a better relationship with users. I use ChartMogul now, which I really enjoy using.
Maybe https://liquidsky.singtel-labs.com/ can help you with that transition, especially when/if you need to iterate through your cloud environment repetitively until you get to a comfortable one
As far as I can tell, the cost of the SSL endpoint on Heroku is almost exactly the same as the cost of a separate Elastic Load balancer, which you need to terminate the SSL connection before it reaches your app. So Heroku is really just selling this feature at cost.
I don't remember additional charge for using SSL with ELB (but ELB is extremely dumb so if you want to apply fancy headers and rules, ELB is out of question, either roll out Nginx or HAProxy with ELB governing failover. Direct R53 failover only works with public zone).
It seems like Heroku's business model is to make it easy for startups to build on their platform and then bleed them dry of all their investment money.
The only potential problem is that Cloudflare doesn't check the validity of the herokuapp.com cert, so in theory someone could MITM the connection. In practice, I'm not sure how you'd even get Cloudflare to connect to the rogue proxy, short of taking control of the DNS of *.herokuapp.com.
That's not true, when you setup crypto in cloudflare you need pick between "Full SSL (Strict)" which requires a valid certificate, and "Full SSL (non-strict)" which allows you to use a self-signed certificate or what not, but there's no reason you should be using that mode if you already have a valid certificate (as is the heroku case).
"Full SSL (Strict)" doesn't work with the certificate provided for free by Heroku:
By default Heroku offers a wildcard SSL certificate which only covers
‘*.herokuapp.com’. This means that ‘Full SSL’ can be utilized as a default,
which does not require that the SAN contains your FQDN. To utilize
Full (Strict) you will need to add your own SSL certificate to your
Heroku app, which can be done by using their ‘SSL Endpoint’ add-on.
I think people are misunderstanding the SSL costs for Cusion.
The SSL certificate expense is listed as $99/year from namecheap, ie. $8.25/month, so there seems no need to get a Symantec one.
However they/it/he are paying Heroku $20/month for handling the SSL traffic, not the certificate itself, which probably includes running a load balancing server in front of their apps that automate the SSL termination across nodes. Which seems like a fair cost to me.
I think the description of this cost is the error which causes the confusion.
Hi! I run Cushion. Moving off Heroku is definitely high on my to-do list. So far, since I'm a solo dev, I've taken the mindset of paying for convenience services that would allow me to spend more time working on the app. Now, I'm getting into the phase where I can start reducing these costs and transition to a system that doesn’t induce fear when I need to add another server. Right now, I have two web workers and two background job workers. The one background job handles integrations with other services, so I wanted to keep that separate from other jobs, like delayed emails, downloading backups, etc.
Thanks so much for sharing this, I will probably use it as an example for a book about hosting costs that I'm trying to finish for far too long.
By the way, your use of Heroku makes total sense given that you're basically alone, I hope that the lock-in won't be too difficult to overcome when moving away from it.
Thanks for the reply. I think you're doing great. Keeping your focus on the core components of your product is probably the best thing you are doing for your business.
I also want to say that your site has a great design. Are you doing that yourself or would you mind sharing the service you're using?
Thank you for noticing this. The $175/month came from the intercom services. Given that support is likely his main selling point, I think $175 is perfectly fine for the added functionality.
One sure can have a DO droplet for a minimal rate, but unlike Heroku/AWS, they don't offer anything oriented to high availabilty, resilience, auto-scaling, performance (via workers), etc.
Exactly. In addition to what you listed, the ability to quickly roll-back to previous releases with a single click on Heroku has literally saved me from losing paying customers. I don't know what I would have done this past week where I had to do just that after a bad release.
Cloud services can work out super expensive for a startup. To get going often all you need is one server that runs both the database and web server, which you can rent cheaply from any number of hosting companies out there. Sure you have to manage it yourself and it won't scale right off the bat, but sometimes that's fine. This is how we ran Scirra for a long time. Eventually you need to scale up, but that's a nice problem to have!
Dividing your architecture into separate components is not only needed for operating at scale (the "nice" problem to have), but for having high availability (or even decent performance) at all.
Depending on one's business, HA/performance may not matter that much, but personally my startup is having a 'proper' design from day 1 - downtime is not acceptable for my business model.
I'm not disagreeing that in some cases it's true. But as someone who provides infrastructure for a great many businesses including startups, most everyone changes their tune when their server goes down.
That's entirely true. But maybe those startups which 'changed their tune' didn't have to worry about servers for N months, focusing in e.g. market fit and growth hacking instead. Lean Startup mode.
Personally I'm pretty confident about market-fit and expected growth even before the launch day of my startup.
Dividing your architecture into separate components will probably cause you more downtime and have much worse performance.
Less servers and components mean less things to break.
Grab a dedicated server and host your shit on it (have a DB replica because losing data sucks, but that is easy). You will be able to run like that for much longer than you probably suspect and the chance of downtime will be massively reduced.
Why? Because a single server has a very low chance of having anything going wrong.
Not only that, but there is a lot of overhead in performance when you split everything up into components. You will be surprised how many requests you can handle on a single inexpensive dedicated server when you haven't put too much effort into "architecture".
When you actually find yourself running out of headroom on that setup you will know much more about your application and it's requirements then you did at the start, so you will have a better idea of what to split out, and a better understanding of what parts of your app need more performance anyway.
Workers are a must for any minimally sophisticated web application. You cannot degrade HTTP response time (and whole app server performance) just because you are performing work synchronously before the request finishes.
Similarly, if the work you are performing has any value for your business, it must be retryable, and the queued items must be safely stored (i.e. don't use a single Redis server - use something distributed like SQS instead).
All of that calls for a 'proper' architecture. I'm not talking microservices or anything fancy - just a webapp/worker/db separation and the like. Which is exactly what dedicated server (or DigitalOcean) users tend to lack at early stages.
Yeah, that's true. We were traditional one-off software license sales via our website, so a brief outage would not be the end of the world. There are definitely cases where it would be irresponsible to only have one server, but I think there are still a lot of startups that could get by with it, and it would save them a lot on the early running costs, which could really make a difference.
Btw, love the front-page demo of your product. Open with childhood experience, cheap laugh with quarters we lost, prop up 2D today, and then drop the demo. Great, fun ad. Good luck on the business. :)
I definitely plan to write about it on the journal (http://cushionapp.com/journal). I have channels for new users, Stripe charges/subscriptions, errors from Sentry, build notifications, and support notifications. I plan to also build some Slack bots that would help automate a few things.
If that's all you're using it for, you'd be better of with Hipchat, which serves as a functional notification channel and from what I recall is free for <10 users. I guess Slack would be worthwhile if you're using it for multiple projects, though.
I can slash about 4000 of those dollars down to 600 by renting hardware to run those services on. Makes you responsible for your setup but if you are bootstrapping, you can find better first year uses for 3400 dollars I'm sure.
At the time, I was eager to build the app and focus on its features and getting it online. I don't know much about renting hardware, setting up a system, and scaling, so I was gladly willing to pay to make that easier on myself. The $3400 lost actually bought me a lot more of my own time, which I consider more valuable in the long run. I'd be interested in reading a blog post about how you'd slash the cost, if you ever end up writing one.
From the heroku costs, it looks like you are using one web dyno and 2 workers dyno. The database is on standard.0, which is the first base tier production level database they offer. There's no reason you couldn't host all these on one server on digital ocean. I don't really know what load your server is getting, but seeing as you only need one dyno, it's probably not much so one server should suffice.
---
# Pros of setting up your own host.
- Huge cost savings.
# Cons
- Heroku is way easier to scale, just drag the slider. Upgrading databases / redis is still a pain. I recommend going with RedisCloud instead of Heroku Redis. With database you have to do a migration and copy the data over.
- Having to deal with securing your own server and maintenance.
- High upfront cost due to setup
At the end of the day, it's about what your time is worth :). There's no right answer. I go with Heroku myself. The instant gratification is hard to beat.
Then it was a good decision for you. Time vs money should always be weighed carefully. I tell people: "If you have IT expertise, use it to keep efficiency. Otherwise, outsource it and focus on what you do best."
Even if you do have IT expertise, it's often more efficient to outsource it for your own projects (while selling your IT expertise to the highest bidder).
It's to be decided on a case by case basis. I used to deploy whole infrastructure outside the wiring in a day with little mgmt overhead. The trick was doing as much ahead of time as possible. Each success fed into next project.
That's why I recommended in a other comment using people that do stuff similar to what you need and split cost among several companies.
My tips don't even warrant a blogpost. CentOS + KVM then segment the bare metal into VMs to do the services. One machine hosts an nginx I port forward to from the host. Figuring out how to use iptables was the most time consuming bit.
Heroku is very, very expensive. We switched an early Discourse setup from Heroku to Digital Ocean (using our Docker setup, which is another reason why it exists) and it saved them probably $100 per month. Everything but the largest communities fit fine on a $20/month, 2 CPU, 2GB ram DO droplet.
No one is understanding that having a few extra hours a month to build features and find customers is worth a couple hundred bucks. It's probably worth thousands of dollars a month.
It also performed a lot better, if that helps. Granted for some orgs, even a $10k per month hosting bill might be no big deal. But we package Discourse for usage, and generally speaking less expensive means more people using it.
There is a point of diminishing returns, of course, pushing down to $5/month is a lot of pain -- super low cost super low resource hosting, etc.
This is an excellent breakdown, thanks. As others have pointed out you can make it cheaper here or there but I find it awesome that you can run your business without any dedicated IT resources a sysadmin/network/ops kind of person. Adding one employee might switch it from a nice side income to losing money.
Thanks! Fortunately, the app's usage is pretty casual—users check it a couple times a day, or sometimes week-to-week. Because of that, I haven't had too many issues with server load. Honestly, I could probably run it on a single server no problem, but it's good to have a backup just in case. I freelance alongside of this, so I'm trying to slowly make Cushion my full-time gig. Hopefully, I can make the final push this year.
I always find breakdowns like this interesting. Just to get a feel for what others are using, their reasoning, and to generalize what others are spending monthly/annually. With more breakdowns like this from more companies, I think it provides a great understanding of costs. It also can give people a good baseline for understanding what costs are at certain levels of users and with certain functionality.
While I understand that these numbers also give an opportunity for people to point out areas where there may be potential cost savings or opportunities to improve infrastructure, I am not sure generic comments of saying you are spending too much or too little, or I don't spend that much are beneficial. A few key pieces of information that would help correlate these numbers in my opinion are total users on each plan, monthly active users, and the average amount of data required for the users.
Just based off the Stripe numbers I would estimate that they have between 6000-7000 annual paying users (this number may actually be a little lower if the majority of users are paying monthly instead of annually). That does not include members that only trial the service, although I would suspect this number is small.
Long story short, every business has different requirements and usage. What works for one company with 6000 users may not work great for the another company with 6000 users. I just hate seeing people say that someone is spending too much without knowing all the details.
I think you're overstating it on mentioning alternatives. There's certainly factors that go into picking a particular provider that may justify the price or negate an alternative mentioned. So, I'm also against anyone mentioning them as a critique without context.
However, the flip side is that many services have alternatives that do about what they do. They might do so with a lower price or more capabilities. People mentioning those in comments of articles like this facilitates the discovery of those products for people not aware of them.
An example for me was arguing that much of what "cloud" stuff does can be done with dedicated hosting if it was setup correctly. That we just need a combo of the two to satisfy certain markets focused on security and dependability where sharing hardware isn't permitted but they want flexibility & low costs. Thanks to a commenter, I found out I had re-invented the concept of "bare metal hosting" and SoftLayer provided exactly what I was thinking about. I wouldn't have discovered that if everyone and I were just giving props to the OP or further elaborating their own use case.
So, I think exploring the reasons behind the choices of company in question and bringing up alternatives are beneficial. One facilitates understanding. One discovery.
I agree and would add that it's easy to look at their spend and think that's a waste, but I think with many endeavors, an individual's time is usually the scarcest resource.
Web services expenses aren't even close to their cost of adding an additional employee to solve these problems in-house.
Learning to automate something a more seasoned *NIX veteran could do in an hour is not always a predictable time expense, and then they might create bugs or run into other issues. $100/month can a bargain and buy you time to do more important things.
Researching services to use can turn into a black hole. There is always a better service that you don't know about. It can be better to decide "I will spend one hour researching metrics/dashboard services, and pick the best one I find, and spend another testing out the service." rather than trying to end up with the most perfect solution and testing every solution there is.
That said, sometimes it's more time efficient to avoid having one extra service to maintain. One has to avoid using a service in a way that the business is too dependent on it, or that the service is not easily replaced. That's just part of what makes running a business hard.
That's a good point about individual's time being a resource. There are many things I am good at and there are many things I am not so good at. I would gladly pay a premium to a service to have them handle the things I am not good at, especially when just in the startup phase. So while one person may think that saving a buck here is an option, for me that buck may be the difference between being able to continue business, having to hire someone internally, or not having a business at all.
That's another good point about researching services can turn into a black hole. It is almost like the question of what framework/language should I use to build my startup.
Exactly, isn't that the reason why startups "need" so much money anyways? To get the best people to solve problems? In that case, it's a one off and you solve the issue. Definitely seems like the smarter business move to me too.
If you assume that all users are paying monthly (not unreasonable, the $2/month savings are tiny vs getting locked into $96/yr), then they only have about 600 users since they're charging each of them once a month (hence the 7k billing events).
It's a 20% discount for a time/budget management/forecast solution that I don't think makes much sense to stop using every other month -- so I'd be surprised if most that go from free trail don't convert to yearly subscription? (They could of course have 600 different users each month, and be failing due to big churn with their customers...)
The majority of paying users are monthly, but mostly because of the demographic—freelancers. Most freelancers aren't full-time freelancers, so they don't always freelance (and don't always need a freelancing tool). That makes churn the biggest challenge. I follow up with users who leave and have learned that they either don't have a lot of freelance gigs at the moment and plan to come back, or they took a full-time job.
Yes it is always great to learn what other people are doing.
I have no issue over the costs with paying for all these services as it is almost always cheaper than building/setting up the same tools yourself, the issue I have is the lack of skills/understanding that going down this route encourages. I like to build everything myself so I learn, not because this is the most efficient way to run things. When something breaks (it always does at some point) it means I can fix it.
There is also the other point which is I love learning about all this stuff :)
You are certainly not building "everything" yourself. You are drawing a line just like OP, that where you draw that line is different than theirs is not at all surprising.
I actual do “build" most thing - typing ./config && make && make install is not too hard, but I do think there is real value is building/installing your own tools beyond just the cost aspect.
As some have said, this seems expensive. However, I would point out that the costs of using Heroku should be outweighed by the convenience and ability to focus on features. You are getting what you pay for here, the velocity you gain from not worrying about your infrastructure.
I do think that when you are first launching a startup, you should probably do everything as cheaply as possible, which probably means using a VPS or IaaS. As soon as your infrastructure becomes more complicated, you can use Heroku instead of hiring people to manage it for a while. Then see where your growth takes you -- you will probably find that eventually it makes more sense to hire a couple people to manage some IaaS or even renting racks and buying hardware.
As with all things in life, it really depends. I believe this particular configuration makes sense for companies who want to focus on product, not infrastructure -- at a cost of expensive hosting.
Cost of getting one dedicated devops engineer would surpass Heroku budget by 10X. If devops engineer is far fetched as per scale of the company than count in number of hours put in by developers for initial up, upgrade, new emlpoyee on boarding tada tada would easily bet it too.
However I wish they would have disclosed traffic handled by Heroku servers as well.
Maybe if you hire them from Silicon Valley. There's plenty of skilled people in areas with low cost of living that can handle the job without 10x the pay of whoever. There should be at least one such person on the team anyway if the business is a web application. Plus, aren't there alternatives to services like Heroku these days that come as appliances or VM's with simple deployment? I'm seriously asking because I don't work on that stuff but swore such products pop up all the time.
Their total costs between Heroku and Redis infrastructure are less than $3000. I don't think you'd be able to hire a devops engineer for less than $30k, anywhere.
It's surprising to me that people think they're spending a lot on infrastructure when they spend $800 on fonts, etc. When you drop the payment processor charges (which are per transaction) they're at $7k to run a site with a lot of functionality for a year. Seems pretty reasonable.
You don't get an engineer in my model: you usually share one with others. Someone who throws together this stuff all the time usually has premade images, scripts, checklists, etc that make it very efficient. You become one of their many customers. This might be a person or a firm. It's just one that focuses on great cost-benefit as differentiator.
So, one $30-50k engineer gets split 5-10 ways on infrastructure. Most maintenance is automated with setup and occasionally fixing something using up most time. You usually negotiate 1 day to a week of dedicated time out of the month w/ split being flexible.
Heroku and Redis might still come out a better deal. It's just not going to be nearly as different as people think because one doesn't have to load-up on IT people to deploy or maintain common configurations. All kinds of consultants and smaller fish that will do it way cheaper [than you mentioned] with cost spread across multiple, small businesses.
This doesn't take into account the following transactions:
- The time and money it takes to find said $30-50k engineer
- The risk (and therefore potential cost) if the engineer doesn't do the job correctly
- The unknown cost of "occasionally fixing" could be $50 or $50,000. Given how nebolus the DevOps position is, you also can't be sure the stuff this individual is doing is considered best practice.
- There is overhead time spent on managing such a resource (teleconferencing, IM's, etc)
- There is potential that the resource is unavailable and therefore a potential downtime that could cause your business loss
There's a reason Heroku exists and has been successful, and these are just a few of them.
I like when people lay out all the obstacles people would face because I think the probability of creating an solution to this is > 0 and higher when people question models.
But this one i would like to ask some questions about:
"The unknown cost of "occasionally fixing" could be $50 or $50,000. Given how nebolus the DevOps position is, you also can't be sure the stuff this individual is doing is considered best practice."
Who gets to decide what's best practice? How does such information about such best practice propagate? How much does it cost to acquire such best practice knowledge? Is trying to mitigate for this all the time a theoretical constraint on all possible solutions (i.e. without trying to mitigate for this, things will not work at all) or for any of the issues you bring up that are probably valid assumptions for some set of solutions?
I'm curious - do you have first hand experience with companies that does this (and does it well)? I might want to work (at first part time, remote) for such an outfit, if I they provide a good service (and good/skilful colleagues).
I've seen a lot of people that hire devops in-house (technically, it's a little hard to see how "devops" might work with the "ops"-person just doing drive-by installation and maintenance once every month -- but I suppose if the "dev" part isn't "special" (web application server + database + static resources + database) one could argue there's not much need for "devops", just "ops"?).
Although 30k for full-time ops works sounds really low, anywhere? (As a customer, I'd probably prefer one tenth of a 100k/year engineer than one tenth of a 30k/year engineer... even if the price might be 3-4 times as high).
I used to do it myself and know people who do. None of us are or were in DevOps. Plenty of independents out there that might make same arrangement. We did it just to save us time and make extra money. :)
For JVM apps Boxfuse (https://boxfuse.com) gives you dead-easy deployment to plain AWS by generating minimal VM images that run unchanged both locally on VirtualBox and in the cloud. It follows the principles of immutable infrastructure (machines are disposable and never modified) and all updates are performed as zero-downtime blue/green deployments.
Now that is a bit closer to what I'm talking about. I appreciate the tip about this service as I know some Java people that might like it. You might find it interesting that these Java VM services are actually similar to what safety-critical, embedded industry has been doing for a while:
They run POSIX and safety-critical apps side-by-side on a microkernel (eg QNX, INTEGRITY) w/ latter usually on a built-for-purpose runtime like PERC. They also would combine app, dependencies, and runtime into one image that could go into a ROM. Services like Boxfuse are doing it for cloud but the model goes back over a decade with proven benefit. Goes back further in security kernels (eg Aesec's GEMSOS or BAE's XTS-400) where they did same thing for preventing sensitive components from leaking to ported POSIX/Linux apps. So, it goes back almost three decades now. That's how long it takes industry to learn apparently. ;)
EDIT: Wait, you're the founder rather than just a user. You might have heard of some of this advanced, clever Java stuff already then. Haha.
> As soon as your infrastructure becomes more complicated, you can use Heroku instead of hiring people to manage it for a while.
I would expect the other way around. In fact, "for a while" doesn't work in real life. So moving from AWS to Herkou then migrate to another IaaS? That's going to be really costly. My pitch is use the tool you are more comfortable with at the beginning. When you grow to a reasonable size, I would start creating infrastructure on AWS directly (or other IaaS providers). I work with AWS daily so if I were to start my business I'd build on AWS first (plus there is AWS startup credits that you can apply). YMMV.
"As soon as your infrastructure becomes more complicated, you can use Heroku instead of hiring people to manage it for a while"
Having a Iaas/Paas doesn't mean you don't have to manage anything - that's a myth. Sure, it might be doable if you own a trivial app with just 1 web server + 1 app server + 1 database server. But once you're doing anything remotely complex like building a 10-node Elasticsearch cluster that is running out of RAM, or a 10 Cassanda node cluster, or a Kafka queue that can overfill, you always need an engineer to manage your infrastructure.
Wouldn't call that true. There are hosted solutions for ES, Cass, and AWS has kafka alternatives. That's not to say no enginenering time will be spent, but certainly you can start without a full time infrastructure guy
Yes, you can start, but once your app matures and you rely on it for production, you definitely need an engineer that you can rely on if things blow up.
I know companies that don't rely on any hosted cloud services for our infrastructure, and have just 1-2 people they rely on to be that infrastructure guy, and even they are not 100% full-time. It's only when things blow up that they tell him to stop what he's currently doing, and address the issues. Unless you have 10K+ employees, you'll be hard pressed to find companies that have full-time engineers that just twiddle their thumbs, and stare at a monitoring dashboard 40 hours a week to monitor a component.
Yeah, fonts are expensive, and I wouldn't go near a free font I found on the web unless it was hosted by someone like Google. So if you don't like any of Google's free font options, you gotta pay.
Also, there is no cost for a designer in the pricing, so either that is a startup cost not included in the ongoing pricing, or is done in house. And a designer is not cheap.
It's a one time cost. The others are monthly costs.
Most likely they're paying for a redistributable licence which is much more expensive than other licences.
For instance, take a look at this font I'm currently integrating into the app I'm working on:
Correct—the font licenses were a one-time cost. Previously, I was using Typekit, but I hit an issue with the requests stalling for ~3 seconds. Apparently, Chrome has a threshold of not loading more than ~5 requests from a single external location at once, which resulted in the stall. I reached out to the font foundries and they were incredibly helpful in letting me try out the fonts for performance testing. When loaded from my own CDN, they loaded instantly. At the time, I had some extra cash from a freelance gig I just wrapped up, so I went for the font licenses. To me, as the designer/developer/one-man-team, it was important to have design I intended for Cushion along with good performance—and supporting the font designers who were so helpful.
I think the limit for Chrome is a bit higher - 6 or so resources from the same location will cause the stall. You can usually see this happening if you watch the "Resources" tab in Chrome Dev Tools and watch the timings for loading each resource.
I really dont understand. Im deploy rails webapps all the time with pretty large postgres backends.
You could totally spin up multiple digital ocean instances to get a much cheaper infrastructure. I usually use one for load balancing, 2+ for running the app, and usually one for the backend postgres. You could also get away with using something other than digital ocean for the postgres.
But for 6000 users, depending on the app. I can probably get away with like $30 a month with this setup. Including the database.
Total setup time might be a day the first time. But it scales pretty well, costs significantly less.
The hosting costs appear to be about $70 ($30 for the hosting and $50 for the DB). That doesn't seem excessive to me.
The other expenses appear to be various other services (slack, support site, logging service, etc). Lot's of 3rd part services to handle stuff.
Obviously it's a lot cheaper if you can do many of these things yourself (run your own support web app, use a free chat service, etc, etc). But we all have to decide what services we want to pay for vs run ourselves - in some cases it is cheaper to not have to spend the time!
This is the first time I have ever heard of Papertrail. I really hate loggly and am looking for a better option - can anyone shed any light on if this is a good option ?
Which log aggregation services is everyone using ?
We got frustrated with the UI on Loggly and wound up moving to Logentries (https://logentries.com/).
I've not used Papertrail, so can't compare to them, but it's significantly easier to use than Loggly, they've been kind to us through the odd overage, and it does what we need it to do (Centralise a load of application logs and allow us to search over them and set alerts). The live tail feature is nice as well.
Also have had great experiences with Logentries. Easy to integrate on every platform/OS, very configurable alerts, team has been pleasant and responsive to pull requests.
I'm using Papertrail and the experience so far is not so great. Dashboard UI looks like it wasn't updated since 2005. Also, the free plan is a joke: 10Mb of logs per day. When I started I managed to stay under this quota, but recently I've added a couple features that spits additional log data. Now I have to either filter some valuable logs or become a paying customer (which I can't afford for such thing as logging).
We used Papertrail for about 1 year. It works for fairly well for basic reporting, but it gets pretty expensive once you have enough volume. We moved to Logstash, Elasticsearch and Kibana. The power to aggregate, digest, and search the logs from various apps is quite awesome.
From the expense report, it says you have a wildcard SSL (and it seems to be in use in the subdomain for the signup page for example) yet your main root site seems to be unavailable via SSL?
Is there any reason for this or just an oversight?
The marketing site is static-hosted on S3 and I think when I first set that up it was either too expensive for SSL or not possible. I'm not 100% sure, but I'll definitely look into it again. The wildcard SSL was a recent addition after coming out of beta, which didn't have the multiple subdomains.
Hot tip for when you get around to setting it up - SSL on S3 is far easier if you use CloudFront as the endpoint. At least it was 6 months ago when I was playing around.
I appreciate your breakdown. Helps those of us evaluating various web services get an idea what people are spending and what capabilities we might have overlooked. Neat stuff.
This is interesting as I'm doing a lot of these things in my spare time on my own sites, eg running postgres servers and backing them up in an automated way, as well as developing them and holding down a full time job.
As others have said, Heroku seems like a very expensive way to lock yourself in to one provider.
One of my initial thoughts was that this list gives an interesting pricing signal to the various services that are shown.
It's interesting to consider whether certain services are pricing for perceived value and what the market will bare, or under-pricing for acquisition in an immature market and going for the land grab, or just underpricing themselves.
Why not include the cost of your time to maintain and improve the service? Without including the cost of your own time it seems hard to make budget decisions (e.g. the tradeoff between a PaaS and using a provisioned VPS). Even if you're not paying yourself anything, it seems important to include what you could be making if you were doing something else with your time.
I definitely plan to share the cost of my time to work on Cushion. I've been tracking my time since the original idea. To give you an idea, I just crossed 2,000 hours for design and development. I didn't track time for customer support and writing, but it's up there. If I were to charge hourly for all my time spent on this app, I'd be sitting pretty. :)
Anyone here have any thoughts on AppEngine vs Heroku?
I use AppEngine for my experimental projects because I can get a simple data model and some handlers running in minutes and a (supposedly automatically scalable) useful backend up in less than a day, and at very low cost. (Virtually free if I use an *.appspot.com domain).
You do get lockin issues but don't you also get lockin from Heroku? Is there big advantages to Heroku that make it worth it?
I haven't used AppEngine, but getting a heroku instance up takes less than an hour. If I did it more often, I'd think it could be done for a typical CRUD app in < 30 minutes.
An instance on DigitalOcean would be more like 2 hours.
There's no lockin that I've experienced on Heroku.
Setting up your requirements for heroku is not that simple. You have to create all the necessary information, like your requirements.txt etc. Turn on some level of real postgres, things of that nature.
Nicer framework generators will produce dependency configuration, have params for what database backend to produce configuration for, and produce applications prepared to be 12-factor (log to stdout, accept configuration via environment) out of the box.
More generators than those will allow you to create reusable app templates which have these features so you don't have to repeat the (still very simple) configuration process for each app.
You get lockin issues with any vendor - even server colocation :)
AppEngine is a great tool for a great price. It scales smoothly with an affordable increase in pricing.
Heroku is sort of famous for getting too expensive at large scale. Their advantages are their ecosystem (most of the additional services listed in the OP can be installed quickly, and you're still only billed by Heroku), and their UIs which tend to be easy to understand and operate.
Parse.com is similar to Heroku in that its simple to get started and has a free tier but gets real expensive once you go over that. I'm rebuilding my Parse back-end and going with Google app engine managed vms for a number of reasons.
Heroku offers more integration addon/services than App Engine ever does. Heroku was definitely invented to compete with App Engine. Google did not have a Cloud Platform at the beginning for you to create and integrate MySQL as a service with App Engine app.
People are talking about the cost of Heroku, but it doesn't seem that outrageous to me.
I mean, $180/month for a running app, that is making money, isn't that much. Once the traffic grows, and you're spending $500 or $1000 per month, yes, it becomes worth it to look into alternatives (because $1000 on Heroku can probably be replaced by $150-200 on more powerful hardware). But moving from Heroku to a less managed option just to save $80/month doesn't seem particularly worth it.
Heroku is not 100% management-free, and if you know how to deploy the same infrastructure they provide, using VMs, dedicated servers and whatever, then there is little reason to use them at all. You get better flexibility, lower costs and very little more management requirements over what you get at Heroku.
Of course if you don't know how to do that, then you pay them to do it for you, until you can hire someone to do it on your own.
224 comments
[ 7.4 ms ] story [ 299 ms ] threadWithout a doubt, this cost has increased over time, but in the beginning of the startup, it was just $20 or $30 a month.
Ive built and home grown all of these products or at least what I needed in house.
Shameless plug: https://rdnation.com and https://snation.com
The details of the WOT report make it seem like it could be an outdated report of spam/malware.
Simply not having to think about how to stage, locate and run an app saves a lot of visible and hidden engineering time.
Source: I work for a company which donates engineering time to what is, in some respects, a Heroku competitor (Cloud Foundry).
Looking at https://devcenter.heroku.com/articles/ssl-endpoint using SSL as an addon for $20/M is expensive. With ELB I can pay namecheap for an SSL cert for like $10-$30 once and be done with it.
I think that starting startup can cut a lot of corners by using open source resources and cheap cloud servers like DO.
The SSL certificate expense is listed as $99/year from namecheap, ie. $8.25/month, so there seems no need to get a Symantec one.
However they/it/he are paying Heroku $20/month for handling the SSL traffic, not the certificate itself, which probably includes running a load balancing server in front of their apps that automate the SSL termination across nodes. Which seems like a fair cost to me.
I think the description of this cost is the error which causes the confusion.
I would try to reduce costs in those Heroku workers... $175/month seems steep.
It may pay off for certain organisations, but it currently a quite a rabbit-hole, with a constantly changing landscape, complexities of its own, etc.
By the way, your use of Heroku makes total sense given that you're basically alone, I hope that the lock-in won't be too difficult to overcome when moving away from it.
https://www.hostingforappdevelopers.com/read.html
I also want to say that your site has a great design. Are you doing that yourself or would you mind sharing the service you're using?
Depending on one's business, HA/performance may not matter that much, but personally my startup is having a 'proper' design from day 1 - downtime is not acceptable for my business model.
Personally I'm pretty confident about market-fit and expected growth even before the launch day of my startup.
But that's not the case, most times!
Less servers and components mean less things to break.
Grab a dedicated server and host your shit on it (have a DB replica because losing data sucks, but that is easy). You will be able to run like that for much longer than you probably suspect and the chance of downtime will be massively reduced.
Why? Because a single server has a very low chance of having anything going wrong.
Not only that, but there is a lot of overhead in performance when you split everything up into components. You will be surprised how many requests you can handle on a single inexpensive dedicated server when you haven't put too much effort into "architecture".
When you actually find yourself running out of headroom on that setup you will know much more about your application and it's requirements then you did at the start, so you will have a better idea of what to split out, and a better understanding of what parts of your app need more performance anyway.
Similarly, if the work you are performing has any value for your business, it must be retryable, and the queued items must be safely stored (i.e. don't use a single Redis server - use something distributed like SQS instead).
All of that calls for a 'proper' architecture. I'm not talking microservices or anything fancy - just a webapp/worker/db separation and the like. Which is exactly what dedicated server (or DigitalOcean) users tend to lack at early stages.
Fortunately database warm standbys are WAY easier than true multi-master.
---
# Pros of setting up your own host.
- Huge cost savings.
# Cons
- Heroku is way easier to scale, just drag the slider. Upgrading databases / redis is still a pain. I recommend going with RedisCloud instead of Heroku Redis. With database you have to do a migration and copy the data over.
- Having to deal with securing your own server and maintenance.
- High upfront cost due to setup
At the end of the day, it's about what your time is worth :). There's no right answer. I go with Heroku myself. The instant gratification is hard to beat.
That's why I recommended in a other comment using people that do stuff similar to what you need and split cost among several companies.
Might not be that important in your case, I am guessing.
I'm confused.
No one is understanding that having a few extra hours a month to build features and find customers is worth a couple hundred bucks. It's probably worth thousands of dollars a month.
There is a point of diminishing returns, of course, pushing down to $5/month is a lot of pain -- super low cost super low resource hosting, etc.
While I understand that these numbers also give an opportunity for people to point out areas where there may be potential cost savings or opportunities to improve infrastructure, I am not sure generic comments of saying you are spending too much or too little, or I don't spend that much are beneficial. A few key pieces of information that would help correlate these numbers in my opinion are total users on each plan, monthly active users, and the average amount of data required for the users.
Just based off the Stripe numbers I would estimate that they have between 6000-7000 annual paying users (this number may actually be a little lower if the majority of users are paying monthly instead of annually). That does not include members that only trial the service, although I would suspect this number is small.
Long story short, every business has different requirements and usage. What works for one company with 6000 users may not work great for the another company with 6000 users. I just hate seeing people say that someone is spending too much without knowing all the details.
However, the flip side is that many services have alternatives that do about what they do. They might do so with a lower price or more capabilities. People mentioning those in comments of articles like this facilitates the discovery of those products for people not aware of them.
An example for me was arguing that much of what "cloud" stuff does can be done with dedicated hosting if it was setup correctly. That we just need a combo of the two to satisfy certain markets focused on security and dependability where sharing hardware isn't permitted but they want flexibility & low costs. Thanks to a commenter, I found out I had re-invented the concept of "bare metal hosting" and SoftLayer provided exactly what I was thinking about. I wouldn't have discovered that if everyone and I were just giving props to the OP or further elaborating their own use case.
So, I think exploring the reasons behind the choices of company in question and bringing up alternatives are beneficial. One facilitates understanding. One discovery.
Web services expenses aren't even close to their cost of adding an additional employee to solve these problems in-house.
Learning to automate something a more seasoned *NIX veteran could do in an hour is not always a predictable time expense, and then they might create bugs or run into other issues. $100/month can a bargain and buy you time to do more important things.
Researching services to use can turn into a black hole. There is always a better service that you don't know about. It can be better to decide "I will spend one hour researching metrics/dashboard services, and pick the best one I find, and spend another testing out the service." rather than trying to end up with the most perfect solution and testing every solution there is.
That said, sometimes it's more time efficient to avoid having one extra service to maintain. One has to avoid using a service in a way that the business is too dependent on it, or that the service is not easily replaced. That's just part of what makes running a business hard.
That's another good point about researching services can turn into a black hole. It is almost like the question of what framework/language should I use to build my startup.
I have no issue over the costs with paying for all these services as it is almost always cheaper than building/setting up the same tools yourself, the issue I have is the lack of skills/understanding that going down this route encourages. I like to build everything myself so I learn, not because this is the most efficient way to run things. When something breaks (it always does at some point) it means I can fix it.
There is also the other point which is I love learning about all this stuff :)
I do think that when you are first launching a startup, you should probably do everything as cheaply as possible, which probably means using a VPS or IaaS. As soon as your infrastructure becomes more complicated, you can use Heroku instead of hiring people to manage it for a while. Then see where your growth takes you -- you will probably find that eventually it makes more sense to hire a couple people to manage some IaaS or even renting racks and buying hardware.
As with all things in life, it really depends. I believe this particular configuration makes sense for companies who want to focus on product, not infrastructure -- at a cost of expensive hosting.
However I wish they would have disclosed traffic handled by Heroku servers as well.
It's surprising to me that people think they're spending a lot on infrastructure when they spend $800 on fonts, etc. When you drop the payment processor charges (which are per transaction) they're at $7k to run a site with a lot of functionality for a year. Seems pretty reasonable.
So, one $30-50k engineer gets split 5-10 ways on infrastructure. Most maintenance is automated with setup and occasionally fixing something using up most time. You usually negotiate 1 day to a week of dedicated time out of the month w/ split being flexible.
Heroku and Redis might still come out a better deal. It's just not going to be nearly as different as people think because one doesn't have to load-up on IT people to deploy or maintain common configurations. All kinds of consultants and smaller fish that will do it way cheaper [than you mentioned] with cost spread across multiple, small businesses.
- The time and money it takes to find said $30-50k engineer
- The risk (and therefore potential cost) if the engineer doesn't do the job correctly
- The unknown cost of "occasionally fixing" could be $50 or $50,000. Given how nebolus the DevOps position is, you also can't be sure the stuff this individual is doing is considered best practice.
- There is overhead time spent on managing such a resource (teleconferencing, IM's, etc)
- There is potential that the resource is unavailable and therefore a potential downtime that could cause your business loss
There's a reason Heroku exists and has been successful, and these are just a few of them.
But this one i would like to ask some questions about:
"The unknown cost of "occasionally fixing" could be $50 or $50,000. Given how nebolus the DevOps position is, you also can't be sure the stuff this individual is doing is considered best practice."
Who gets to decide what's best practice? How does such information about such best practice propagate? How much does it cost to acquire such best practice knowledge? Is trying to mitigate for this all the time a theoretical constraint on all possible solutions (i.e. without trying to mitigate for this, things will not work at all) or for any of the issues you bring up that are probably valid assumptions for some set of solutions?
- The time and money it takes to find and evaluate providers like Heroku
- The risk (and therefore potential cost) if the SaaS solution doesn't handle your needs as advertised
- Unknown costs that crop up both in paying the provider and handling stuff outside the provider.
- Overhead spent managing cloud applications
- Potential that local Internet or cloud startup experiences downtime.
There's a reason dedicated IT people exist and have been successful, and these are just a few of them.
I've seen a lot of people that hire devops in-house (technically, it's a little hard to see how "devops" might work with the "ops"-person just doing drive-by installation and maintenance once every month -- but I suppose if the "dev" part isn't "special" (web application server + database + static resources + database) one could argue there's not much need for "devops", just "ops"?).
Although 30k for full-time ops works sounds really low, anywhere? (As a customer, I'd probably prefer one tenth of a 100k/year engineer than one tenth of a 30k/year engineer... even if the price might be 3-4 times as high).
https://news.ycombinator.com/item?id=10878468
http://www.aonix.com/pdf/PercDatasheet.pdf
They run POSIX and safety-critical apps side-by-side on a microkernel (eg QNX, INTEGRITY) w/ latter usually on a built-for-purpose runtime like PERC. They also would combine app, dependencies, and runtime into one image that could go into a ROM. Services like Boxfuse are doing it for cloud but the model goes back over a decade with proven benefit. Goes back further in security kernels (eg Aesec's GEMSOS or BAE's XTS-400) where they did same thing for preventing sensitive components from leaking to ported POSIX/Linux apps. So, it goes back almost three decades now. That's how long it takes industry to learn apparently. ;)
EDIT: Wait, you're the founder rather than just a user. You might have heard of some of this advanced, clever Java stuff already then. Haha.
I would expect the other way around. In fact, "for a while" doesn't work in real life. So moving from AWS to Herkou then migrate to another IaaS? That's going to be really costly. My pitch is use the tool you are more comfortable with at the beginning. When you grow to a reasonable size, I would start creating infrastructure on AWS directly (or other IaaS providers). I work with AWS daily so if I were to start my business I'd build on AWS first (plus there is AWS startup credits that you can apply). YMMV.
Having a Iaas/Paas doesn't mean you don't have to manage anything - that's a myth. Sure, it might be doable if you own a trivial app with just 1 web server + 1 app server + 1 database server. But once you're doing anything remotely complex like building a 10-node Elasticsearch cluster that is running out of RAM, or a 10 Cassanda node cluster, or a Kafka queue that can overfill, you always need an engineer to manage your infrastructure.
I know companies that don't rely on any hosted cloud services for our infrastructure, and have just 1-2 people they rely on to be that infrastructure guy, and even they are not 100% full-time. It's only when things blow up that they tell him to stop what he's currently doing, and address the issues. Unless you have 10K+ employees, you'll be hard pressed to find companies that have full-time engineers that just twiddle their thumbs, and stare at a monitoring dashboard 40 hours a week to monitor a component.
I've always thought that bigger companies use font services like that and not startups.
Also, there is no cost for a designer in the pricing, so either that is a startup cost not included in the ongoing pricing, or is done in house. And a designer is not cheap.
For instance, take a look at this font I'm currently integrating into the app I'm working on:
https://www.myfonts.com/fonts/intelligent-foundry/averta/ https://www.myfonts.com/fonts/intelligent-foundry/averta/buy...
If you want to put these on the web, you're paying $200 per font type. If you want to buy the whole family, that's almost $1200.
Companies with a design focus are happy to pay this in order to differentiate themselves from others.
We got 6 months free Mailchimp and fee-free on Stripe for the first 60k i think, really helped at the time
You could totally spin up multiple digital ocean instances to get a much cheaper infrastructure. I usually use one for load balancing, 2+ for running the app, and usually one for the backend postgres. You could also get away with using something other than digital ocean for the postgres.
But for 6000 users, depending on the app. I can probably get away with like $30 a month with this setup. Including the database.
Total setup time might be a day the first time. But it scales pretty well, costs significantly less.
The other expenses appear to be various other services (slack, support site, logging service, etc). Lot's of 3rd part services to handle stuff.
Obviously it's a lot cheaper if you can do many of these things yourself (run your own support web app, use a free chat service, etc, etc). But we all have to decide what services we want to pay for vs run ourselves - in some cases it is cheaper to not have to spend the time!
Which log aggregation services is everyone using ?
I've not used Papertrail, so can't compare to them, but it's significantly easier to use than Loggly, they've been kind to us through the odd overage, and it does what we need it to do (Centralise a load of application logs and allow us to search over them and set alerts). The live tail feature is nice as well.
Looking to migrate to https://logentries.com At least their free plan offers 5Gb/month.
Is there any reason for this or just an oversight?
I used that for my static S3 site, it's working fine.
As others have said, Heroku seems like a very expensive way to lock yourself in to one provider.
I had no idea they were free.
It's interesting to consider whether certain services are pricing for perceived value and what the market will bare, or under-pricing for acquisition in an immature market and going for the land grab, or just underpricing themselves.
Why not include the cost of your time to maintain and improve the service? Without including the cost of your own time it seems hard to make budget decisions (e.g. the tradeoff between a PaaS and using a provisioned VPS). Even if you're not paying yourself anything, it seems important to include what you could be making if you were doing something else with your time.
I use AppEngine for my experimental projects because I can get a simple data model and some handlers running in minutes and a (supposedly automatically scalable) useful backend up in less than a day, and at very low cost. (Virtually free if I use an *.appspot.com domain).
You do get lockin issues but don't you also get lockin from Heroku? Is there big advantages to Heroku that make it worth it?
An instance on DigitalOcean would be more like 2 hours.
There's no lockin that I've experienced on Heroku.
Most of my time is spent consulting docs.
More generators than those will allow you to create reusable app templates which have these features so you don't have to repeat the (still very simple) configuration process for each app.
AppEngine is a great tool for a great price. It scales smoothly with an affordable increase in pricing.
Heroku is sort of famous for getting too expensive at large scale. Their advantages are their ecosystem (most of the additional services listed in the OP can be installed quickly, and you're still only billed by Heroku), and their UIs which tend to be easy to understand and operate.
https://web.archive.org/web/20071105005403/http://heroku.com...
It's fairly likely they were working on them in parallel without knowledge of each other, though.
I mean, $180/month for a running app, that is making money, isn't that much. Once the traffic grows, and you're spending $500 or $1000 per month, yes, it becomes worth it to look into alternatives (because $1000 on Heroku can probably be replaced by $150-200 on more powerful hardware). But moving from Heroku to a less managed option just to save $80/month doesn't seem particularly worth it.
Of course if you don't know how to do that, then you pay them to do it for you, until you can hire someone to do it on your own.