If you were to advise someone starting to build a SaaS product from an idea, what cloud providers would you recommend knowing more about beyond the bigger three?
Unless you consider it essential for your product to scale by few orders of magnitude on a dime, it would be sensible to add "non-cloud" hosting into consideration - Hetzner, OVH, Scaleway, ...
Those are considerably cheaper, suffer less from unpredictable performance, a.k.a. "loud neighbour" problem, and are less likely to make you lock into some API/feature that would become a huge headache if you ever decide to move.
We had a lot of noisy neighbour issues on Hetzner's VPSes, to the point that we stopped using them entirely. Their bare metal is a great offer though. And fully agreed about lock-in. If you can, design cloud-agnostic from the start. You get a lot more flexibility and you're in a much better negotiating position as you grow. Not realistic or necessarily sensible for every company/team though.
You don't say where you notice that packet loss from, so I guess you are talking about the US?
The by far biggest cable goes from Finland to Germany. No idea what happens after that. I believe Hetzner uses it and I have never heard that they wouldn't have enough bandwidth. But I have neither inside knowledge nor do I use them.
Being in Finland myself I have better latencies from real cheap Scaleway in Paris than from far more expensive AWS in Frankfurt. So it's not always you get what you pay for. Still no drops from either.
Maybe Hetzner has more noisy neighbor issues because they are cheaper? The same could hold for Scaleway, but I have never noticed anything.
> The by far biggest cable goes from Finland to Germany. No idea what happens after that. I believe Hetzner uses it and I have never heard that they wouldn't have enough bandwidth. But I have neither inside knowledge nor do I use them.
Packet loss can have numerous causes, not having enough bandwidth is one but faulty or misconfigured gear or misconfigured routing can also cause issues.
> Maybe Hetzner has more noisy neighbor issues because they are cheaper? The same could hold for Scaleway, but I have never noticed anything.
Yeah, in general, cheaper virtual servers are more likely to have these problems, just like cheaper bandwidth is more likely to be oversold. Low-cost providers can push it pretty far if most of their customers are hosting low-to-moderate-traffic websites or other HTTP services, which are pretty forgiving of steal time, low-level packet loss & latency.
I'm from Canada. The packet loss can be really random. Despite having various checks between our servers at various providers, and being fairly sure the problem is Hetzner, they always blame the other network.
Scaleway isn't really non-cloud. Their services are very similar to EC2, S3, Glacier etc. I would compare them more with Digital Ocean. They're just cheaper and you can actually chat with them on slack without going to all the 'raise a ticket' rigmarole that all the big players do. At work I deal with Microsoft's double-outsourced "premium" support which is awful. If the question is not in the docs (in which case I would have found it myself) tickets keep ping-ponging for months without solution, going through the same useless troubleshooting over and over because they keep handing over to different agents.
I use scaleway privately only but very happy with them. I wish I had their kind of support from the vendors at work. You can often try out upcoming services for free, too.
Hetzner wants a copy of your ID which I was not willing to provide so I couldn't try them. Germans are normally really privacy-conscious so this surprised me. This was > 10 years ago though so perhaps they stopped this practice.
We don't run on one yet -- we are at the stage where we are expanding my understanding of the cloud service offerings and am curious what is available with good reputation.
SaaS is at whiteboard phase; where we invest our engineering time will be important as we recognize the issues of lock-in for something like AWS.
I wouldn't worry about being cloud agnostic, just pick one. My recommendation is GCP because it takes less engineering effort. No matter what, you will have vendor specific code, but you can minimize it.
The more important question you might ponder is how to validate the idea is something people both want and will pay for, before investing time & money in a good cloud setup. If you select smaller providers, expect to spend engineering time reproducing existing products in the big ones. Saving money on VM hourly rates often costs you more elsewhere.
The only time I recommend worrying about multiple clouds is when you have a "deploy to your cloud" product and strategy.
Agree on stripe as well as crm tool. Good thing about aws (and I’m sure others) is you can ride on their soc2 report, security is a huge concern when selling a SaaS tool and when buyers hear aws they assume you’ll pass security no prob, others, ymmv and you’ll have more work to do.
This question potentially lacks the situational context necessary to provide the answer/insight it's really looking for.
What's the SaaS going to look like technically? Lots of compute processing? Or maybe large volumes of data? Perhaps machine learning? Or is it just a simple website and app deal?
Then there's the question of the technical competence on the ground (and the opinions that tag along for the ride). On the one hand, what's the expertise level? On the other hand, what do people prefer using? (And separately, what to people have experience with?)
I wonder (I don't have enough context to competently move the slider to "proper accusation") if somewhere along the way a general search for un-turned-over rocks got specialized/narrowed into "the overhead's all in the sheet metal benders" (the hardware). If this *is* the case, the only correct course of action (IMO) is to wind the train of thought backwards (oohc oohc) back to that frame of reference, then apply https://en.wikipedia.org/wiki/Five_whys until you're somewhere alien and interesting.
(Reiterating the caveat at the start of the previous paragraph, this is all massive conjecture and just-in-case assumption.)
If there was in fact a point at which hardware was specifically called out as a primary focus of optimization, I would loudly note that this type of hyperfocus leaves space for entire forests' worth of trees to fall over without ever being noticed, in this case for all of the software. Not just certain bits of it but like the whole kit and caboodle, unnoticed.
A related alternate possibility is that hardware optimization might be being treated like an axis point to orbit around, which can contribute to seeing things like immovable Mt Everests, and the construction of great and confusing Rube Goldberg machines to work around... perceived resistance that isn't really there.
To offer a 180-degree counterpoint that sorta flies in the face of the abstraction-away you might be trying to do here, if you want to look at different providers, I would suggest adopting a pat-answer strategy of keeping the architecture cloud-agnostic *where reasonable to do so*, and further suggest trying multiple providers - how does their support help out when it's 3am and you have no attention span and you accidentally something ridiculously straightforward? What's the performance like relative to the workload? What do all the engineers that are going to be headdesking against this stuff every day think about different options? Etc.
It's quite possible this reply is entirely misguided, in which case please ignore. I'm still learning the art of reading intent through text, I have a long way to go.
> It's quite possible this reply is entirely misguided, in which case please ignore. I'm still learning the art of reading intent through text, I have a long way to go.
Your mindfulness is quite appreciated and your answer is on point!
We are at the design phase at this point. Fairly data intensive and will definitely have ML components (but that could be centralized and managed with standard MLOps practices). But the data is more on the backend, forward facing components are pretty standard CRUD/API calls against pre-computed and quickly updating data stores.
User-facing data payloads are in design, but expected to be low-to-mid volume.
I'm trying to calibrate a working mindset to get into the industry from a bit of a distance, so thanks for the feedback - and for engaging my somewhat hastily written comment :)
I recently realized (light bulb moment) that cloud platforms like AWS are awesome for prototyping and experimentation: you can spin up seriously wide+deep instances that bill $1/hr, crunch through a complex workload in a couple of hours, and use hundreds of GB of RAM connected to hundreds of CPU cores for the cost of a stereotypical cup of coffee. But that bottoms out after just a very short period of time; a literal couple of hours. I wouldn't be surprised to learn the price scales was intentionally tuned to feel accessible at the prototyping stage - this would take advantage of communicative blind spots on both sides of the management/engineering boundary during evaluation and testing, where management gives engineering a green light budget to play around and learn about the platform, the engineers get insightful field experience after not very much expenditure, it feels possible to do a lot of work without spending too much, and... woops, a short time later they're staring down the barrel of a deadline, the fastest solution is to use their newfound experience and familiarity... and the cycle repeats.
The theoretical blind spot here is of course in the learning/tinkering stage that is so hard to communicate to management - and so when engineering makes positive noises about having some level of understanding and confidence about how to do a particular thing on AWS, the nuance of the hair-thin line between "thing that is generally achievable" and "I've practically played with AWS' implementation of the thing and I understand how to do it in that sandbox" is lost. (And the cycle repeats.) What's the right word for something incredibly well-engineered for altogether depressing reasons? :(
(Of course manglement and disorganization can also be at fault here, although I wonder where the cultural/structural root cause was when NASA forgot the egress bandwidth costs of migrating to AWS :D https://news.ycombinator.com/item?id=22626097)
Maybe the moral of the above story/thought experiment might be to theorize that AWS's market dominance has had a significant impact on broad tinkering and experimentation, and adjust for that by explicitly authorizing a broad-spectrum tinkering budget - "go and play" meets "engineering feasibility study" or something. Take the brakes off just enough that the engineering opinions that eventually come back are the product of having had the chance to stare into the horizon blankly for a bit, that sort of thing. Hypothetically.
Regarding the data intensity bit you mentioned, I'm personally up to the "just buy the whole computer" point at the moment mindset-wise, since this presents the most cost savings where it's feasible to do so. One somewhat cute but relatively transferable/normalizable way of broadly articulating the hard differences is to find vendor workstation/server configurators, adjust all the settings until the big number with the $ in front won't go any higher :), then (try and) configure something comparable on EC2.
> Oracle Cloud - good alternative for bandwidth heavy products (used by Zoom)
please don't.
I have nothing but the worst experiences with Oracle Cloud. Granted, I was always a little biased against Oracle as a company; but I have a friend working on OCI and assumed that since what I needed was really basic it would be fine.
It really wasn't, it was awful, confusing and opaquely and egregiously expensive.
> It really wasn't, it was awful, confusing and opaquely and egregiously expensive.
That is true, but it is also very much an understatement of just how miserable every step of the OC experience is.
I was once forced by a customer to endure the torture that is OC, and the only thing that saved me was when Oracle's terrible management software just deleted our entire account and deployment one weekend. Gone without a trace, could not even log in anymore. Luckily, that was enough to persuade the client to move to Azure.
I won't even take on clients if they are using OCI now.
I've had previous clients whos entire account with production servers just get nuked without even so much as a curtsey email and OCI support WILL NOT HELP.
Do you know if this occurred to a free or paid account?
To be fair, I have been testing Oracle Cloud Infrastructure(OCI) on a paid account for about 2 years now and actively monitor server hosting forums.
During that time I have had no "abnormal" problems with paid production account.
However, it is well known that Oracle accidentally nuked free/trial accounts early on, and again recently accidental nuked free/trial accounts reserved public IPs.
However, I know of no such events for paid accounts, and paid accounts on OCI should be production ready quality in my experience.
On the other hand support really is somewhat bad, but support is free and the cloud costs are much less than AWS, while having better products than similar priced cloud products such as Lnode, Vultr, Digital ocean.
Whilst I appreciate the comparison, it makes no sense for there to be a difference in the level of alerting, stability, security, safety, etc of a free/paid tier and a full production tier.
You can bet your bottom dollar I trial things on lower tiers, and much later on I may very well take the things I learned into production. My production budget isn't as big as some, but its still a tidy 6 figure sum each year just for cloud hosted services, with no licences or wages in that figure. If my trial on your platform goes tips up without notice, I'm going to likely conclude you're not a particularly safe place for my production stack for whatever it is I'm working on. You should also count on me telling my contemporaries about my experience too.
On the topic of their "free accounts", I did personally attempt to go through that process last month and I really don't understand how they have managed to fuck it up so bad.
The basic idea is simple, "give developers free personal resources so they might be inclined to start using your cloud for work related thing". Instead, the "free account" is basically just a guided tour on OCI incompetence before it boots you out onto the street.
On the surface they have given it some thought with setting up the system perfectly with the whole, "if your account is free you can never be charged thing" but they can't seem to stomach people actually using their "free account".
I was trying to price up something between hetzner and AWS and as a starting place AWS ends up cheaper until you get to a certain point then hetzner ends up cheaper. Basically wanted 3 servers. One for load balancer and then 2 web servers so I can pop them in and out. But with hetzner it’s minimum like $30/m per server when I only need 2c/2g I get 3 servers at AWS for $32.
At least some Hetzner offers used to be used servers and they can suddenly fail. I haven't followed them.
But I guess you need to think about your backup plans anyway (depending on what service you run). So using a less reliable provider might just mean that you practice your backup plan slightly more frequently. Some OVH customers can tell you the story when one of their data centers burned down.
Also Oracle OCI has OKE, which is Oracle Kubernetes Engine. It works nicely and the quality of service has been very good in my experience over the past four years.
Why is it a bold assertion? Looks like the Oracle folks just need to submit a PR to the CAPI docs to include a link to their provider which is available now.
It's usually quite hard to find people who don't work at Oracle who speak positively about Oracle Cloud.
We tried to use it at work, it was the second-biggest disaster of a cloud provider we've ever encountered (Huawei is the first). Stuff is just broken so often it's hard to ever feel like you can trust it.
This sounds like a nifty solution for compute, but I suppose for data storage and such you would want a separate managed service? Or does this hook into that for various providers too?
Storage is already abstracted via Kubernetes with persistent volume claims and CSI (container storage interface)
You can specify the default storage class to use the provider's CSI driver. The PVC is then specified and consumed the same way in a portable manner across providers.
We run almost all of our production databases on k8s - postgres, clickhouse, cockroachdb and more. Many others do too. K8s itself already comes with a db of sorts - etcd so it’s not a non-standard usecase.
Block storage? When you create your persistent volume claim, you can specify an accessMode. "readWriteOnce" will target block storage of the storageClass, "readWriteMany" will use file.
You can also choose something specific like a mounted hostVolume which is not provider specifc.
planetscale (if I need rdbms) or firestore or dynamodb (if I need just document).
I checked latency of planetscale and dynamodb from frankfurt,paris,singapore from DO/linode/scaleway to db in same region; these are low latency (mostly single digit).
Cloudflare. Between Pages, Speed, Access, Workers, durable objects, KV store, R2, Images, etc they’re maybe a small handful of product releases away from being able to replace the providers you mentioned for many users and applications. Plus I’d wager that many applications can be built 100% on Cloudflare today (and are).
Benefits?
- Pricing. Compare pricing on Workers to functions across those platforms as one example. For many use cases you can use the Bundled pricing model. It’s insanely cheap. Crazy cheap. Then start looking at pricing on bandwidth (ESPECIALLY BANDWIDTH), KV, R2, DO, etc. You’ll be pleased.
- Pace and market disruption. AWS is 20 years old. That makes them essentially the IBM of cloud. No one ever got fired for buying AWS but seriously compare uptime, pricing, etc.
- Speed, uptime, performance. Cloudflare’s edge network and architecture is truly impressive. KV store? Workers? R2? Durable objects? Forget worrying about regions, etc. All of this and more runs at their edge in more than 270 cities worldwide.
- Less vendor lock-in, more standards. R2 is S3 compatible. Workers are actually WebWorkers. Pages is Jamstack.
I’m not exaggerating when I say that Workers (as one example) have 2x the performance at 1/4 the cost compared to Lambda. Then start looking at bandwidth…
In my mind where AWS truly shines is exactly why they’re the IBM of cloud. Support for enterprise, government, etc. Really complicated access and management, regulatory requirements, extensive customer requirements, an entire cottage industry and ecosystem of consultants for everything you can imagine, various supplemental SaaS offerings, the list goes on.
Everyone likes to think you can’t go wrong even just starting out with AWS. I don’t like to crush dreams but the reality is most efforts fail in a year or less and only what, maybe 1/10,00,000 projects/companies will ever need the full scale of what I just listed.
Not shilling Cloudflare - just a happy customer who’s truly baffled why Cloudflare’s full product suite isn’t leveraged more often and more fully.
Also forgot to mention - for compute providers, other storage, bandwidth, etc look at their bandwidth alliance. Free bandwidth to/from any of the members.
That’s all great and amazing for the edge but not for general compute which the majority of applications still need. Yes you probably could design it with just workers but most popular frameworks just don’t lend themselves well to that and using things like accessing private databases from workers is still hairy at best and almost certainly not certifiable.
If it’s not a good fit it’s not a good fit, we have choice!
However, as I said the vast majority of applications don’t involve message queues for hundreds of servers or Lucene. Many not even SQL.
I also pointed to compute and bandwidth alliance members that augment Cloudflare very well so you can build whatever you want while still taking advantage of the Cloudflare edge.
Or put Cloudflare as a CDN or augment with what they do have in front of whatever cloud you want. Azure and GCP are limited members of the alliance that offer reduced pricing egress to Cloudflare.
I promised myself that I would never try to learn Cloudflare until it made its pricing clearer and easier to calculate. How do they expect someone to use their services when no one can do a simple math without contacting their sales team?
Not sure why you wouldn't want to go for one of the bigger three. The advantage of using them is that you can find help online very easily. Also easier to hire an aws or azure specialist.
Depending on your architecture and use cases you can save much more than a few salaries by moving to another provider.l, especially if outgoing bandwidth is involved and not compensated for.
I'm surprised there's been many mentions of DigitalOcean, but none of Linode, which I think of as occupying almost the exact same space in terms of offerings.
Linode's future is very hard to predict with them very newly part of Akamai. I wish them the best, of course. and I currently use them for my personal VPS. But it would be a risky time to pick them as the platform on top of which to build a new business.
Whether the future is consistent with the past on the points you listed is hard to predict so soon after they got acquired, but I suspect it will increasingly diverge over time.
We trialled Linode for a bandwidth-heavy product and found that their network has quite heavy contention (decent enough speeds for bulk TCP stuff, but frequent periods of elevated packet loss, latency and jitter that caused problems for our realtime application). It makes sense given how cheap their bandwidth is, and probably isn't an issue for hosting websites / API servers / etc, but something to be aware of.
tl;dr: Starting out with containers that have no own state, running on x86 on Linux has the highest chance of success I'd say, and when you grow, you have the capability to flow between IaaS, PaaS and SaaS and all-in-one clouds as you wish. That means that your 'first' service provider is irrelevant if you don't have pre-existing knowledge or affinity. If you can produce good container images, you can run anywhere.
At some point 'the other clouds' aren't as relevant.
Think about the following interactions instead:
- CNCF compatibility (interpret as deep and as wide as you like)
- Infrastructure vs. Platforms vs. Services
- Legal boundaries
- Locality (can interact with legal limits, but also latency, transfer costs)
- Scope of services vs. scope of what you actually need
A lot of providers are good at a thing, and bad at everything they tack on to it. Some providers are reasonably good at many things, but win on integration between those things. Others are simply too dissimilar to orchestrate, so either you'll have to bring your own orchestration or not use it in orchestrated scenarios.
Instead of knowing about the clouds, know about requirements engineering. Fitting your needs and the services you pay for is way more important than the details of those needs and services.
If you just need some random compute (read: a shell into an OS, a complete VM, a container, things like that) and nothing else, do NOT use some cloud. It will require you to do a lot of other things as a side-effect of using those services at all, and will cost a lot for what you need.
On the other hand, if you need to be highly elastic, have completely managed RDBMs on-demand available and orchestrate networking, IAM, object storage, block storage, compute and ingress, do start out with a cloud.
Regardless of what you are building, make sure you know ahead of time if:
- What your scaling is going to depend on (usage, work hours peaking, seasonal peaking, tenants)
- What your scaling is going to be like (horizontally scale and spread the load? vertically scale for a few weeks until you reach the scaling limit and then rebuild the application the right way instead? deploy one instance per customer?)
- What availability rules are you going to have? (downtime? data loss? time to recover?)
- what legal limits will it have?
Example for a MVP SaaS: say you want to manage shopping lists for consumers, you might call it Shoppr and build a PWA and a app-wrapped PWA so you get immense reach. You mostly have front-end engineers, but you do know a bit about metrics and scaling.
I'd say that means:
- Downtime for a few hours unlikely to tank the business
- Legal limits are basically just generic data protection
- Scaling is likely linear
- Since your data is mostly basic CRUD, any read-replicated system will do
This can be built using any stack, and as long as your persistence can keep up you're golden. Don't fuck it up with an ORM that doesn't know how to create the proper indexing rules on tables and you can easily get a couple of million customers on an IaaS-only provider that just has virtual machines or containers, and only has one flavour of persistence store. Plonk Cloudflare in front of it and done.
You can make this infinitely more complicated, but as an example add this feature to make this entire setup suck and the entire infrastructure incompatible with the needs of the application: international receipt scanning to recommend/autocomplete shopping lists for customers. Suddenly your requirements are expanded with:
- Incoming upload queue
- Object storage for image blobs
- OCR or ML pipeline to process images
- ML or Analysis pipeline to make sense of the contents of the now processed/read images
- Instances or expansion for all of the above per region
To make your traffic bill not suck you'll need endpoints in most majo...
If you need many of their services, AWS is very difficult to beat. Nobody else quite matches up with them on breadth offerings. If you don't, do not use AWS due to the cost and the mental overhead of management.
If you just need to spin up some servers and want them to be fast and cost effective, Hetzner is the current champ in the US market (their new US datacenter). DigitalOcean, Linode and Vultr are entirely reasonable options after Hetzner.
If you have a $50-$100 / month or less budget, know how to set up and secure a linux server, and want to stretch your dollars as far as you can with a top notch cloud provider, Hetzner wins at present. And then throw Cloudflare out in front of it until or unless you can justify paying for a service. Keep your costs low, keep your infrastructure simple, keep your runway long, focus on selling selling selling (customers).
104 comments
[ 3.2 ms ] story [ 173 ms ] threadThose are considerably cheaper, suffer less from unpredictable performance, a.k.a. "loud neighbour" problem, and are less likely to make you lock into some API/feature that would become a huge headache if you ever decide to move.
The by far biggest cable goes from Finland to Germany. No idea what happens after that. I believe Hetzner uses it and I have never heard that they wouldn't have enough bandwidth. But I have neither inside knowledge nor do I use them.
Being in Finland myself I have better latencies from real cheap Scaleway in Paris than from far more expensive AWS in Frankfurt. So it's not always you get what you pay for. Still no drops from either.
Maybe Hetzner has more noisy neighbor issues because they are cheaper? The same could hold for Scaleway, but I have never noticed anything.
Packet loss can have numerous causes, not having enough bandwidth is one but faulty or misconfigured gear or misconfigured routing can also cause issues.
> Maybe Hetzner has more noisy neighbor issues because they are cheaper? The same could hold for Scaleway, but I have never noticed anything.
Yeah, in general, cheaper virtual servers are more likely to have these problems, just like cheaper bandwidth is more likely to be oversold. Low-cost providers can push it pretty far if most of their customers are hosting low-to-moderate-traffic websites or other HTTP services, which are pretty forgiving of steal time, low-level packet loss & latency.
I use scaleway privately only but very happy with them. I wish I had their kind of support from the vendors at work. You can often try out upcoming services for free, too.
Hetzner wants a copy of your ID which I was not willing to provide so I couldn't try them. Germans are normally really privacy-conscious so this surprised me. This was > 10 years ago though so perhaps they stopped this practice.
https://render.com
https://digitalocean.com
Why do you think it's important to know more than the one you run on?
We don't run on one yet -- we are at the stage where we are expanding my understanding of the cloud service offerings and am curious what is available with good reputation.
SaaS is at whiteboard phase; where we invest our engineering time will be important as we recognize the issues of lock-in for something like AWS.
The more important question you might ponder is how to validate the idea is something people both want and will pay for, before investing time & money in a good cloud setup. If you select smaller providers, expect to spend engineering time reproducing existing products in the big ones. Saving money on VM hourly rates often costs you more elsewhere.
The only time I recommend worrying about multiple clouds is when you have a "deploy to your cloud" product and strategy.
For ease of use, I like GCP.
Hetzner
What's the SaaS going to look like technically? Lots of compute processing? Or maybe large volumes of data? Perhaps machine learning? Or is it just a simple website and app deal?
Then there's the question of the technical competence on the ground (and the opinions that tag along for the ride). On the one hand, what's the expertise level? On the other hand, what do people prefer using? (And separately, what to people have experience with?)
I wonder (I don't have enough context to competently move the slider to "proper accusation") if somewhere along the way a general search for un-turned-over rocks got specialized/narrowed into "the overhead's all in the sheet metal benders" (the hardware). If this *is* the case, the only correct course of action (IMO) is to wind the train of thought backwards (oohc oohc) back to that frame of reference, then apply https://en.wikipedia.org/wiki/Five_whys until you're somewhere alien and interesting.
(Reiterating the caveat at the start of the previous paragraph, this is all massive conjecture and just-in-case assumption.)
If there was in fact a point at which hardware was specifically called out as a primary focus of optimization, I would loudly note that this type of hyperfocus leaves space for entire forests' worth of trees to fall over without ever being noticed, in this case for all of the software. Not just certain bits of it but like the whole kit and caboodle, unnoticed.
A related alternate possibility is that hardware optimization might be being treated like an axis point to orbit around, which can contribute to seeing things like immovable Mt Everests, and the construction of great and confusing Rube Goldberg machines to work around... perceived resistance that isn't really there.
To offer a 180-degree counterpoint that sorta flies in the face of the abstraction-away you might be trying to do here, if you want to look at different providers, I would suggest adopting a pat-answer strategy of keeping the architecture cloud-agnostic *where reasonable to do so*, and further suggest trying multiple providers - how does their support help out when it's 3am and you have no attention span and you accidentally something ridiculously straightforward? What's the performance like relative to the workload? What do all the engineers that are going to be headdesking against this stuff every day think about different options? Etc.
It's quite possible this reply is entirely misguided, in which case please ignore. I'm still learning the art of reading intent through text, I have a long way to go.
Your mindfulness is quite appreciated and your answer is on point!
We are at the design phase at this point. Fairly data intensive and will definitely have ML components (but that could be centralized and managed with standard MLOps practices). But the data is more on the backend, forward facing components are pretty standard CRUD/API calls against pre-computed and quickly updating data stores.
User-facing data payloads are in design, but expected to be low-to-mid volume.
I recently realized (light bulb moment) that cloud platforms like AWS are awesome for prototyping and experimentation: you can spin up seriously wide+deep instances that bill $1/hr, crunch through a complex workload in a couple of hours, and use hundreds of GB of RAM connected to hundreds of CPU cores for the cost of a stereotypical cup of coffee. But that bottoms out after just a very short period of time; a literal couple of hours. I wouldn't be surprised to learn the price scales was intentionally tuned to feel accessible at the prototyping stage - this would take advantage of communicative blind spots on both sides of the management/engineering boundary during evaluation and testing, where management gives engineering a green light budget to play around and learn about the platform, the engineers get insightful field experience after not very much expenditure, it feels possible to do a lot of work without spending too much, and... woops, a short time later they're staring down the barrel of a deadline, the fastest solution is to use their newfound experience and familiarity... and the cycle repeats.
The theoretical blind spot here is of course in the learning/tinkering stage that is so hard to communicate to management - and so when engineering makes positive noises about having some level of understanding and confidence about how to do a particular thing on AWS, the nuance of the hair-thin line between "thing that is generally achievable" and "I've practically played with AWS' implementation of the thing and I understand how to do it in that sandbox" is lost. (And the cycle repeats.) What's the right word for something incredibly well-engineered for altogether depressing reasons? :(
(Of course manglement and disorganization can also be at fault here, although I wonder where the cultural/structural root cause was when NASA forgot the egress bandwidth costs of migrating to AWS :D https://news.ycombinator.com/item?id=22626097)
Maybe the moral of the above story/thought experiment might be to theorize that AWS's market dominance has had a significant impact on broad tinkering and experimentation, and adjust for that by explicitly authorizing a broad-spectrum tinkering budget - "go and play" meets "engineering feasibility study" or something. Take the brakes off just enough that the engineering opinions that eventually come back are the product of having had the chance to stare into the horizon blankly for a bit, that sort of thing. Hypothetically.
Regarding the data intensity bit you mentioned, I'm personally up to the "just buy the whole computer" point at the moment mindset-wise, since this presents the most cost savings where it's feasible to do so. One somewhat cute but relatively transferable/normalizable way of broadly articulating the hard differences is to find vendor workstation/server configurators, adjust all the settings until the big number with the $ in front won't go any higher :), then (try and) configure something comparable on EC2.
An HP Z8 workstation with 2x28 core Xeon 8280s, 3TB of RAM, 172TB of storage, and 3 NVIDIA GPUs will set you back $111,575: https://zworkstations.com/products/hp-z8-workstation/?config...
An EC2 x2iedn.24xlarge (wat) with 96 CPUs, 3TB of RAM, 2 1.4TB SSDs, and 50TB outbound bandwidth, will set you back
- $192,625 for 1 year at standard monthly rates: ht...
Heroku - For a more "abstracted" cloud that just works
Vultr/DigitalOcean - Cheap VMs, additionaly some services like managed DB and K8s
Hetzner/OVH - Cheap dedicated hosts
Netlify/Cloudflare Pages/etc - for static websites + functions/3rd party services for dynamic stuff
please don't.
I have nothing but the worst experiences with Oracle Cloud. Granted, I was always a little biased against Oracle as a company; but I have a friend working on OCI and assumed that since what I needed was really basic it would be fine.
It really wasn't, it was awful, confusing and opaquely and egregiously expensive.
That is true, but it is also very much an understatement of just how miserable every step of the OC experience is.
I was once forced by a customer to endure the torture that is OC, and the only thing that saved me was when Oracle's terrible management software just deleted our entire account and deployment one weekend. Gone without a trace, could not even log in anymore. Luckily, that was enough to persuade the client to move to Azure.
I've had previous clients whos entire account with production servers just get nuked without even so much as a curtsey email and OCI support WILL NOT HELP.
To be fair, I have been testing Oracle Cloud Infrastructure(OCI) on a paid account for about 2 years now and actively monitor server hosting forums.
During that time I have had no "abnormal" problems with paid production account.
However, it is well known that Oracle accidentally nuked free/trial accounts early on, and again recently accidental nuked free/trial accounts reserved public IPs. However, I know of no such events for paid accounts, and paid accounts on OCI should be production ready quality in my experience.
On the other hand support really is somewhat bad, but support is free and the cloud costs are much less than AWS, while having better products than similar priced cloud products such as Lnode, Vultr, Digital ocean.
You can bet your bottom dollar I trial things on lower tiers, and much later on I may very well take the things I learned into production. My production budget isn't as big as some, but its still a tidy 6 figure sum each year just for cloud hosted services, with no licences or wages in that figure. If my trial on your platform goes tips up without notice, I'm going to likely conclude you're not a particularly safe place for my production stack for whatever it is I'm working on. You should also count on me telling my contemporaries about my experience too.
The basic idea is simple, "give developers free personal resources so they might be inclined to start using your cloud for work related thing". Instead, the "free account" is basically just a guided tour on OCI incompetence before it boots you out onto the street.
On the surface they have given it some thought with setting up the system perfectly with the whole, "if your account is free you can never be charged thing" but they can't seem to stomach people actually using their "free account".
Out of the frying pan into the fire, eh?
Did you check out hetzner "cloud"? 2c/2g seems to be roughly 5$/Month, x3 would be 15$
But I guess you need to think about your backup plans anyway (depending on what service you run). So using a less reliable provider might just mean that you practice your backup plan slightly more frequently. Some OVH customers can tell you the story when one of their data centers burned down.
- Oracle Cloud - IBM Cloud - Alibaba Cloud
* Alibaba Cloud
* AWS
* Azure
* Azure Stack HCI
* Baidu Cloud
* BYOH
* Metal3
* DigitalOcean
* Exoscale
* GCP
* Hetzner
* IBM Cloud
* KubeVirt
* MAAS
* Nested
* Nutanix
* OpenStack
* Equinix Metal (formerly Packet)
* Sidero
* Tencent Cloud
* vSphere
Disclaimer: I currently work at Oracle.
https://github.com/oracle/cluster-api-provider-oci
We tried to use it at work, it was the second-biggest disaster of a cloud provider we've ever encountered (Huawei is the first). Stuff is just broken so often it's hard to ever feel like you can trust it.
You can specify the default storage class to use the provider's CSI driver. The PVC is then specified and consumed the same way in a portable manner across providers.
You can also choose something specific like a mounted hostVolume which is not provider specifc.
Cutting edge stuff like NVMe-oF is even possible, the CSI ecosystem is very active. https://github.com/spdk/spdk-csi
For single server, I'd just use sqlite.
Also forgot to mention - for compute providers, other storage, bandwidth, etc look at their bandwidth alliance. Free bandwidth to/from any of the members.
How can I get a message queue on Cloudflare and have hundreds of servers read and write to it?
How can I get run a Lucene search index on Cloudflare and search against it?
Also R2 is still in beta development
However, as I said the vast majority of applications don’t involve message queues for hundreds of servers or Lucene. Many not even SQL.
I also pointed to compute and bandwidth alliance members that augment Cloudflare very well so you can build whatever you want while still taking advantage of the Cloudflare edge.
Or put Cloudflare as a CDN or augment with what they do have in front of whatever cloud you want. Azure and GCP are limited members of the alliance that offer reduced pricing egress to Cloudflare.
How hard is for them to build something similar to https://calculator.aws ?
- no nonsense prices
- solid hardware, with good, consistent performance & low steal (even on shared nodes, but dedicated is there if you need it)
- great support (humans you can call)
- adding more and more managed services over the last few years
https://www.akamai.com/newsroom/press-release/akamai-complet...
Whether the future is consistent with the past on the points you listed is hard to predict so soon after they got acquired, but I suspect it will increasingly diverge over time.
At some point 'the other clouds' aren't as relevant.
Think about the following interactions instead:
A lot of providers are good at a thing, and bad at everything they tack on to it. Some providers are reasonably good at many things, but win on integration between those things. Others are simply too dissimilar to orchestrate, so either you'll have to bring your own orchestration or not use it in orchestrated scenarios.Instead of knowing about the clouds, know about requirements engineering. Fitting your needs and the services you pay for is way more important than the details of those needs and services.
If you just need some random compute (read: a shell into an OS, a complete VM, a container, things like that) and nothing else, do NOT use some cloud. It will require you to do a lot of other things as a side-effect of using those services at all, and will cost a lot for what you need.
On the other hand, if you need to be highly elastic, have completely managed RDBMs on-demand available and orchestrate networking, IAM, object storage, block storage, compute and ingress, do start out with a cloud.
Regardless of what you are building, make sure you know ahead of time if:
Example for a MVP SaaS: say you want to manage shopping lists for consumers, you might call it Shoppr and build a PWA and a app-wrapped PWA so you get immense reach. You mostly have front-end engineers, but you do know a bit about metrics and scaling.I'd say that means:
This can be built using any stack, and as long as your persistence can keep up you're golden. Don't fuck it up with an ORM that doesn't know how to create the proper indexing rules on tables and you can easily get a couple of million customers on an IaaS-only provider that just has virtual machines or containers, and only has one flavour of persistence store. Plonk Cloudflare in front of it and done.You can make this infinitely more complicated, but as an example add this feature to make this entire setup suck and the entire infrastructure incompatible with the needs of the application: international receipt scanning to recommend/autocomplete shopping lists for customers. Suddenly your requirements are expanded with:
To make your traffic bill not suck you'll need endpoints in most majo...If you need many of their services, AWS is very difficult to beat. Nobody else quite matches up with them on breadth offerings. If you don't, do not use AWS due to the cost and the mental overhead of management.
If you just need to spin up some servers and want them to be fast and cost effective, Hetzner is the current champ in the US market (their new US datacenter). DigitalOcean, Linode and Vultr are entirely reasonable options after Hetzner.
If you have a $50-$100 / month or less budget, know how to set up and secure a linux server, and want to stretch your dollars as far as you can with a top notch cloud provider, Hetzner wins at present. And then throw Cloudflare out in front of it until or unless you can justify paying for a service. Keep your costs low, keep your infrastructure simple, keep your runway long, focus on selling selling selling (customers).