Serverless a simple idea that copying a snippet to the cloud and auto-scale.
You know we used to place the code in .php files and ftp it and call it a day. There were tons of php hosting (anyone remember Dreamhost?) that can auto-scale in some way and bill by page views.
I don't really agree with the critiques. E.g. GCP will run a Docker container in a serverless fashion, so you can run any language you like using that.
The big issue, from my point of view, is that the programming model takes all the problems of microservices and adds more. Microservices make even trivial features into a distributed system, bringing all the difficulties of reasoning about code that distributed systems entail. Then add an opaque runtime and almost certainly a number of vendor specific services to handle state and other features the serverless model cannot address directly, and you have a recipe for slow development and frustrating debugging.
I don't see serverless as having failed at all. FAAS is a great model, you can run a functional monolith and easily break pieces off or integrate stuff in different languages without changing patterns. Doing it with cloud services like lambda can be expensive but if you have kube setup it has the best of all worlds.
Same here! We're fans of Cloud Run at my company, but it'd be great if they made it easier to pull messages from Pub/Sub using Cloud Run. Sometimes you just don't want push.
That’s interesting to hear. What problems did you have with push? Having used this setup before, I found it pretty easy and the only time I found myself reaching for pull (on a service deployed to K8s rather than cloud run) was to achieve greater throughput. Was it the same story for you?
Maybe I'm doing something wrong here, but one issue I've faced when integrating third-party external systems with Pub/Sub and Cloud Run is the lack of an easy way to throttle the data flow when the external system falls behind. It would be really helpful if the Cloud Run service could easier support pull from Pub/Sub instead, allowing it to consume messages at a pace the external system can handle.
I could of course do flow control myself in the Cloud Run service, but it'd be nice if I could just switch to pull.
Or if you can take a bit of up-front setup cost, running it on GKE with Autopilot means you've got many of the benefits of lambda on a K8s cluster (e.g. immutable container-driven development, no need to manage long-running VMs) and only pay for used resources (not idle machines). The main caveat is no scale-to-zero, but as mentioned previously, that's unlikely to happen.
How come? I see Autopilot is $0.10 per cluster hour, or ~$75 a month, add the standard GKE cluster management overhead of $150, that's $225 a month for a cluster which rounds to zero given any reasonable load. Then in my experience GCP VMs have always been price competitive with basically any other cloud provider.
The issue we had was getting the bin-packing right on our cluster, and we often ended up with 20% unallocated capacity per VM. Autopilot basically solves that (and probably pays for itself with only a few VMs) as you only pay for the K8s defined resources, and therefore only have 1 level of cost optimisation at the pod level, rather than 2 at the pod and VM level.
<3s is our target 95% is <2.5s. We have a keep warm function that always keeps 5x "warm"-ish so we rarely hit cold starts. Keep warm is not fool proof as AWS got smart about people constantly doing keep warms, but it definitely helps.
That's a very fat function to run on Lambda. What's the advantage compared to run the same container in Fargate? Compared to a VM I guess you're saving the sysadmin time.
Do you have some recurring activities to babysit Lambda or is it zero maintenance for your team?
Quite low. We're a b2b application, so each customer is worth a lot but very very underutilized CPU and relatively few customers.
Main advantages: Server $$ significantly lower than CI/CD; Bottleneck becomes DB, can have 1000x instances no problem (but never use more than 5x). No EC2 to manage just deploy the docker.
It gives you the flexibility to easily swap in pieces in another language without making architectural changes while being as simple to work with (after devops work) as a monolith. If you have an orchestration setup to manage your functions rather than using Lambda/etc it doesn't cost any more than a monolith either (aside from competent devops salary).
> It gives you the flexibility to easily swap in pieces in another language
Which you can also do with a Webhoster. Well, it depends a bit on what languages or environment they offer of course.
> without making architectural changes
I guess those pieces have a way to communicate with each other? So the point here is more that you are forced to work with an architecture which is from the beginning designed for pluggable pieces.
I should've put context, but we are a b2b and barely have any CPU utilization and mostly in bursts.
Normally we would have at minimum 2x Fargate "mediums" sitting around for redundancy. ~US$130/mth now our server costs <$5/mth. Our CI/CD is WAY more expensive.
Article is 3.5 years old, best put the year in the title, OP!
I think the definition of serverless is too narrow (AWS lambdas/ Azure functions) and that serverless is really "I want to build apps, not manage infrastructure". That's not the same thing as putting everything onto functions.
I have a monolith we're preparing to move onto a managed container system (prolly AWS AppRunner). I don't want to manage K8 if I can help it, our app doesn't need complex server architecture.
Personally, I think it's just the next layer of abstraction up. Some won't benefit from serverless in the same way some are better off with in-house tin. I know some that need custom chipsets in-house, so can't even buy a stock rack server! However, many many web apps don't really need the control and will probably use facets of serverless over time. But it is still a revolution for old people like me that just don't want to manage servers anymore!
AWS right now feels like the most lagging in serverless containers.
If you get a chance, try out the DX with Google Cloud Run or GKE AutoPilot. Building and shipping to GCR is fast enough that it feels like a local build-run worflow. Google Cloud Run jobs are also fantastic and you get a pretty hefty free monthly grant (~60h of compute).
Both Azure Container Apps and GCR are true scale to zero whereas AppRunner is not and always maintains a minimum monthly baseline cost (~$5-6).
AWS feels the most behind in terms of its serverless container workload experience (I use AWS every day professionally, but use GCP and Azure for side projects).
> its roots can be traced all the way back to 2006
Serverless was already a thing in 70s, in the mainframe era. In the 90s the pendulum moved to servers and server farms and nowadays is sliding back.
> The Problems With Serverless:
> Limited Programming Languages
> Vendor Lock
> Performance
None of this is a real problem, the only thing that matters is the cost-effectiveness, and currently (2024) it's stil cheaper to run most of your software on "traditional VMs".
I think appengine is one of the most underrated "serverless" offerings out there. I deploy everything on it, with almost zero concern about infra or server architecture
My use cases are probably regarded as small/medium, which need a handful of instances running, which you can effectively scale down to zero with Googles free tier (basically allows you go goto 1 instance for free), so cost has not been a major consideration.
I hate Azure functions. It's the weirdest shit I ever had to work with and the fact that most documentation relies on clickety click examples on vscode or some javascript tool I have no access through my build and deployment environment infuriates me more. Please for the love of god give me a proper server.
The pricing model of serverless platform is enough for me to not want to use them. Being charged for every little thing your app does is crazy especially when you see horror stories of runaway bills because a function has a simple bug in it that you didn't catch and overnight it ran up your bill so high you can't pay it and are now praying the company feels bad for you.
I'm scared to use serverless platform unless I don't have to put in my credit card lol.
I feel safe paying the same price every month despite my app growing. Servers are powerful these days and are very simple to maintain and scale using cloud platforms because they do it all for you lol.
GCP's max instance count is helpful to a certain extent, but this doesn't completely protect you unless from DDOS attacks. If you have 3 nodes that can all handle 1000 requests per second, and those requests are doing something expensive the max instance limit doesn't help you.
Although I guess if you have 3 nodes that can only handle 100 requests each the damage of a DDOS would be much more limited.
I enjoyed Google's GCP for quite a long time until they removed the ability to cap expenses with a budget limit. (It used to be that if you hit your budget limit you can make your site error out). Now everyone on GCP is one DDOS away from a nightmare cloud bill. I'd rather use a traditional server and be able to sleep at night. I moved all my sites and client sites off GCP.
The most annoying part is that Googles infrastructure for cloud computing is so much better than the others if you're willing to work within their ecosystem. Simple deployments, version management, rollbacks, etc... There is nothing quite like it. (Im not saying there aren't competitors, just that Google seems easiest to use)
> The concept of letting users pay only for the time that code actually runs has
> been around since it was introduced as part of the Zimki PaaS in 2006, and the
> Google App Engine offered a very similar solution at around the same time.
You could argue the way you architect "serverless" services feels very similar to the way we built CGI scripts in the 90s. If you kept those scripts small, everyone of those would have been the equivalent of a serverless function today.
Didn't old-school mainframes with discrete transistor CPUs use time-share mechanisms and charge enough for the time that people cared about minimizing it? UNIX goes back 50 years, but IIRC there was a substantial amount of "pre-history" before that too.
It stalled the second people realised the actual costs are significantly north of using your own ifrastructure, and is often burried in a line like "oh by the way you'll need a NAS Gateway and it'll cost you an aditional $50k a month"
Stuff like Fargate, GKE Autopilot and Cloud Run I would also consider "serverless" just in a weaker sense. I can see these platforms having pretty wide appeal for workloads that don't benefit from stronger control over the underlying VMs and networking.
The biggest selling point that seems somewhat legitimate is a bunch of these have scale to zero capability. This is pretty useful for on-demand code that is required very infrequently and for which cold-starts are a non-issue.
I think where they fall down is that once you run up against one of their hard constraints is that the cost of moving to something without those constraints acts as a high enough activation energy to instead contort the application architecture to work around said constraints. This I feel is very bad and leads to horrible results, especially when aforementioned platform is FaaS.
In practice if you know you are going to want to run any non-trivial long-running or stateful services you are just better off going k8s from the start. The API is good, the managed options are good, life is just less complicated when you don't need to deal with proprietary bullshit.
>The biggest selling point that seems somewhat legitimate is a bunch of these have scale to zero capability. This is pretty useful for on-demand code that is required very infrequently and for which cold-starts are a non-issue.
I don't quite understand this. You can pay for 1 physical/virtual server and have apache serve 2 different domain names, right? Can't you just drop your infrequent app there so it shares resources with, I don't know, your more regularly accessed website?
The only scenario I can imagine this would be useful is if you own zero servers or the code is incompatible with the server you do own for some reason.
Well you do understand it, enough to work out how to live without it. :P
That said there are some key things a scale to zero serverless app has that solution does not.
1. Isolation. Your infrequent function may very well be infrequent but it's execution could then impede other code executing on that box (if it's computationally or I/O intensive for instance), or expose other code on the box to security vulnerabilities.
2. Independence. Similar to above but more addressing logical segregation. Once you are sharing a server or say piggybacking on a monolith as an extra endpoint there is now a lot of conflated concerns. For instance auto-scaling algorithms etc may not work well with the sort of intermittent/infrequent execution and it would be much better living in it's own context for that reason. Other things that end up all jumbled up: Cost accounting, IaC/deployment/CI/CD, ownership, observability, etc.
Generally all of this still isn't enough to justify serverless but sometimes it could be... but in most of those cases just eating the cost of 1 replica and autoscaling on k8s is usually better.
When did go fast break things become go fast be stupid.
Serverless is like driverless.... Sure you might be in the backseat but there is a chauffeur up front and you pay a lot of money for that.
Unless your not paying for it (directly) --- serverless makes a LOT of sense if you have light weight tasks that can be distributed and you send it to "excess" infrastructure you already own/run.
For me, serverless solved a problem at the opposite side of the spectrum of “scalability”. It helped me to start new projects very easily.
Since I like React and the Next framework, Vercel is a one-stop shop for me now. With their new storage feature, I can start creating a fullstack web app very easily. Frontend, functions, and Postgres, all hosted one Vercel and very easily integrated.
Having a Postgres DB makes it very easy to export to a proper server in the future, if I need it
>resources used on serverless frameworks are typically paid for by the minute (or even by the second). This means that clients only pay for the time they are actually running code. This contrasts favorably with the traditional cloud-based virtual machine, where often you end up paying for a machine that sits idle much of the time.
I'm going to ask something very stupid because I'm very ignorant but... when do you need this?
In my mind you have scenario 1: the app never runs because no users, so running it 24/7 is a waste. In this case why cloud? Pay $5 a month for a simple web host?
Scenario 2: you are hitting 100% on a server and you need a second server but that second server will only use 1% most of the time. Again, why not just upgrade the server with your web host?
Scenario 3: you have a bajillion users on peak hours and a tenth of a bajillion on other hours. I assume you would have a business model that affords you to hire someone to manage scaling your infrastructure?
most critiques do not apply if you look at providers that have open source runtimes. for services running on deno or workerd/cloudlfare workers you can add your own runners on big cost effective servers that allow longer execution and serve core locations or certain workloads with the same code. "limited languages" still applies as these are limited to js/wasm languages. "not being able to run entire applications" still applies only if you include the database servers, as these will need special operation. (but running databases was most of the time a seperate thing from the core application anyways)
We cut one system’s cloud bill by two thirds by moving to virtual private servers. The amount of time/effort serverless and other x-as-a-services were saving in maintenance was not worth the cost and other constraints they caused.
72 comments
[ 1.6 ms ] story [ 42.8 ms ] threadYou know we used to place the code in .php files and ftp it and call it a day. There were tons of php hosting (anyone remember Dreamhost?) that can auto-scale in some way and bill by page views.
The big issue, from my point of view, is that the programming model takes all the problems of microservices and adds more. Microservices make even trivial features into a distributed system, bringing all the difficulties of reasoning about code that distributed systems entail. Then add an opaque runtime and almost certainly a number of vendor specific services to handle state and other features the serverless model cannot address directly, and you have a recipe for slow development and frustrating debugging.
I could of course do flow control myself in the Cloud Run service, but it'd be nice if I could just switch to pull.
I think the serverless revolution is here just not in the way people originally dreamed.
The issue we had was getting the bin-packing right on our cluster, and we often ended up with 20% unallocated capacity per VM. Autopilot basically solves that (and probably pays for itself with only a few VMs) as you only pay for the K8s defined resources, and therefore only have 1 level of cost optimisation at the pod level, rather than 2 at the pod and VM level.
Do you have some recurring activities to babysit Lambda or is it zero maintenance for your team?
Main advantages: Server $$ significantly lower than CI/CD; Bottleneck becomes DB, can have 1000x instances no problem (but never use more than 5x). No EC2 to manage just deploy the docker.
Which you can also do with a Webhoster. Well, it depends a bit on what languages or environment they offer of course.
> without making architectural changes
I guess those pieces have a way to communicate with each other? So the point here is more that you are forced to work with an architecture which is from the beginning designed for pluggable pieces.
Normally we would have at minimum 2x Fargate "mediums" sitting around for redundancy. ~US$130/mth now our server costs <$5/mth. Our CI/CD is WAY more expensive.
"your code that handles HTTP requests must be able to startup, run, and shutdown effectively instantly"
aka "serve your HTTP without bloated long running processes."
aka `cgi-bin` scripts.
I think the definition of serverless is too narrow (AWS lambdas/ Azure functions) and that serverless is really "I want to build apps, not manage infrastructure". That's not the same thing as putting everything onto functions.
I have a monolith we're preparing to move onto a managed container system (prolly AWS AppRunner). I don't want to manage K8 if I can help it, our app doesn't need complex server architecture.
Personally, I think it's just the next layer of abstraction up. Some won't benefit from serverless in the same way some are better off with in-house tin. I know some that need custom chipsets in-house, so can't even buy a stock rack server! However, many many web apps don't really need the control and will probably use facets of serverless over time. But it is still a revolution for old people like me that just don't want to manage servers anymore!
If you get a chance, try out the DX with Google Cloud Run or GKE AutoPilot. Building and shipping to GCR is fast enough that it feels like a local build-run worflow. Google Cloud Run jobs are also fantastic and you get a pretty hefty free monthly grant (~60h of compute).
Both Azure Container Apps and GCR are true scale to zero whereas AppRunner is not and always maintains a minimum monthly baseline cost (~$5-6).
AWS feels the most behind in terms of its serverless container workload experience (I use AWS every day professionally, but use GCP and Azure for side projects).
I guess when this was written (3+ years ago) a lot of people were still high on the marketing hype so the reality seemed flat.
But I think we got some nice options.
(Which, of course, aren't silver bullets that solve all the problems.)
Serverless was already a thing in 70s, in the mainframe era. In the 90s the pendulum moved to servers and server farms and nowadays is sliding back.
> The Problems With Serverless: > Limited Programming Languages > Vendor Lock > Performance
None of this is a real problem, the only thing that matters is the cost-effectiveness, and currently (2024) it's stil cheaper to run most of your software on "traditional VMs".
I'm scared to use serverless platform unless I don't have to put in my credit card lol.
I feel safe paying the same price every month despite my app growing. Servers are powerful these days and are very simple to maintain and scale using cloud platforms because they do it all for you lol.
GCP lets you set a max instant count in Cloud Run. It’s in the first screen of setting up a service. It really doesn’t get much simpler than that.
Still a big, complex platform with tons of features and things to spend money on, but much more friendly than some others (looking at you, AWS)
Although I guess if you have 3 nodes that can only handle 100 requests each the damage of a DDOS would be much more limited.
The most annoying part is that Googles infrastructure for cloud computing is so much better than the others if you're willing to work within their ecosystem. Simple deployments, version management, rollbacks, etc... There is nothing quite like it. (Im not saying there aren't competitors, just that Google seems easiest to use)
We do all of this with Azure
> The concept of letting users pay only for the time that code actually runs has
> been around since it was introduced as part of the Zimki PaaS in 2006, and the
> Google App Engine offered a very similar solution at around the same time.
You could argue the way you architect "serverless" services feels very similar to the way we built CGI scripts in the 90s. If you kept those scripts small, everyone of those would have been the equivalent of a serverless function today.
The whole dynamic web was bunch of PHP scripts in your home folder.
Even the early Amazon was a collection of Perl scripts glued together.
What's old is new again.
I think FaaS turned out to be a largely bad idea.
Stuff like Fargate, GKE Autopilot and Cloud Run I would also consider "serverless" just in a weaker sense. I can see these platforms having pretty wide appeal for workloads that don't benefit from stronger control over the underlying VMs and networking.
The biggest selling point that seems somewhat legitimate is a bunch of these have scale to zero capability. This is pretty useful for on-demand code that is required very infrequently and for which cold-starts are a non-issue.
I think where they fall down is that once you run up against one of their hard constraints is that the cost of moving to something without those constraints acts as a high enough activation energy to instead contort the application architecture to work around said constraints. This I feel is very bad and leads to horrible results, especially when aforementioned platform is FaaS.
In practice if you know you are going to want to run any non-trivial long-running or stateful services you are just better off going k8s from the start. The API is good, the managed options are good, life is just less complicated when you don't need to deal with proprietary bullshit.
I don't quite understand this. You can pay for 1 physical/virtual server and have apache serve 2 different domain names, right? Can't you just drop your infrequent app there so it shares resources with, I don't know, your more regularly accessed website?
The only scenario I can imagine this would be useful is if you own zero servers or the code is incompatible with the server you do own for some reason.
That said there are some key things a scale to zero serverless app has that solution does not.
1. Isolation. Your infrequent function may very well be infrequent but it's execution could then impede other code executing on that box (if it's computationally or I/O intensive for instance), or expose other code on the box to security vulnerabilities.
2. Independence. Similar to above but more addressing logical segregation. Once you are sharing a server or say piggybacking on a monolith as an extra endpoint there is now a lot of conflated concerns. For instance auto-scaling algorithms etc may not work well with the sort of intermittent/infrequent execution and it would be much better living in it's own context for that reason. Other things that end up all jumbled up: Cost accounting, IaC/deployment/CI/CD, ownership, observability, etc.
Generally all of this still isn't enough to justify serverless but sometimes it could be... but in most of those cases just eating the cost of 1 replica and autoscaling on k8s is usually better.
Serverless is like driverless.... Sure you might be in the backseat but there is a chauffeur up front and you pay a lot of money for that.
Unless your not paying for it (directly) --- serverless makes a LOT of sense if you have light weight tasks that can be distributed and you send it to "excess" infrastructure you already own/run.
Since I like React and the Next framework, Vercel is a one-stop shop for me now. With their new storage feature, I can start creating a fullstack web app very easily. Frontend, functions, and Postgres, all hosted one Vercel and very easily integrated.
Having a Postgres DB makes it very easy to export to a proper server in the future, if I need it
>resources used on serverless frameworks are typically paid for by the minute (or even by the second). This means that clients only pay for the time they are actually running code. This contrasts favorably with the traditional cloud-based virtual machine, where often you end up paying for a machine that sits idle much of the time.
I'm going to ask something very stupid because I'm very ignorant but... when do you need this?
In my mind you have scenario 1: the app never runs because no users, so running it 24/7 is a waste. In this case why cloud? Pay $5 a month for a simple web host?
Scenario 2: you are hitting 100% on a server and you need a second server but that second server will only use 1% most of the time. Again, why not just upgrade the server with your web host?
Scenario 3: you have a bajillion users on peak hours and a tenth of a bajillion on other hours. I assume you would have a business model that affords you to hire someone to manage scaling your infrastructure?
What am I missing in this picture?