Ask HN: Skeptical about my company going “full serverless”. What am I missing?
An Azure Cloud Architect has joined our company. They are recommending that all our web apps that are currently deployed in Azure App Services/ VMs should be (gradually) split up into Azure Functions going forward.
I'm skeptical - I was under the impression that serverless was for small "burstable" apps with relatively low traffic, or background processing.
The two products I work on are both REST APIs that send and receive data from a user interface (react) with roughly 60 API routes each. They have about 100 concurrent users but those users use the apps heavily.
The consensus on the internet seems to be "serverless has its use cases" but it's not clear to me what those use cases are. Are the apps I'm working on good use cases?
149 comments
[ 4.0 ms ] story [ 246 ms ] threadIf there is no local state than serverless is a feasible solution, if not the best. If there is then you need to find some substitute for that local state and the case for serverless is much worse.
It's common for servers to take a while to start up, it's common to see the first request to each endpoint take a bit longer, it's a common optimisation to add keepalive to downsteam services, or tweak your database connection pooling. These issues are normally straightforward optimisations, and startup time isn't usually too much of an issue. With FaaS platforms these become significant hurdles that take engineering work to overcome, require introducing more services, more cost, etc.
Whether or not it's for you has a lot to do with what's important to you. How much weight you put on runtime cost, versus ease of development / deployment, versus whatever other benefits it might bring (integrated logging, monitoring, multi-region deployment, etc). And, of course, whatever downsides it brings...like cold starts, team ramp up on how it works, etc.
What problem does migrating to a new architecture solve? Does the current deployment have scaling, maintenance, or other troubles? Going from something that's broken to something that works is one thing, but going from something that works to something else that works is pointless unless there are tangible benefits.
If the only reason is to make the Cloud Architect feel better or pad people's resumes, the correct answer is no.
What benefits does the "cloud architect" say the migration will bring? It sounds like you have a reasonable backend api setup that works. There needs to be a strong motivation to do a migration like that.
I'm also not convinced you're at a scale where you need a cloud architect, but it's hard to say from your description. I bet their main motivation is delivering a project that justifies their role.
Everywhere else, I went with traditional deployment of a monorepo.
You only have 100 concurrent users. You do not need serverless. You could serve 100x that amount easily with a simple nginx reverse proxy to your webserver.
This is almost comical enough for me to suspect that it is satire, but unfortunately there are too many examples out there of this type of thinking. It's just infra bloat and a waste of money.
My $0.02, having used serverless before. Those use cases are:
* Very very low traffic apps. POST hooks for Slack bots, etc.. Works well!
* Someone who is an "architect" can now put "experience with Serverless" on their CV and get hired somewhere else that is looking for that keyword in their CV scans.
Those are any and all uses cases.
This is absolutely a huge step backwards in terms of architecture. And methinks this architect doesn't have the broadest understanding of Azure and is reaching for the easiest tool in the toolbox.
Edit: "serverless" is a broad term. There's a difference between shoehorning every microservice into a lambda-like service (bad), and migrating from a collection of VMs to a managed Azure service that does the exact same thing (can be good).
* Good for "minimally dynamic" apps
* Easier in compliance/regulation-heavy environments (the kind that impose burdens on OS configuration and maintenance)
* Like you said, great for low traffic apps (the kind of things we usually build, toss over the fence to another department, and then never look at again)
* Also like you said, resume driven development
Lambdas and workflows are like SQS and workflows. you get a few of them just use lambdas and workflows, you have a lot of them switch to a workflow engine.
I would add, that it offers an extension point / hook-in for a variety of AWS products. I can add custom functions to be called when a Cognito user is created, updated. I can add custom functions to be called to authorize routes transiting API Gateway.
At "The Firm" - we're going heavily in on AWS for our architecture - Fargate for k8s on top of the usual RDS, S3, etc.
We've begun roll-out of an external-facing API authenticated with OAuth2. Our backing store here is Cognito and our routing happens via API Gateway. We use the API Gateway's baked-in Cognito token authorizer for incoming requests, which route to pods hosted in Fargate. These pods represent a variety of projects.
As such we have two Lambda functions providing the /oauth/secret and /oauth/token endpoints for exchange of One-Time-Password for a secret token and secret token for access tokens - they experience very low N call rates (dozens?) of times per day and don't fit into any given API we host via Fargate.
For example:
- resizing images - we can do a size per lambda all in parallel. This means we can process images quickly (with minimal latency) without having to have loads of slack memory & CPU on our backends
- queue processing - we have an app that needs to copy files from user provided URLs to S3. We do this by dumping them in a SQS queue and having a lambda fire for each queue item. Means we can do lots in parallel without filling up an EC2/fargate instance’s network port
- dynamically processing images using Lambda@Edge & Cloudfront - similar to my first one, but on the fly when requested, instead of ahead of time
- To run something at deploy-time with custom CloudFormation resources. As an example, a Lambda can be invoked during each deployment to run tests. If the test fails, the stack(s) can be automatically rolled back. Tricky to get right and not often the right tool in the bag, but sometimes useful.
- As handlers for CloudWatch events. Useful for forwarding events to third-party log services, or for taking some sort of complex action based on a given event.
AWS is nearly “all in” on Lambda at this point, so it’s a good bet that if you want to hang some kind of task off an AWS service, they’ll give you a hook to fire off a Lambda. From a cost and simplicity perspective this works out well.
Scales down to 0, so effectively no cost overnight when used infrequently, and if you use a scripting language like Python or JavaScript the startup times are in the ms for a new instance when scaling up.
When I would use K8s would be if there is heavy processing involved, so want a language like Java or C to do the heavy lifting, image processing, encoding or encrypting and would be long running.
Disagree with serverless only being for low traffic, I've used it for a a fair amount of high traffic situations and it is great for scaling up quickly.
AWS Lambda and Google Cloud Functions support containers.
> and would be long running.
Unless it's running 24/7, you're better of with serverless batch processing systems or you'll need scale-to-zero on your Kubernetes cluster.
While Lambda supports containers, you lose out one one of the largest benefits of having Amazon patch/update the runtime.
Which brings me to my opinion on the value proposition. There’s a concrete cost curve where ECS/EC2 is cheaper than Lambda in what you pay AWS, there’s a much higher inflection point where you are responsible for maintenance/operations. Under that line there’s still a place for serverless.
Lamba works really well for reactive processing, such as supporting an API. You lose that benefit if you deal with something like Java because of the slow startup times gimps the processes.
Hence, running it in an environment which will be on constantly with consistent throughput that is just a pipeline 24/7. The scenario I've used it with was GIS and satellite image processing, which would have blown through the Lambda limits and with constant images being published meant that it was essentially processing 24/7.
I am an old school kind of engineer, and if you tell me your web requests take 2s instead of 20ms due to an architectural decision you made that doesn't have any other strong upsides, I would agree that that is insanity.
Then, the email verification is checked by a second function. You click the link, it updates a field in the DB and redirects to the main page.
login is the same, we validate the creds and give a token.
And then the only infra we maintain is the stuff we have always on and serving mass requests. There's a lot of stuff that doesn't run that often that is more or less a single function and running a fleet or redundant machines for that doesn't really add value.
It just depends on how your app is designed as to if it makes sense.
We were hosted on AWS, and pretty much only used Lambda + API Gateway + DynamoDB.
I think Serverless is good in some areas, S3 and Dynamo are both good products for example.
I have a few big issues with serverless: 1. It is harder to develop for. Sure you get to ignore server configuration but honestly a well made infra team should be removing that concern for the development team anyways. The problem is when you are running it locally you so rarely can actually run the code. So setting up things is annoying, especially when you get into the final stage of serverless which is some object lands in SQS which fires a lambda, which puts puts another object in another queue which fires another lambda which load s3 which writes to a db, etc. This all ends up more complicated than just writing an application for it, but its harder to develop for it. Often the only way to actually run this stuff ends up being setting up the whole infra in the cloud and running it through that way, so that means dealing with deploys, and you lose a lot of debugging ability.
2. It doesn't save money. A single lambda that runs quickly on some event does save money vs a server all of the time. But most companies seem to over-provision servers so that's easier. But once you include the prod environment, dev environment, and the serverless things running all of the time, it does not save money since often 100 lambdas could be a single instance.
3. It doesn't save time. Developer messing around with setting up hundreds of new services and the corresponding rules and configurations and deployments and cicd pipelines , I don't think saves dev time vs a normal well maintained infra with servers and a good a cicd pipeline. Often the time savers are vs like manually configured bare metal servers moving to serverless, but there are better ways to save time.
You should never need leave your IDE to complete a change>execute>change inner loop. https://modal.com/docs/guide/ex/hello_world
You can scale down app service/VM based infra either on a timer or in response to metrics, so it's not like serverless is your only option if cost is the motivator.
1. The same stocks in $cloud_platform_provider that your architect has bought.
2. A bunch of certifications for $cloud_platform_provider so you also want to lock everyone and their mother down into that platform.
But if you compare Azure Functions with stored procedures in a DB, then it's pretty cool to have a kind of hot swapping at the function level.
I'd be cautious, but with a gradual migration there's hopefully time for reflection as well. Going 100% on anything is rarely a good idea, so hopefully your architect isn't religious about this.
If I was starting from scratch, I'd use serverless. If you're migrating everything, I think that is a giant project that needs justification. I'd ask, "What specific current problem do you have that it would solve?"
Cold boot is only a (minor) issue on the first hit, that's quickly amortized.
This is most likely a waste of time and the "cloud architect", like most cloud proponents, has no fucking clue what they're talking about.
Just don't get attached to company and don't work late.
- Easily scalable/autoscaling
- Drastically reduced operations/maintenance/devops overhead
- CI/CD can be much simpler
- Observability is built in (metrics, logging, alerting is built in)
- Built in connections to other cloud products
We've used AWS Lambda for about 4 years, and it's been so good and so cheap that I'm shifting literally everything (except Redis) to serverless. Also, GCP has a better serverless offering (Cloud Run, Spanner), so we're switching from AWS to GCP to take advantage of that. I bet we're going to see a massive cost reduction, but we'll see.
Things I like about serverless (again, from the perspective of a very small startup, with 5 engineers, and me being the primary architect):
* It's so liberating to not worry about EC2 servers and autoscale and container orchestration myself. All our Cloud Formation templates add up to around 3,000 lines, which maybe doesn't sound like a lot, but it's a lot. There are tons of little configuration things to worry about, and it adds up. (Not to mention the sheer amount of time it took to learn.) ECS Fargate takes care of some of this, but it doesn't autoscale based on demand or anything (not without settings things up yourself). (This is a big reason why I want to switch to GCP: Cloud Run is like Fargate in that it runs containers, but unlike Fargate it autoscales from 0 based on load.)
* It's very cheap in practice, at least for loads like ours that respond to events: API services that sometimes see a lot of use and sometimes see very little use; queue consumers sometimes have a lot to do and sometimes have very little to do. AWS Lambda bills down to the milisecond in terms of resolution, and GCP Cloud Run/Cloud Funcitons bills down to the next 100 miliseconds. These are very fine resolutions and for us at least, we've seen costs be small.
* For database serverless products (like DynamoDB for example), it's very liberating to never have to think "Hm, do we have enough CPU provisioned?"
Things I don't like about serverless
* Pushing source code sucks. Lambda will just one day decide your version of Python or whatever isn't good enough and force your customers to upgrade all their user-written code to the latest Python version. (But! Cloud Run supports containers, and so this won't be a problem.)
I am starting to evaluate AWS for GCP for serverless. What, in your opinion, makes GCP better? Is the comment in the context of containers or functions?
GCP Cloud Run is like the best of both worlds between AWS ECS Fargate and AWS Lambda. (Yes, the comment is in the context of containers. Sort of.)
* Like Fargate, Cloud Run hosts containers and takes care of figuring out where they actually live. Unlike Fargate, you don't have to say exactly how many containers you want running at once; GCP will automatically scale the # of containers up and down based on HTTP load and will scale down to 0. This should make Cloud Run cheaper than Fargate. (If you want to hook up Fargate to a webserver and you don't have autoscale figured out, you'll have to keep a lot of workers alive doing nothing.)
* Like Lambda, Cloud Run bills by the amount of time spent processing at least one request. But unlike Lambda, Cloud Run lets one container handle more than one request at a time (it sucks to have to spin up a lot of Lambda invocations that do a bunch of IO). Web servers that are good at concurrency shine here. This should save money.
* Cloud Run has more generous limits in many respects than Lambda. Cloud Run lets you set up SIGTERM hooks, so you can do some cleanup logic in your container (to e.g. write performance data to a timeseries table or whatever).
That's Cloud Run. On the database side: GCP Firestore is very interesting and we're going to build a big feature around it. AWS has nothing like it. On the queue side of things: We're planning to build around GCP Cloud Tasks; We've more or less built Cloud Tasks ourselves using a mix of MySQL and AWS SQS (and it was hard and we haven't done a good job).
I'd love to start a Discord or something to discuss these thoughts more. It's so hard to get good practical information for system architects/CTO types who just need to hammer stuff out.
I don't profess to be a GCP expert so it's hard for me to make a judgement call on what's better. I can, however, say that most of this post ignores some of the real serverless power provided by AWS. AWS AppSync, AWS API Gateway, DynamoDB, CloudFront Functions, Lambda@Edge. It also makes comparisons that are not very fair.
Ah I see, launched originally in May 2021. That's probably why they weren't aware of it. Yes, this looks cool. Very much what I was looking for.
The differences that I can see are...
* AWS App Runner lacks an advertised free tier. Not a big deal for all but the smallest projects though.
* AWS App Runner bills rounded up to the next second, whereas GCP Cloud Run rounds up to the next 100 millisecond.
* AWS App Runner doesn't charge per request (?!), whereas GCP Cloud Run charges $0.40/1M requests.
* AWS App Runner has fewer CPU/RAM configuration options. The lack of low end options may be a blocker for us.
* It's cheaper than GCP Cloud Run - $51.83 @ vCPU/1GiB, but 2GiB minimum, in Runner vs $69.642 @ 1 vCPU/1GiB (v1) $97.50 @ 1 vCPU/1GiB (v2) in Cloud Run.
* I'm confused by the networking model. In App Runner, you have to make an ENI for your App Runner service to access your VPC? Weird. There's some extra cost there I think.
Things that I can't determine based on the documentation...
* Does App Runner support committed use discounts?
* Does App Runner throw a SIGTERM before shutting the container down? I hope yes but I can't find docs on it.
* Is there a file system accessible on App Runner and is it just in memory or is there actually a disk to write to?
* The quotas & limits page on App Runner feels incomplete and I'm left with a lot of questions about it.
* Is there an SLA?
* In fact the documentation for App Runner just feels a little incomplete.
It looks like AWS definitely wants App Runner to be the answer to Cloud Run, but to me, it feels like it's not quite there yet.
It's also weird, that ECS Fargate lets you run a container without thinking about the server that it runs on, and App Runner does too, just with a few extra things. Why is it a whole separate service? Why didn't they just add it onto Fargate?
Re: Other services. I've only heard of API Gateway, DynamoDB and Lambda@Edge; I'll have to spend time investigating the other ones. Thank you for mentioning them!
https://cloudonaut.io/fargate-vs-apprunner/
That lack of WAF support stood out. So Googling that:
https://github.com/aws/apprunner-roadmap/issues/58
"Hello, we are looking at supporting WAF in App Runner and will have more updates on this thread going forward. "
A new company called Momento just launched that offers a serverless cache. Might be of interest to you.
https://www.gomomento.com
The deployment story looks good though, with the `sam deploy`, but for now I can't get over the developer experience.
Also, don’t put any logic in the handler itself unless it’s extremely trivial.
Been working on a lambda, dynamodb, typescript react app for about 6 months for work and it's been just mind-melting how much money and complexity we've saved switching. I'm talking like 5x the cost drop and we can more easily onboard devs to the project because it's just simpler, no need for a devops hire honestly.
The only thing that comes to mind better isolation between parts of the app, but this could be achieved with any architecture if done right.
Every team I've known that adopted Lambda + DynamoDB (or equivalents) gave up on running their app locally, adding a lot of friction to the development process.
I'm in the early stages of this rearchitecture but so far I've had no difficulty with local development.
If you need more complex cases like deploying docker containers to a Lambda function, take a look at AWS's SAM library. Also supports local dev _and_ makes deployment easy (its essentially a wrapper around Cloudformation so its very powerful).
We used GCP at our previous startup (sold ) and ran our own K8S, when it was very new (2015). There were lots of pains in those days. So when we started our current startup in 2018, we started with App Engine (flexible, which supports containers). This was fine, but lots of drawbacks. After a year or two we ended up back on K8S, using GCP's GKE (managed K8S). Our team is pretty good with K8S, so it was fine. But regardless, the little stuff adds up.
Fast forward to about 6 months ago. We had used GCP's Cloud Run off and on for little stuff, and it kept getting better. One day someone asked the question why we shouldn't just use it for everything. Everyone was a bit defensive, but we kind of stared at each other and couldn't think of great reasons (for our use case), so we tried it.
Our setup consists of a primary API service (Golang), and a dozen or so smaller microservices, mainly in Python. We even moved most of our React apps to cloud run.
6 months in, and I can't really say anything bad. We turn off scale to 0 for the services where it matters. It scales up quickly to loads, zero down time over 6 months, no troubleshooting (so. much. time. saved.), super easy to deploy, swap traffic between versions, etc.
I'm not saying it a silver bullet, nor that it's perfect for everyone... but I couldn't say enough good things about _container-based_ serverless like Cloud Run.
That said, breaking big systems down to the function level (Lambda, GCP Cloud Functions, etc) sounds like a nightmare to me. I'm sure there are ways, but that's a different ballgame. We do use FaaS for some tasks.
YMMV.
Edit: Oh, and our hosting bill went from ~$5k a month to $500 a month (in part to other things, but primarily the lack of need for big node pools.)
* Scalability is real. We have some bursty traffic, sometimes with extreme burst, and we've had no problems scaling to meet that need.
* Our traffic is still predominately during business hours in the U.S. That's an extremely important point - because our site is effectively being used for only 12 hours or so per day. The remainder of the day and on weekends it's unused. We looked at the cost of using EC2 instances and Elastic Beanstalk and the full serverless is still cheaper.
What we've discovered in our cost analysis is if you have a site that's hit 24x7, 7 days per week then you'd be better off hosting on EC2. If your traffic is constant and there's not much variability over time then it may make more sense to host on-prem. In our case we have highly variable traffic during standard business hours. Serverless is the way to go for that scenario.
Serverless functions are great if you have a lot of small services that need to be "on standby at all times".
For example if you have 5000 separate services, it doesn't makes sense to have them all running all the time if 4,000 of them have very low traffic. So one of the main benefits is that you get the ability to "increase your library of services at a very low cost". Serverless also really shines with quick stateless actions.
However, converting an app to all serverless is a huge task and for most apps it doesn't make sense.
Two major drawbacks:
1) You're bound to only the language versions that are currently supported.
2) You're writing code specifically for the platform so without a heavy lift you're "locked in"
If the goal is to go serverless and get rid of the server management, I'd suggest looking into containerizing your existing apps and deploying on a "serverless" managed service like Fargate (or your favorite cloud provider's equivalent). This approach is also lets you go to a different cloud provider if you want... or even move back to your own datacenter with no code changes.
Make sure the serverless model include all gimmick you currently have such as firewall, waf, cache, ssl termination, load balancer, current traffic levels etc.etc.