Ask HN: How has your experience with AWS Lambda been?

The idea looks good on paper but in practice it is pretty pricey and impractical. You get free sandboxing, scaling, and stateless deployment out of the box but are charged by the second. It's really optimized for simple web apps (or microservices) - think backend for form processing or resizing a picture. Anything moderately complex e.g. python scientific software is impossible due to the 50MB compressed app size limit (including all your dependencies), if your app takes more than a couple seconds of compute time, it will burn a hole in your pocket.

Usage: Algorithmic trading, Alexa voice apps, Brokerage price matching

http://KloudTrader.com

I’m also curious about it. My friend who have used it told me it’s difficult to debug. Is it true?

Expensive, especially if you are utilizing API Gateway. Otherwise, it's amazing.

I never really understood why people would want to build "applications" on top of Lambda. I have always been under the impression that the primary function of Lambda is to assist in programming the AWS environment (in conjunction with SNS and SQS).

In my experience so far (devOps and managing infrastructure) it is a great way to run one-off scripts in Python, JS and now Golang to invoke other services (eg. step functions, or other lambdas), publish to SNS topics or describe tables in dynamo, you get the gist. The advantage is that you only get charged for the time you use it, and their free tier is generous, so our bills from Lambda are negligible.

The paradigm I use is that of treating Lambda as "glue" between services, and it has worked out amazingly well for that. Quick invocations, seemingly infinite scaling. CW logs (monitoring) could be better, but tools like Apex help with parsing them.

AWS lambda should be thought of as functions that can be scaled independently.

I can write the functions in any one of the languages that are supported and they are all interlace-able through an interface. If I need to use ML and prefer python libraries, I can do that and call python from a nodejs function or vice-versa. All this without having to worry abt keeping a server up and secure and scaling.

There are issues with loading times and cost, but easily the best thing that happened for me as a single dev proving out a product MVP.

Started using Lambda 2 years ago now and my thoughts of it have evolved over time. I was initially frustrated by it but a lot of the pain points have disappeared over time (especially no Python 3 support; now supports 3.6).

Some random thoughts and gotchas in no particular order:

1. Don't do your own deployments. Use Serverless.js (https://github.com/serverless/serverless). It's in JS but that doesn't lock you into the nodejs lambdas, it can deploy to any of the runtimes.

2. Use Zappa if you want to host a "serverless website". But frankly, serverless websites are only for hobby projects right now.

3. CPU vs Memory scaling sucks. You have to scale both at once; you can't do high CPU/low memory. This hurts the wallet for compute-intensive tasks.

4. 5-min hard timeout on all functions. Don't use Lambda for long-running tasks.

5. Low-CPU lambdas (128MB tier) are terrible at cold starts if you have lots of stuff to import (which WILL BE THE CASE for Zappa applications!). If a cold start exceeds your timeout (which defaults to 30 sec), your Lambda will never complete and never get out of cold start.

6. If you're playing with S3, PUTs will hurt your wallet. A common application of Lambda (the famed "resize images" example) has a workflow that looks like this: GET on API gateway, PUT to s3, triggers an event which fires a Lambda, lambda processes payload, PUTs result back to S3. If processing your payload is fast, the PUTs in this case could be the most expensive part of your flow.

7. X-Ray is atrociously confusing.

8. No cloudwatch metrics for memory usage. Had to do my own using log metrics. Dumb.

9. There's a cool versioning system for the code artifact, kinda like ECS has. It's really nice to use except that unlike ECS it doesn't have native lifecycle rules, which means you have to do your own cleanups. Dumb.

10. Lambda patterns can be implemented more cheaply and more efficiently using tasks queues and autoscaling EC2s. Amazon's "efficient" compute allocation is outweighed by the margins on Lambda pricing. As with other Amazon services, you're paying for the system to be managed.

11. Aurora Serverless is super interesting, but FWICT nowhere near ready for prime time.

12. API gateway is a solution looking for a problem. If you have to use it (eg. you must trigger your lambda through an http request and you don't want to run a web server), use it in proxy mode.

13. Golang lambdas are very promising. I want to try them out.

14. There's a native canary system in Lambda/APIGW nowadays. Also want to try it out. Had to roll my own before it existed and now too scared to touch the production system.

15. If you hit concurrency issues, your problem might not be load, it might be throughput. Is your DB slower than usual? Common pattern: Increase in Lambda requests hitting your DB, slows your DB down, which in turn slows your lambdas down, which causes you to hit concurrency limits. All these metrics are tightly related.

16. Your tests should not assume a Lambda environment unless your app actually depends on the lambda environment (it generally shouldn't).

17. Use this thing: https://github.com/lambci/docker-lambda - Among other things, you can use it to build binary libs/dependencies you want to run on Lambda (cairo for example).

18. Use this thing: https://github.com/spulec/moto -- And also this thing: https://github.com/localstack/localstack

19. Cloudwatch logs suck. If you want your app to be debuggable, make sure you know how to find logs ...

We've been using AWS Lambda function to asynchronously render PDFs, transform data or execute on different pipeline triggers with success for a few years now. The horizontal scaling that avoids having a pressured queue during high-traffic spikes is pretty nice.

Another case we've had success with is transforming and relaying webhooks from different services (relaying events into Datadog for services that don't have a native integrations) or acting on occasional webhooks that don't happen too often (creating a default set of labels on a new Github repo).

If you're doing anything asynchronously, it's pretty nice. Be aware that the amount of memory you allocate to your functions is directly proportional to the CPU amount (and price) you're given. There's also the caveat that invocations perform _at least once_ but not necessarily once. A few months ago we saw a significant increase in repeat lamdaba invocations/retries even when the original execution was successful.

Background: I just started a systems integration project that leverages Lambda pretty heavily as part of a messaging API. Behind an API Gateway Lambda functions will be responsible for processing initial message requests, routing the requests to certain SQS queues based on certain criteria, and then making requests to other systems as messages come into the various queues. Each of these parts of the process have their own set of functions that will run independently. I’m writing my Lambda functions using C#/.NET Core 2.0 (I started with Go initially but I’ve used C# before and find that I personally prefer it).

So far I think the development process has been good. Deployment has been okay, and I’ve found it easier to deploy my .NET functions vs the old Go functions.

Logging/debugging is a bit of a chore but not awful. Cloudwatch alarms are fairly useful for knowing when something critical happened, but you still have to dig in fairly deep to find the relevant error information.

Performance wise, Go was really fast even with the smallest amount of memory. Cold starts were around 1s and requests after that were around 30-40ms on average. C# is substantially slower unless you bump up the memory.

All that being said, overall I really enjoy developing Lambda functions and not having to worry about containers, etc., because I’m not a DevOps person. I would definitely use it again for other projects, and probably will.

I gave it a shot because my api was fairly simple and I really did not want to go through the hassle of setting up a server. The overhead turned out to be massive, like half a second per query. So I ditched it for node.

I used API Gateway and Lambda to serve 20M API calls last month on ipdata.co for a little over $100.

Something to note, latencies decrease with increased usage due to fewer cold starts. I've seen latencies as low as 17ms in our busiest region with ~40ms in less busy regions.

Running a pretty simple app on Lambda: take DMARC reports, parse them, ship them off to somewhere useful. Same app deploys to Lambda+SES and Google App Engine.

Lambda app: just keeps chugging. GAE app: perpetual disaster.

I presented my solution to import/process relatively big files of data to a Data Lake on Berlin Summit last year and importing data from SNS to the some Data Lake in the AWS Hamburg Meetup Group. The experience was VERY good. It scales, very low price, ...

As a HTTP endpoint, the cold-start is a pain, but after I'm getting answers in 10-20 milli-seconds. You may contact me directly to tell you how to go around this cold-start issues.

I tried using API Gateway and Lambda to create some dumb endpoints late last year (most just return static data with some returning updateable data).

All endpoints/operations exposed through API Gateway.

Thought it would be a lot simpler to set up than it was. It's pretty convoluted doing API Gateway -> Lambda, especially when you make changes.

I had thought it would be way simpler. single click define endpoint, then just write some code.

The AWS UI most definitely got in the way trying to accomplish this.

I like lambda for API's when used along side the serverless framework. Makes deployment of a simple SPA and rest API really simple as you don't need any manual configuration of AWS everything is setup for you via cloud formation. Also makes hosting really cheap as you don't even need one EC2 instance running for minimal support when not in use.

Only real downside so far is there can be some spin-up delay of a couple of seconds for the first consumer of your API. Not a big deal in my scenario of mainly a backoffice type app.