I think acloud.guru is served through lambda. I think if you go the single page app route with S3 + cloudfront CDN + API gateway + lambda (so many words/things) you can deliver a really high quality product. You can also cache responses out via elasticache. It's not perfect but the development velocity and potential is really exciting.
I was curious about the same thing and setup a jvm lambda to test. The setup was served through API gateway, connected to a postgres RDS, selected a couple records from a single database table and returned them as json, and deployed in us-west.
From the west coast, I'm getting between 150ms and 300ms response times for most requests, but it does jump around quite a bit, with occasional requests taking 500ms-800ms to complete.
A lot of the variability seems to be coming from execution time of the lambda itself, which would probably be because of the connection to RDS? I'm using HikariCP so the connection is pooled but maybe something like dynamo would be better.
So latency is all over the place, but generally under half a second.
Connection would only be pooled on a per-instance basis, which can be cached but sometimes are not for many parallel running processes (to my understanding).
Listen folks - it's still a fantasy. Latency is still 5s on many requests = no deal.
This has been going on for years.
Would someone form AMZ please stand up?
I suggest that the latency is 5s for the 'first call', then, subsequent calls are fast - and if you don't use your lambda for a minute or so, it goes back to having latency.
I suggest maybe this has something to do with loading it into memory when it's used?
There must be a way to keep the lambdas 'hot' or else it will never really get used for anything but background tasks.
We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.
"You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.
> We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.
I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible.
Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.
What I'm asking for is definitely within grasp of Amazon's team :) they just need to get there.
Moreover, it's a rather fundamental opportunity for them: Lambda's are a form of 'container' and they are exactly what so many of us want. The deal-breaker for us was the latency issue.
Amazon is a pretty smart company, and it's a big opportunity, I'll bet they eventually get this sorted out.
What you're describing is managed hosting. In order to do it, you either have to oversubscribe like hell, and deal with latency, or pay for wide open swaths of server. Pick your poison.
Keeping a Lambda container in memory is probably not going to take more than 500MB of RAM. Paying a monthly minimum fee per lambda per machine per month to keep it in RAM should be an option. It would still be far cheaper than a real server.
You could have cloud watch run the function every minute? Again, that won't guarantee that your function will be "hot". But it's a quick hack to potentially reduce the frequency of your slow initial start.
That said customer feedback is super helpful in making great services so I'll get this feedback to the lambda team (or they may see it here).
If you ever need any help my email is in my profile and I'm happy to work with you.
I checked out the doc and didn't see any reference to latency or how to mitigate it, perhaps I'm missing something?
Listen - I'm a massive fan of AWS. We use some core services and are very happy.
I know you listen to customers - but I suggest that Lambda was probably designed for specific kind of services (maybe IT, admin, batch processing etc.) - I'm going to fathom that maybe 'people running web-sites' were top of mind? Again, purely speculation on my part.
But if you want people to seriously switch over - and they will - in droves - if you get it right, is to basically enable fairly responsive services that scale with a minimum of fuss, latency concerns included.
So - if I'm running a web-portal - and I have a customer pull some data to build a chart from somewhere - and I have a Lambda specifically geared for that microservice - then it needs to be quick.
There really should be no need to use other services to have to hack around the latency issue.
Surely - you can parameterize the feature if you want, and require users to set thresholds for how long Lambda's 'stay hot' (and forgive my crude metaphors), or how much throughput etc. - and you could even bill them based upon those parameters. As long as it's fairly concise and understandable.
Perhaps DynamoDB is a good example. We've used it before - and what we love about it is that as long as we design our data correctly we never have to worry. Bursts of traffic, etc. are generally handled well, it scales largely without concern. One poor limitation of that service is the requirement to indicate throughput thresholds, which is too bad, but beyond that, it's ideal.
Amazon Lambdas are basically the 'last piece' of the golden dream of cloud computing - the ability to develop a truly serverless, microserver-based infrastructure, and never have to worry about platforms again. As much as we love EC2s, we would drop them instantly and go to Lambdas if they worked well. (And it's been at least 18 months since we last experimented, BTW). There are tons of SMEs that would do the same - I don't think anyone who isn't directly in the business of tech, and is providing some other kind of consumer or business service, wants the pain of managing EC2s or servers. 'Elastic Beanstalk' etc. is a bandaid - it's not ideal.
Anyhow - Lambdas are very close to being a super great thing, more than that - transformational - if they could just cross that 'real world' usability threshold.
Again - if something has changed WRT latency, and there is something in those docs I'm missing, please let me know.
Not having PCI compliance in Lambdas are a huge drawback that keeps us on EC2s despite having a large amount of our webservice now on Lambda. API Gateway latency would be a big problem if we were doing a high perf webapp. Fix those 2 problems and we get to do serverless all the way.
I understand that Lambdas are 'relatively quick' when they are 'hot' - which is my crude metaphor for 'up and running'.
The issue is, if the Lambda has not been in use for some time (seconds, minutes?), then the 'first call' can have quite a bit of latency - often up to 5 seconds - which killed it for us.
There is no way we could reasonably deploy a service wherein customer would have to wait an additional 5 seconds for a reaction in most use cases.
So close, yet so far.
This was about 18 months ago.
If you have some data that addresses this specific artifact - then that would be great. Possibly things have changed in 18 months.
And I'm not a hater, to us AWS is the best thing since sliced bread.
Actually I was making the point that they are not "relatively quick" even when hot: close to 300ms, almost entirely due to the API Gateway. (I even measured API Gateway on top of bare ec2 and the same penalty applies).
Lambda itself is fairly quick (~75ms), but API Gateway needs some work.
The 5 second cold request thing is what GAE had 5 years ago. Eventually they gave up trying to make startup faster and allowed a paid plan to stay always hot. Which was good, but then also moving you much closer to a managed server (vs a pure pay as you use thing)
No way you're getting 15 ms on a TLS connection over the internet. I've set up similar pings with API Gateway and API Gateway -> Lambda, and it seems with the APIGW the TLS connection is the most expensive part. On an established connection I've seen 10 ms overhead which is not great but worth the value in some cases.
You could have certain requests that don't come that often, even on a high traffic site.
And if you don't know for how long lambdas stay hot, how are you supposed to time it? Even a call every few minutes may not be enough to keep them hot.
Ah, I see. Perhaps that would be a reason to keep an ec2 instance spun up to host the functionality that must always have latency < warmup+runtime? Surely suboptimal. This makes me consider running a cron job to poke timing critical lambda workflows every few minutes...which to me imply writing my functions in such away as to expect this "heartbeat" and act accordingly...and that just feels plain icky...
Some people prefer to call it Function as a Service, which in most cases is closer to what it actually is.
"serverless" does not refer to the absolute lack of servers - it refers to the fact that your CODE (applications and software in general) should not be AWARE of being executed on a physical or virtual machine.
The platform should be as abstract as it gets. I always imagine it as coding on a piece of paper: in a way, my code should work and make "sense", even though we all know that a piece of paper won't execute any instructions. And this is part of the magic of "serverless" software: it imagines a perfect, abstract machine where it can run, forgetting about the complex, unreliable, physical machine underneath.
It's an almost philosophical approach to writing software and getting it needs some time and thought. And needless to say, it does not solve ALL your problems, but it MIGHT solve SOME of your problems.
AWS Lambda and similar services have literally nothing to do with hardware vs software.
They're about running a function in response to an event on someone else's stack, almost like the `map` part of a `map-reduce`, if the input was considered to be distinct events/objects.
Yep. It's good old fashioned webhosting. Drop your files in the amazon equivalent of ~/public-html/ . If it serves your needs, it's not a bad thing, but it's hardly revolutionary.
Webhosting usually have a flat monthly fee so maybe Lambda it's more like old fashioned mainframe timesharing, billed per resource usage. Not that I was there in its heydays but the two concepts seem very similar.
A consequence is that you really want to terminate quickly and use little RAM. If your JS function terminates in 110 ms and get billed for two 100 ms units you might be tempted to switch to a faster language (or ASM what you can) to save half the bill. Webhosting doesn't have that dynamic.
I have just recently got through a migration from express to a lambda based stack. In the end, performance is pretty good (once the lambdas are warm), but there were a few tricky parts, but nothing impossible.
Serving backend requests is easy with Lambda, but the code for managing the backend can get quite large, quickly. If you are loading up data and it's going to take more than 5 minutes to perform a task, you need to build a queueing system from AWS primitives.
If anyone's interested in doing the math about the costs involved, I have two quick notes: first, the API Gateway, at $3.50 per million requests, is typically more expensive than the Lambda functions themselves (and comes with no permanent free tier, unlike Lambda); second, the 128MB minimum memory for a Lambda function can be on the tight side for an Express app, so you might want to base your calculations on more memory.
The only thing I'm bothered by is lack of environment variables (have to hard code them) and terrible API Gateway latency - see https://news.ycombinator.com/item?id=12681926 further down the page
Serverless was nowhere near ready the last time I looked.
Are they still requiring sn unrestricted iam profile? That is a non starter for most orgs.
Also, I found working with serverless to be pretty unintuitive. You are relying entirely on large config files and it was very difficult and time consuming for me to test and rollback several different configs.
If I would have to name one thing, it's that the configuration of API Gateway via the UI is unbearable if you have a nontrivial amount of routes (> 3). I need to configure Authorization, Error code mappings etc. for every route, although they are exactly the same.
If there would be per-gateway defaults I can set with per-route overrides, that would cut down the setup time by 95%, which is the main thing holding me back from using API Gateway for projects that would suit it.
Right now the only option I seem to have is using a tool like Terraform to automate it, which is a pain to adopt if you already have a running API Gateway.
You could also write a quick script using their REST API, although that's a bit cumbersome (lot of IDs that need to be passed around after creation of each resource).
It gets you to hello lambda (python) in one shot, from the cli. Anything written down in one repo is going to be far more undertstandable than piles of hidden behaviors and obscure error messages. I think there's a new one on their blog for node.
I've recently deployed a small Flask based micro-service for an internal tool leveraging Zappa[0] to deploy it to Lambda/API Gateway. Overall it's been a really smooth experience. I'm looking forward to using it again.
So, Apigee had this exact service. They did something pretty cool, it seems they implemented nearly all (!) of the Nodejs APIs in a JVM variant, then designed them to run on demand [0].
I actually ported an Express app to Apigee's service, to run webhooks and ETL. It worked flawlessly. The one downside was that they kept their own internal registry of compatible packages and there was no way they could keep up with the Nodejs ecosystem even with few holes in their API.
I still have a ton of hope that Google does something smart with this, I actually wanted to pay Apigee for their service, but their pricing started at an Enterprise tier and none of my projects made it that far.... yet.
I hope they solve the latency issues for a basic function that returns text (a basic callback(null, 'hello world') with the api gateway that is running 'hot' with the container model and yet still has so much latency.
I'm always surprised at the lack of discussion around database connection reuse with AWS Lambda.
It's a pretty big deal that every single API call requires a new database connection. The only solutions I've seen so far are to run a separate app to interface with the database, or moving the connection outside of the handler (which still has issues).
Am I missing something or is everyone just really happy to use DynamoDB?
How exactly have you been doing that? It would be a cool blog post if you had the time. I've been thinking about really high volume applications that connect to haproxy instances that do DB load balancing but even that has it's own set of initial latency issues. I haven't fully figured out that architecture. In python in particular I'd really love to know how to setup the connection reuse.
Check out the handler source code in Zappa for an example of a pattern like that. Similarly, if you use Zappa to deploy your application, if you create your database connection when the application loads, it'll just work.
Stop by the Zappa slack if you want to explore this in more detail! https://slack.zappa.io It could always use more investigation, but there are quite a few Zappa users at extremely high loads now without any issues.
This project looks super interesting. Thanks for sharing it!
Now I want to come up with a use case to test it out over the next few days, maybe something for the Echo? I joined your Slack, we'll see where this takes us :)
I am curious what sort of stuff is being built that need microservices (presumably so they scale) yet still has a database back end (usually the IO is the bottleneck in most applications).
DynamoDB is not immune from that problem. There is no way out: if the container is terminated the connection goes down. The key is not making Lambda stop and rm the container (in docker terms). If there are enough requests the container is reused and the connection stays up, but you must initialize it outside the function. An example with DynamoDB
Usually 15 or 20 ms hit to make a new connection even in MySQL.
Also adds a fairly big server load. I suspect if a given MySQL instance supports 1000 long lived connections.. it would only support 100-200 connections that are closing every request. (Have not benchmarked this side, would be curious to see )
Hi Everhusk, I don't specifically call it out in the article, but the code is actually reusing database connections. As mentioned in other comments, the trick is to establish the connection outside of the handler so that the connection only happens once per container.
I hate the term serverless.
It involves a server. AWS Lambda is a service bound to a port therefore a server. It runs user defined functions, true. But it is a server.
Call it whatever you want, but I like to use the language of those who hold the purse-strings. If they call it "serverless", then I call it "serverless". If delivering a lecture about why it's not really serverless improves your product, then go ahead and deliver that lecture, but I doubt it will.
It's more like driverless cars. Something clearly is driving the car, but it isn't you.
Moreover the fact it's even a car is now an implementation detail. It could just as well be a motorised shipping container that slots into a hyperloop when it reaches the station. The assumption that the AWS Lambda infrastructure is server hardware is just that - it may not be true now, and you could build a function-as-a-service platform out of a massive array of Raspberry Pis and it'd still have solid at-least-once invocation semantics.
And I'm repeating this, but you're flat wrong about the listening server-on-a-port. AWS Lambda functions are bound to queues and receive events. The closest you'll get to having a socket-on-a-port is that the provisioning API has a REST endpoint.
So if it isn't necessarily server hardware, and there's no server endpoint to connect to, that "hate" you started this thread with looks rather misplaced.
It would help if people referred to this technological concept as Function as a Service. That is a lot less ambiguous and buzzwordish than 'serverless'.
There is a driver! It's just in software! The car's still got something driving it. It's just not a person!
People tell me, "Dude, that's pedantic. Clearly what people mean when they say 'driverless cars' isn't that there's literally nothing driving it. They mean that having people driving cars has a lot of downsides and having the benefits of cars available without having an in-person driver opens up a lot of opportunities and frees us from personally having to drive ourselves."
I am currently using AWS API Gateway + Lambda for the serverless backend of a social traveling startup. It is absolutely awesome: we are few people in IT and it is amazingly simple to manage the whole stuff.
Moreover imho they are mature services at this time, they are flexible enough with enough configuration properties to fulfill a nearly full control of the development experience.
We reduced our service server costs to 1/6th by moving from Heroku to Lambda. And that's not even considering the main benefit of a big computational request not blocking other requests (which is why we moved to it in the first place).
@hobofan, take a look at our recent API Gateway features and see if the greedy paths and pass-through settings provide what you're looking for (they're also supported by CloudFormation). We tried to simplify the "configure every route" problem, but always looking for additional suggestions to make API config easier.
@jomamaxx (& others): Reducing latency (and latency variability) is a critical goal for our team. We've improved p99 variability in API Gateway-related latency over the last few months, and will be addressing some of the Lambda-related latency that occurs when we refresh containers in the coming months. Additional latency optimizations coming at all levels of the stack, including networking.
A clarification on the discussions about "managed hosting": Lambda is not classic web hosting; in fact, we block Lambda functions from calling socket.listen. In the Lambda model (whatever you prefer to call the broader category), the cloud service sees every request in order to perform scaling and load balancing on the function's behalf.
Happy to chat offline with anyone who has additional questions or feedback: DM me on twitter @timallenwagner
Tim, when are we getting Python 3 support on Lambda?
Being stuck on 2.7 has been the greatest source of issues and the #1 problem we have with Lambda. If Google/Microsoft came out today with Python 3 support in their Lambda competitor, we'd move in an instant.
I'm in the same boat. We're developing a new product with no real reason to stay on Python 2, besides Lambda. Python 3 support would make my life so much easier.
Yep, I hear about the lack of Python3 all the time, and I know our Python users are waiting for it. On the roadmap, and partially complete, but competing with some other language work at the moment.
The point is "serverless" is a sham. Your clients still need to rely on another service API Gateway to communicate with the outside world. So stop with this stupid marketing.
Your are selling a basic pubsub architecture as something "innovative" when the only real innovation is the pricing. That's the exact definition of marketing, and at the same time you are making the community of web developers sound like idiots that don't understand what a server is. You can make plenty of money without being missleading, why do you feel the need to use a misleading language ?
Of course there's a server, or do you think you're the only genius in the room who understands client server communication? "Serverless" is a legitimate phrase to describe a growing phenomenon. Most reasonable people read "serverless" as "server I don't have to admin", not "there is no server". No one wants to read your pedantic bloviating, get over it!
> Most reasonable people read "serverless" as "server I don't have to admin"
Most reasonable people find the "serverless" moniker obnoxious to begin with. If something doesn't require administration then call it "adminless", not "serverless"
I would want to know, how do I manage user credentials in AWS lambda. Lambdas are stateless, there is no session, so how do I keep track of which user is executing the current lambda?
How do you do it with a load balanced stateless web server? People have been doing this 10 years, most language should have some frameworks or plugins around it....
Common is to give the client a cookie with a session id (KJASDJKASDASDS) in a cookie. Then in a RDBMS you store this (session KJASDJKASDASDS = User 1234)... and the first part of each web (or lambda call), you go look up the user by session and know who it is.
(You can also store the user ID directly in an encrypted cookie, but that has a few other problems)
1) A NoSQL DB for the main data providing replication and scalability(e.g. Cassandra or DynamoDB).
2) Another DB for quick access and transient Data where replication is not so important, e.g. storing the session cookie. Relational vs Non-relational would not be an issue since I'm only storing very little data here and want fast access and minimize costs(in DynamoDB you still have to pay per operation). Therefore I'm looking for some simple solution like ElastiCache.
I'm doing a social networking site, so I don't know how much data I will end up with. My reasoning was as follows:
1) The data model is rather simple, basically one table with the user data(name, location, age, etc...) and some images. So it shouldn't be too difficult to do this with NoSql as opposed to SQL. Yes, simpler with SQL but not significantly so.
2) If I ever need to scale, I'd rather make the right choice from the beginning(NoSQL) so as not to need a migration later.
3) In any case NoSQL would be a learning experience for me, so another plus.
If there is a mistake in my reasoning I would be grateful for any one pointing it out.
Just curious, what would be the major drawbacks of not using relational here? My understanding is that AWS is pushing Dynamo quite a bit and it seems cheaper, so if they're only needing to query off of session id and maybe user id, shouldn't that be sufficient?
Relational is super handy. Very easy to report on and do various things down the road.
If all he needed was session id, sure, NoSQL is more or less the same. But what about when he adds other fields that are related? Say customer address or reports or .... His life may very well be easier with relational.
You should generally start relational. Then branch out if you are hitting the brick walls of relational. People using NoSQL for a 100MB database are making their life SERIOUSLY more difficult than it needs to be. (I have done it before, not fun).
One significant problem with the API Gateway / Lambda integration is that the protocol they use to communicate is JSON. JSON is a text-only encoding, i.e. anything that isn't valid UTF8 isn't valid JSON, and so if you need to pass something binary to/from the Lambda (e.g. an image), you need to base64 encode it, which I'm actually not even sure is at all possible with the API/GW templates. There are other issues, but this alone is a huge headache if you ever have to work around it.
Serverless via Lambda has been, frankly, a disappointment so far. The benefit is that I'm supposed to not have to manage servers anymore, yet I find myself, well, managing servers.
I have to build and assign IAM roles, subnets, set up application configuration files in fixed external services (since you can't do things like set environment variables), configure the endpoints in nginx^h^h^h^h^h API Gateway, suffer through cold starts, fight against arcane packaging issues with site directories and .pth files (though this is probably just be a Python thing), fight concurrency throttling, external timeouts, manage out-of-band database connections, logging which requires even more IAM permissions...
This doesn't even consider how many times I've had to tear down and re-create an API Gateway setup because it got "stuck" and would stop working.
Finally, WTF is up with not being able to test API Gateway -> S3 integrations if the S3 bucket is in the same region as the gateway instance? It's a two plus year old bug by now, and a real pain (especially when coupled with other AWS services which require the bucket to be in the same region as the rest of a service - such as CodePipeline).
Perhaps it's just because I'm familiar with setting up and provisioning servers, but for basic web services and periodic tasks, Lambda is much harder to work with most of the time. Its biggest benefit so far has been the low cost.
> Perhaps it's just because I'm familiar with setting up and provisioning servers
I'd be inclined to say that may be the case. Our codebase leans heavily on Lambda, primarily as a wide stage in our data pipeline. And I love it.
Using a configuration file in S3 - or any other means of dropping a text file on AWS - is barely more complicated than open("file.txt"). The true frustration is when you need to work with files beyond your memory limits, but even that can be mitigated by divide and conquer approaches.
That said, the API Gateway is annoying in that it makes complex things impossible and simple things more configuration than they need. Yet, I'd still take it over actually standing up a server.
> is barely more complicated than open("file.txt")
Well, that depends. First, you have to set up the IAM roles to access the bucket which owns file.txt. Then you have to ensure that "file.txt" is properly shared between accounts if you separate production from staging from development. Then there's three lines of code to set up a client, pull the file, and read the contents. If it's an encrypted file, you also get to do a bit more setup work to ensure that you're using the v4 signature in the library, since it defaults to v3 signatures. There's also the additional error handling and points of failure to account for.
The requests are far from instantaneous as well, so you need to set up an out-of-band cache for the contents of the file if you plan to use it with every call of the lambda function.
Is it obtusely hard? Of course not. Is it as simple as `open("file.txt")` (or more appropriately `os.environ["FOO"]`)? Not by a long shot.
I have a feeling that our differences in opinion have more to do with the environment in which we run our lambda functions than what the lambda functions are doing. Once you move beyond a single region on a single account, things get a lot harder.
I may be overly simplifying working with a file, but I'd still argue you're over-complicating it :P You're hitting on a good point with Lambda though, namely the stateless aspect. To get the most out of Lambda, you have to adopt a more Functional-based programming model.
> Once you move beyond a single region on a single account, things get a lot harder.
I could see that.
I'm working in IoT, primarily connecting lots of dumb devices to small translation layers with minute data massaging along the way. In this type of low-intensity environment, the flexibility Lambda (AWS in general) gives you is tremendous.
The second performance(/latency) becomes a primary concern, it's easy to see that trade-off become untenable.
Check out our example on github https://github.com/awslabs/aws-serverless-express/tree/maste.... The lambda wrapper is very thin (4 LOC). So you have two options: run/test your express app locally as you always would, or use the provided `npm run local` command which simulates the API Gateway+Lambda part (you can modify the `api-gateway-event.json` to change the "request" from API Gateway). This is primarily an example and starting point, and there is much more you could do to improve this process.
What is the benefit of AWS Lambda vs getting a cheap VPS or a dedicated server with much better specs and bandwidth, setting it up and adding auto-update? And how does one prevent going bankrupt if a Lambda app is DDOSed?
Article doesn't mention any numbers. OVH VPS is $3.50 for a 2GB RAM and 100mbs (about 14TB a month assuming 50% utilization). That is about 14mln pages with 1MB size vs 1mln requests for API Gateway at $3.50, not including Lambda costs or bandwidth costs.
I don't have exact numbers, but we have dozens of lambda functions written in Python which are parsing and acting on Kinesis log streams (at volumes that cause problems for the Splunk ingest), and regularly hit the 100 process concurrency limit... but our bill is well under $10 a year (yes, year, not month. Hence my use of the adjectives "ridiculously" and "stupidly").
Luck? Hidden discounts? Don't know. Just know that our bill is ridiculously low for everything we're doing.
139 comments
[ 4.3 ms ] story [ 200 ms ] threadFrom the west coast, I'm getting between 150ms and 300ms response times for most requests, but it does jump around quite a bit, with occasional requests taking 500ms-800ms to complete.
A lot of the variability seems to be coming from execution time of the lambda itself, which would probably be because of the connection to RDS? I'm using HikariCP so the connection is pooled but maybe something like dynamo would be better.
So latency is all over the place, but generally under half a second.
This has been going on for years.
Would someone form AMZ please stand up?
I suggest that the latency is 5s for the 'first call', then, subsequent calls are fast - and if you don't use your lambda for a minute or so, it goes back to having latency.
I suggest maybe this has something to do with loading it into memory when it's used?
There must be a way to keep the lambdas 'hot' or else it will never really get used for anything but background tasks.
If that's what you need...you need a server.
We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.
"You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.
I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible.
Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.
Moreover, it's a rather fundamental opportunity for them: Lambda's are a form of 'container' and they are exactly what so many of us want. The deal-breaker for us was the latency issue.
Amazon is a pretty smart company, and it's a big opportunity, I'll bet they eventually get this sorted out.
You're describing container re-use which is documented in the page here: http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct...
You could have cloud watch run the function every minute? Again, that won't guarantee that your function will be "hot". But it's a quick hack to potentially reduce the frequency of your slow initial start.
That said customer feedback is super helpful in making great services so I'll get this feedback to the lambda team (or they may see it here).
If you ever need any help my email is in my profile and I'm happy to work with you.
I checked out the doc and didn't see any reference to latency or how to mitigate it, perhaps I'm missing something?
Listen - I'm a massive fan of AWS. We use some core services and are very happy.
I know you listen to customers - but I suggest that Lambda was probably designed for specific kind of services (maybe IT, admin, batch processing etc.) - I'm going to fathom that maybe 'people running web-sites' were top of mind? Again, purely speculation on my part.
But if you want people to seriously switch over - and they will - in droves - if you get it right, is to basically enable fairly responsive services that scale with a minimum of fuss, latency concerns included.
So - if I'm running a web-portal - and I have a customer pull some data to build a chart from somewhere - and I have a Lambda specifically geared for that microservice - then it needs to be quick.
There really should be no need to use other services to have to hack around the latency issue.
Surely - you can parameterize the feature if you want, and require users to set thresholds for how long Lambda's 'stay hot' (and forgive my crude metaphors), or how much throughput etc. - and you could even bill them based upon those parameters. As long as it's fairly concise and understandable.
Perhaps DynamoDB is a good example. We've used it before - and what we love about it is that as long as we design our data correctly we never have to worry. Bursts of traffic, etc. are generally handled well, it scales largely without concern. One poor limitation of that service is the requirement to indicate throughput thresholds, which is too bad, but beyond that, it's ideal.
Amazon Lambdas are basically the 'last piece' of the golden dream of cloud computing - the ability to develop a truly serverless, microserver-based infrastructure, and never have to worry about platforms again. As much as we love EC2s, we would drop them instantly and go to Lambdas if they worked well. (And it's been at least 18 months since we last experimented, BTW). There are tons of SMEs that would do the same - I don't think anyone who isn't directly in the business of tech, and is providing some other kind of consumer or business service, wants the pain of managing EC2s or servers. 'Elastic Beanstalk' etc. is a bandaid - it's not ideal.
Anyhow - Lambdas are very close to being a super great thing, more than that - transformational - if they could just cross that 'real world' usability threshold.
Again - if something has changed WRT latency, and there is something in those docs I'm missing, please let me know.
Bare ec2: 15ms
Lambda via API Gateway: 276ms
[0]: http://prerender.cloud/lambda-latency [1]: https://news.ycombinator.com/item?id=12678011
I understand that Lambdas are 'relatively quick' when they are 'hot' - which is my crude metaphor for 'up and running'.
The issue is, if the Lambda has not been in use for some time (seconds, minutes?), then the 'first call' can have quite a bit of latency - often up to 5 seconds - which killed it for us.
There is no way we could reasonably deploy a service wherein customer would have to wait an additional 5 seconds for a reaction in most use cases.
So close, yet so far.
This was about 18 months ago.
If you have some data that addresses this specific artifact - then that would be great. Possibly things have changed in 18 months.
And I'm not a hater, to us AWS is the best thing since sliced bread.
Lambda itself is fairly quick (~75ms), but API Gateway needs some work.
250ms we can live with for many things, but not 5x.
But yes - it should be under 50ms at all times I think for people to start taking it seriously.
If your product has only occasional use then you trade cost for latency; spin up an ec2 instance if your request times exceed a few minutes call
"serverless" does not refer to the absolute lack of servers - it refers to the fact that your CODE (applications and software in general) should not be AWARE of being executed on a physical or virtual machine.
The platform should be as abstract as it gets. I always imagine it as coding on a piece of paper: in a way, my code should work and make "sense", even though we all know that a piece of paper won't execute any instructions. And this is part of the magic of "serverless" software: it imagines a perfect, abstract machine where it can run, forgetting about the complex, unreliable, physical machine underneath.
It's an almost philosophical approach to writing software and getting it needs some time and thought. And needless to say, it does not solve ALL your problems, but it MIGHT solve SOME of your problems.
With the exception of para-virtualised device drivers, what code is ever aware of being executed on physical or virtual machines?
AWS Lambda and similar services have literally nothing to do with hardware vs software.
They're about running a function in response to an event on someone else's stack, almost like the `map` part of a `map-reduce`, if the input was considered to be distinct events/objects.
A consequence is that you really want to terminate quickly and use little RAM. If your JS function terminates in 110 ms and get billed for two 100 ms units you might be tempted to switch to a faster language (or ASM what you can) to save half the bill. Webhosting doesn't have that dynamic.
How do I get "old fashioned webhosting" to automatically scale-out and scale-in?
How do I do all that and get per-100ms CPU-time billing?
Serving backend requests is easy with Lambda, but the code for managing the backend can get quite large, quickly. If you are loading up data and it's going to take more than 5 minutes to perform a task, you need to build a queueing system from AWS primitives.
Unless they have made serious updates to the deployment workflow, feature set, and documentation I would not recommend building your products on it.
But that workflow doesn't make sense for real dev teams. So there's https://github.com/serverless/serverless . It's driven by a yaml file and is intuitive.
The only thing I'm bothered by is lack of environment variables (have to hard code them) and terrible API Gateway latency - see https://news.ycombinator.com/item?id=12681926 further down the page
const config = require('./config.json');
Then you do config.secrets.apiKey etc
You have to gitignore config.json and include it in serverless.yml
Other languages might have to read and parse that JSON into one of their data structures.
Are they still requiring sn unrestricted iam profile? That is a non starter for most orgs.
Also, I found working with serverless to be pretty unintuitive. You are relying entirely on large config files and it was very difficult and time consuming for me to test and rollback several different configs.
If I would have to name one thing, it's that the configuration of API Gateway via the UI is unbearable if you have a nontrivial amount of routes (> 3). I need to configure Authorization, Error code mappings etc. for every route, although they are exactly the same. If there would be per-gateway defaults I can set with per-route overrides, that would cut down the setup time by 95%, which is the main thing holding me back from using API Gateway for projects that would suit it.
Right now the only option I seem to have is using a tool like Terraform to automate it, which is a pain to adopt if you already have a running API Gateway.
Still better than manual clicking in my opinion.
http://docs.aws.amazon.com/apigateway/api-reference/resource...
It gets you to hello lambda (python) in one shot, from the cli. Anything written down in one repo is going to be far more undertstandable than piles of hidden behaviors and obscure error messages. I think there's a new one on their blog for node.
[0] https://github.com/Miserlou/Zappa
I actually ported an Express app to Apigee's service, to run webhooks and ETL. It worked flawlessly. The one downside was that they kept their own internal registry of compatible packages and there was no way they could keep up with the Nodejs ecosystem even with few holes in their API.
I still have a ton of hope that Google does something smart with this, I actually wanted to pay Apigee for their service, but their pricing started at an Enterprise tier and none of my projects made it that far.... yet.
Something to think about.
[0]: http://docs.apigee.com/api-services/content/overview-nodejs-...
It's a pretty big deal that every single API call requires a new database connection. The only solutions I've seen so far are to run a separate app to interface with the database, or moving the connection outside of the handler (which still has issues).
Am I missing something or is everyone just really happy to use DynamoDB?
Stop by the Zappa slack if you want to explore this in more detail! https://slack.zappa.io It could always use more investigation, but there are quite a few Zappa users at extremely high loads now without any issues.
However, is Lambda cost effective for services with the amount of requests required to have their containers almost never terminated?
Maybe it's OK for burst shaped traffic. The first request pays a toll, the others are quick.
MySQL has a really fast server side connection pool.
Also adds a fairly big server load. I suspect if a given MySQL instance supports 1000 long lived connections.. it would only support 100-200 connections that are closing every request. (Have not benchmarked this side, would be curious to see )
It is a very misleading term. Especially for non-tech savvy. Sounds like a scam.
It's like calling being passenger on a car "wheel-less transport", because I as passenger don't have wheels.
Don't get hung up on the name. Bananas are not a fruit, but it's fine to put them in a fruit salad.
Moreover the fact it's even a car is now an implementation detail. It could just as well be a motorised shipping container that slots into a hyperloop when it reaches the station. The assumption that the AWS Lambda infrastructure is server hardware is just that - it may not be true now, and you could build a function-as-a-service platform out of a massive array of Raspberry Pis and it'd still have solid at-least-once invocation semantics.
And I'm repeating this, but you're flat wrong about the listening server-on-a-port. AWS Lambda functions are bound to queues and receive events. The closest you'll get to having a socket-on-a-port is that the provisioning API has a REST endpoint.
So if it isn't necessarily server hardware, and there's no server endpoint to connect to, that "hate" you started this thread with looks rather misplaced.
There is a driver! It's just in software! The car's still got something driving it. It's just not a person!
People tell me, "Dude, that's pedantic. Clearly what people mean when they say 'driverless cars' isn't that there's literally nothing driving it. They mean that having people driving cars has a lot of downsides and having the benefits of cars available without having an in-person driver opens up a lot of opportunities and frees us from personally having to drive ourselves."
I hate it when they tell me that.
If you are worried about latency or you want lambda on your own servers it may be worth to take a look.
Feedback welcome
@jomamaxx (& others): Reducing latency (and latency variability) is a critical goal for our team. We've improved p99 variability in API Gateway-related latency over the last few months, and will be addressing some of the Lambda-related latency that occurs when we refresh containers in the coming months. Additional latency optimizations coming at all levels of the stack, including networking.
A clarification on the discussions about "managed hosting": Lambda is not classic web hosting; in fact, we block Lambda functions from calling socket.listen. In the Lambda model (whatever you prefer to call the broader category), the cloud service sees every request in order to perform scaling and load balancing on the function's behalf.
Happy to chat offline with anyone who has additional questions or feedback: DM me on twitter @timallenwagner
Being stuck on 2.7 has been the greatest source of issues and the #1 problem we have with Lambda. If Google/Microsoft came out today with Python 3 support in their Lambda competitor, we'd move in an instant.
Your are selling a basic pubsub architecture as something "innovative" when the only real innovation is the pricing. That's the exact definition of marketing, and at the same time you are making the community of web developers sound like idiots that don't understand what a server is. You can make plenty of money without being missleading, why do you feel the need to use a misleading language ?
There is a server, period.
Most reasonable people find the "serverless" moniker obnoxious to begin with. If something doesn't require administration then call it "adminless", not "serverless"
> Your are selling a basic pubsub architecture as something "innovative" when the only real innovation is the pricing.
Common is to give the client a cookie with a session id (KJASDJKASDASDS) in a cookie. Then in a RDBMS you store this (session KJASDJKASDASDS = User 1234)... and the first part of each web (or lambda call), you go look up the user by session and know who it is.
(You can also store the user ID directly in an encrypted cookie, but that has a few other problems)
https://aws.amazon.com/rds/
1) A NoSQL DB for the main data providing replication and scalability(e.g. Cassandra or DynamoDB).
2) Another DB for quick access and transient Data where replication is not so important, e.g. storing the session cookie. Relational vs Non-relational would not be an issue since I'm only storing very little data here and want fast access and minimize costs(in DynamoDB you still have to pay per operation). Therefore I'm looking for some simple solution like ElastiCache.
1) The data model is rather simple, basically one table with the user data(name, location, age, etc...) and some images. So it shouldn't be too difficult to do this with NoSql as opposed to SQL. Yes, simpler with SQL but not significantly so.
2) If I ever need to scale, I'd rather make the right choice from the beginning(NoSQL) so as not to need a migration later.
3) In any case NoSQL would be a learning experience for me, so another plus.
If there is a mistake in my reasoning I would be grateful for any one pointing it out.
If all he needed was session id, sure, NoSQL is more or less the same. But what about when he adds other fields that are related? Say customer address or reports or .... His life may very well be easier with relational.
You should generally start relational. Then branch out if you are hitting the brick walls of relational. People using NoSQL for a 100MB database are making their life SERIOUSLY more difficult than it needs to be. (I have done it before, not fun).
I don't know if it is that easy to scale once you realize that the demand is growing, see my other reply here:
https://news.ycombinator.com/item?id=12692234
I have to build and assign IAM roles, subnets, set up application configuration files in fixed external services (since you can't do things like set environment variables), configure the endpoints in nginx^h^h^h^h^h API Gateway, suffer through cold starts, fight against arcane packaging issues with site directories and .pth files (though this is probably just be a Python thing), fight concurrency throttling, external timeouts, manage out-of-band database connections, logging which requires even more IAM permissions...
This doesn't even consider how many times I've had to tear down and re-create an API Gateway setup because it got "stuck" and would stop working.
Finally, WTF is up with not being able to test API Gateway -> S3 integrations if the S3 bucket is in the same region as the gateway instance? It's a two plus year old bug by now, and a real pain (especially when coupled with other AWS services which require the bucket to be in the same region as the rest of a service - such as CodePipeline).
Perhaps it's just because I'm familiar with setting up and provisioning servers, but for basic web services and periodic tasks, Lambda is much harder to work with most of the time. Its biggest benefit so far has been the low cost.
I'd be inclined to say that may be the case. Our codebase leans heavily on Lambda, primarily as a wide stage in our data pipeline. And I love it.
Using a configuration file in S3 - or any other means of dropping a text file on AWS - is barely more complicated than open("file.txt"). The true frustration is when you need to work with files beyond your memory limits, but even that can be mitigated by divide and conquer approaches.
That said, the API Gateway is annoying in that it makes complex things impossible and simple things more configuration than they need. Yet, I'd still take it over actually standing up a server.
Well, that depends. First, you have to set up the IAM roles to access the bucket which owns file.txt. Then you have to ensure that "file.txt" is properly shared between accounts if you separate production from staging from development. Then there's three lines of code to set up a client, pull the file, and read the contents. If it's an encrypted file, you also get to do a bit more setup work to ensure that you're using the v4 signature in the library, since it defaults to v3 signatures. There's also the additional error handling and points of failure to account for.
The requests are far from instantaneous as well, so you need to set up an out-of-band cache for the contents of the file if you plan to use it with every call of the lambda function.
Is it obtusely hard? Of course not. Is it as simple as `open("file.txt")` (or more appropriately `os.environ["FOO"]`)? Not by a long shot.
I have a feeling that our differences in opinion have more to do with the environment in which we run our lambda functions than what the lambda functions are doing. Once you move beyond a single region on a single account, things get a lot harder.
> Once you move beyond a single region on a single account, things get a lot harder.
I could see that.
I'm working in IoT, primarily connecting lots of dumb devices to small translation layers with minute data massaging along the way. In this type of low-intensity environment, the flexibility Lambda (AWS in general) gives you is tremendous.
The second performance(/latency) becomes a primary concern, it's easy to see that trade-off become untenable.
If not, this introduces yet another environment variable or it requires changes to be pushed constantly to Lamba for test.
That said, there is a concurrency limit, which would help prevent your server time going to exponential limits.
Luck? Hidden discounts? Don't know. Just know that our bill is ridiculously low for everything we're doing.