I love AWS because it offers immediate solutions at your fingertips, however, I am skeptical of Lambda's ubiquity.
Too many times have I drank the Kool-Aid for `someManagedAWSService` and then been surprised by the resultant costs. For example, be very careful when using things like DynamoDB and Lambda. One can easily use DynamoDB incorrectly and end up wasting lots of money. Similarly, I think it would be very easy to adopt some sort of "replace everything with Lambda" approach which would be a misplaced effort as well. Lambda has a specific time and place, it is not a silver bullet.
When I first began looking to take my web development skills live to a VPS / other services, this was one of the things that drove me away from even playing around with AWS.
Not sure what it is. I went with Digital Ocean, but even compared to Google Cloud or Microsoft Azure, Amazon's console area intimidates me, like I feel like I might get charged or trigger charges that will just rack up if I click on anything there. Maybe I'm just being paranoid, but I don't have the funds to risk playing around with their services and accidentally getting billed $100,000 or something.
This is what I love about their free-tier. For some services, You get quite a bit of processing power per month for free for a year or so. These terms may have changed since I last read them. Still other services give you freemium resource usage. For example, I think CloudFront gives you a certain number of free cache-invalidations per month.
I guess there's always the risk of someone hacking you, then eating up your AWS resources to mine coins. This could rack up a huge bill. I think this hacking risk is true for many cloud services, though. One way to prevent this is to use IAM users with limited resource usage, and basically freeze the root account. AWS is a lot less stressful to use when you assume a "nobody allowed" or "least privilege" approach.
Well, first of all, there are default limits on everything that prevents you from racking up too huge bills (25 running EC2 instances, for instance). You have to go out of your way to get them raised.
Second, they pretty much only charge per resource usage: CPU time, network traffic, and storage. Consume neither of those, and it is free. So it is free to set up really complicated virtual networks in the VPC part of the console, but you start paying once traffic flows in that network.
The only insanely expensive part that you'd hate to enable is the "Dedicated IP Custom SSL" option in CloudFront, because that is $600 per month, as per their documentation[0].
> Because of the added cost associated with dedicating IP addresses per SSL certificate, we charge a fixed monthly fee of $600 for each custom SSL certificate you associate with your CloudFront distributions, pro-rated by the hour.
I mention it because its trivial to provision in AWS accounts.
In sandbox accounts we provision for developers, we use AWS Orgs and service control policies to ensure IAM users with full privs can't provision an HSM.
One could easily configure lots of things wrong in life that would end up wasting money! That said we find that many customers end up saving a lot by moving workloads from more traditional "serverfull" models to Lambda and serverless application patterns. We also see entire startups starting this way and getting to fairly massive scale without ever needing to manage a host anywhere. While not a silver bullet, the use-cases for Lambda are many and proven. More here: https://aws.amazon.com/lambda/resources/#Customer_Case_Studi...
Yeah this is definitely true, it's very easy to rack up huge prices when there aren't bounds, and AWS provides bounds which is great. Lambda, S3, CloudFront, APIGateway, etc. are an incredible boon for any startup.
However, the way that AWS charges for throughput on DynamoDB is deceptive -- you can easily bottleneck the system with a hot spot, yet you would need to pay for bandwidth increases everywhere to raise the limit for that single hot spot. This lack of granularity seems to be due to DynamoDB's technical design, and/or lack of a better charging system.
I remember reading an article on how some company ran into a bunch of problems due to their non-uniform query distribution, but I can't find it. I am not an expert, but I would avoid DynamoDB today and stick with something like Cassandra or MongoDB for general NOSQL work. Hopefully DynamoDB has changed since I last encountered it as the idea is a very good one and fits well in the serverless ecosystem.
Lambda has a long way to go, it's a promising product but if you want to develop anything for it, it's a bit painful and archaic.
I've had issues with the temp memory. It is completely unreliable, you have to store files when processing in /tmp/ and for whatever reason, reading from this results is patchy. I had to resort to storing every file I process in the RAM.
The tooling is only just taking off now, 6 months ago simulation was painful, at least there is AWS SAM now and docker support.
However despite the pain, it's giving me a solution for scalable video encoding that is very cost competitive and I don't have to worry about submitting jobs and monitoring resources.
But what I mean is that its clearly not designed for you to be dropping files onto the local disk.
Video encoding is kind of a corner case in that the size of the data really stops you from doing the whole thing in memory, though the persistance isn't needed in between runs.
Probably what you want to look into is attaching an EBS volume to the job, and doing the work from there.
This feels like an anti pattern. Lambda is event driven, it doesn’t even have a mechanism to schedule activity. You have to abuse some other service to create an event on a schedule.
Surely this is not the best use of the service, even if it is common.
If you need to run tasks on a schedule what’s wrong with cron? Or a cron service?
There is an AWS cron service, it actually runs on top of lambdas[0]. If your definition of "abuse" is triggering an event with cloudWatch, then every service must be "abused" (SNS, S3, dynamodb, etc) as well.
CloudWatch monitors logs for specific keywords, if it finds a certain number of them within a time threshold it creates an "Alarm" which then triggers a Lambda.
Yeah and that feels like a hack to me. What generates the events on the right cadence to trigger this Lambda alarm? Why not just have that run the Lambda? And why not just have that run the code like cron?
I know it works and it fills a gap for people but it just feels odd to me. It's like there is cron -> log -> monitor -> alarm -> event -> lambda -> script when you could just have cron -> script.
I'm probably just hung up on the naming and the fact that these events are branded as part of Cloudwatch. If there was some service spun off that just generated events in AWS I wouldn't raise an eyebrow, even if under the hood it was exactly the same thing.
I appreciate that AWS Lambda has been a win for various people but this case study doesn't really mention what they tried to do with scaling. Whilst 0 to 100k sounds good, they also had seemingly chosen just one scaling solution of a big instance - several smaller instances behind a load balancer which scale up/down would surely have served the same outcome?
Lambda has been sold on "saving costs" but what seems to be often neglected is the developer time spent re-jigging into a "serverless" model. Developer resources are far, far more expensive than mere EC2 instances.
25 comments
[ 2.9 ms ] story [ 80.5 ms ] threadToo many times have I drank the Kool-Aid for `someManagedAWSService` and then been surprised by the resultant costs. For example, be very careful when using things like DynamoDB and Lambda. One can easily use DynamoDB incorrectly and end up wasting lots of money. Similarly, I think it would be very easy to adopt some sort of "replace everything with Lambda" approach which would be a misplaced effort as well. Lambda has a specific time and place, it is not a silver bullet.
Not sure what it is. I went with Digital Ocean, but even compared to Google Cloud or Microsoft Azure, Amazon's console area intimidates me, like I feel like I might get charged or trigger charges that will just rack up if I click on anything there. Maybe I'm just being paranoid, but I don't have the funds to risk playing around with their services and accidentally getting billed $100,000 or something.
I guess there's always the risk of someone hacking you, then eating up your AWS resources to mine coins. This could rack up a huge bill. I think this hacking risk is true for many cloud services, though. One way to prevent this is to use IAM users with limited resource usage, and basically freeze the root account. AWS is a lot less stressful to use when you assume a "nobody allowed" or "least privilege" approach.
Sorry for the edits, I am drunk.
Second, they pretty much only charge per resource usage: CPU time, network traffic, and storage. Consume neither of those, and it is free. So it is free to set up really complicated virtual networks in the VPC part of the console, but you start paying once traffic flows in that network.
The only insanely expensive part that you'd hate to enable is the "Dedicated IP Custom SSL" option in CloudFront, because that is $600 per month, as per their documentation[0].
> Because of the added cost associated with dedicating IP addresses per SSL certificate, we charge a fixed monthly fee of $600 for each custom SSL certificate you associate with your CloudFront distributions, pro-rated by the hour.
[0] https://aws.amazon.com/cloudfront/custom-ssl-domains/
In sandbox accounts we provision for developers, we use AWS Orgs and service control policies to ensure IAM users with full privs can't provision an HSM.
One could easily configure lots of things wrong in life that would end up wasting money! That said we find that many customers end up saving a lot by moving workloads from more traditional "serverfull" models to Lambda and serverless application patterns. We also see entire startups starting this way and getting to fairly massive scale without ever needing to manage a host anywhere. While not a silver bullet, the use-cases for Lambda are many and proven. More here: https://aws.amazon.com/lambda/resources/#Customer_Case_Studi...
However, the way that AWS charges for throughput on DynamoDB is deceptive -- you can easily bottleneck the system with a hot spot, yet you would need to pay for bandwidth increases everywhere to raise the limit for that single hot spot. This lack of granularity seems to be due to DynamoDB's technical design, and/or lack of a better charging system.
https://news.ycombinator.com/item?id=14721920
I remember reading an article on how some company ran into a bunch of problems due to their non-uniform query distribution, but I can't find it. I am not an expert, but I would avoid DynamoDB today and stick with something like Cassandra or MongoDB for general NOSQL work. Hopefully DynamoDB has changed since I last encountered it as the idea is a very good one and fits well in the serverless ecosystem.
I've had issues with the temp memory. It is completely unreliable, you have to store files when processing in /tmp/ and for whatever reason, reading from this results is patchy. I had to resort to storing every file I process in the RAM.
The tooling is only just taking off now, 6 months ago simulation was painful, at least there is AWS SAM now and docker support.
However despite the pain, it's giving me a solution for scalable video encoding that is very cost competitive and I don't have to worry about submitting jobs and monitoring resources.
Lambda is designed to be used for tasks like that, such as resizing images, parsing file etc.
But what I mean is that its clearly not designed for you to be dropping files onto the local disk.
Video encoding is kind of a corner case in that the size of the data really stops you from doing the whole thing in memory, though the persistance isn't needed in between runs.
Probably what you want to look into is attaching an EBS volume to the job, and doing the work from there.
Surely this is not the best use of the service, even if it is common.
If you need to run tasks on a schedule what’s wrong with cron? Or a cron service?
[0]: https://docs.aws.amazon.com/lambda/latest/dg/tutorial-schedu...
I know it works and it fills a gap for people but it just feels odd to me. It's like there is cron -> log -> monitor -> alarm -> event -> lambda -> script when you could just have cron -> script.
I'm probably just hung up on the naming and the fact that these events are branded as part of Cloudwatch. If there was some service spun off that just generated events in AWS I wouldn't raise an eyebrow, even if under the hood it was exactly the same thing.
Lambda has been sold on "saving costs" but what seems to be often neglected is the developer time spent re-jigging into a "serverless" model. Developer resources are far, far more expensive than mere EC2 instances.