They need TS incremental compilation for their TS problems. Which exists. And is used by other large projects to achieve milisecond compiles. But I guess they couldn't figure it out, not clear to me why.
Did the author not bundle their Deno project before uploading it? This would be as if they ran Webpack on Node on each start, so this isn't really a proper comparison
Aside: if you're doing that in production, save yourself some headache and use `npm ci && npm run build` instead; `npm install` chooses dependencies based on your package.json, not your lockfile.
> The main reason for this is the Deno runtime currently downloads external packages on cold-start. There are probably some mechanisms to speed this up, but that's for another post.
I thought I remembered reading that there's an environment variable you set to tell Deno where to look locally for packages? Or you pack everything into the script executing? I imagine one of those would provide a more accurate comparison to Node, if those are how Deno is expected to be run.
Dude, you didn't bundle the app or upload a folder with cached contents. This essentially nullifies the entire point of using Deno.
It's as if your Node function were re-installing all of it's node_modules in package.json each cold-start.
One of the major benefits of Deno is that you can run "deno bundle" which will spider all your imports and deps and create a single JS file that is the JS equivalent of an executable. Giant, 100 dependency app? "deno bundle" gives you "app.js" and just hand it to anyone and they can run it with no further tooling.
Beyond that, I'm not intimately familiar with Lambda's but if they allow multi-file uploads with relative imports, you could have uploaded the cache if you didn't bundle it:
Deno caches remote imports in a special directory specified by the DENO_DIR environment variable. It defaults to the system's cache directory if DENO_DIR is not specified. The next time you run the program, no downloads will be made. If the program hasn't changed, it won't be recompiled either.
Downloading dependencies on cold start not only adds to start latency, but is also an availability risk. You don't want your starts to fail when the package distribution infrastructure is down.
Where possible, it's best to include everything your Lambda function needs to run in the function .zip. That means you can take advantage of all the work we do to make the scale-up and cold-start paths of Lambda highly available and reliable.
Having said that, it's very cool to see folks using Deno on Lambda, and getting real numbers like this starts an important conversation.
thank you for filing, I appreciate the feedback. I've updated the tests to bundle dependencies and it does not make a large improvement to cold-start times and makes no difference in the duration average of warm lambdas.
If I missed something and can further optimize, please let me know.
I'm a little confused how the author could come to the conclusion Deno is not ready for AWS Gateway integration, when they openly admit the main reason for this is that Deno is downloading all packages on a cold start, and that there are "probably some mechanisms to speed this up".
To their credit, the author does point out another potential flaw in the disclaimers section - the Node.js AWS SDK is optimized for node, and AFAIK they haven't released a version for Deno.
Reminds me of a blog post from a couple of months ago where somebody was writing about "how to do X in emacs". The kicker was they were using doom emacs and didn't understand which packages or functions were actually allowing them to do their thing, nor that doom was highly customized. After getting eaten alive in r/emacs they deleted the post.
Hopefully this guy's post stays up and gets an added disclaimer about not writing shit like this without actually deep diving and learning the software they're comparing.
People are curious about Deno and where it will go. Right now it is a new project, so we have more questions than answers.
My pet question: Will there be a CLI-local debugger with a good UX like pry+byebug or ipdb?
With the current chrome-based setup, it takes far more click-and-drag in js than in python and ruby to stick a `debugger;` in a function, run a single automated test which calls that function, and see how variables are defined.
I like how "HOT" deno is becoming. A successor to node would be a great thing in my opinion.
That said, the fact that the author failed to bundle shows us one of the hurdles that Deno has to overcome. When the tooling is "batteries included" people need to remember to pull the little plastic tab out(bundling).
Hey all. Can't believe the response this has gotten! Glad people are reading.
I wanted to address a few points that were coming up in comments:
* Yes, I failed to bundle the deps with the Deno deploy, causing a long cold-start time as it downloaded deps. I was not concerned about cold-start times. I knew that there were ways to optimize it. I decided not to, as it simplified the tests. I was very careful about making sure duration measurements were running on warm lambdas. I don't believe I covered it in the blog but I stop-watched the dynamodb calls and could see them running around 100-120ms each. I was satisfied that the values I was seeing was largely due to API calls to dynamodb, and not the deno runtime.
* In re: API gateway. Yes, the cold-starts are an issue that I believe could be resolved. But even then, the API calls to dynamodb are STILL substantially slower than Node and would be too slow, imo, to be worth putting behind API Gateway where low response times usually trump a lot of other concerns.
* I encourage anyone that wants to try this for themselves to check out the github repo linked in the article. It contains a CDK module you can easily run and recreate all of this in your own AWS account and see for yourself. If I was interpreting this data wrong, PLEASE tell me. The last thing I want to do is be spreading misinformation. If you find easy optimizations, I would love to see a PR and I'll update the blog with your contribution (with your approval).
Thank you all for reading this. This is the first time a blog post has ever made it on to hackernews (afaik). I'm stoked with all the interest, even if I believe some of it is because of misunderstandings because I explained the tests poorly.
29 comments
[ 3.3 ms ] story [ 68.1 ms ] threadhttps://deno.land/manual/tools/bundler
Each subsequent request would not need to go through that process as long as the lambda function was warm, right?
I thought I remembered reading that there's an environment variable you set to tell Deno where to look locally for packages? Or you pack everything into the script executing? I imagine one of those would provide a more accurate comparison to Node, if those are how Deno is expected to be run.
It's as if your Node function were re-installing all of it's node_modules in package.json each cold-start.
One of the major benefits of Deno is that you can run "deno bundle" which will spider all your imports and deps and create a single JS file that is the JS equivalent of an executable. Giant, 100 dependency app? "deno bundle" gives you "app.js" and just hand it to anyone and they can run it with no further tooling.
Beyond that, I'm not intimately familiar with Lambda's but if they allow multi-file uploads with relative imports, you could have uploaded the cache if you didn't bundle it:
Where possible, it's best to include everything your Lambda function needs to run in the function .zip. That means you can take advantage of all the work we do to make the scale-up and cold-start paths of Lambda highly available and reliable.
Having said that, it's very cool to see folks using Deno on Lambda, and getting real numbers like this starts an important conversation.
(Disclaimer, I work on the Lambda team at AWS)
If I missed something and can further optimize, please let me know.
To their credit, the author does point out another potential flaw in the disclaimers section - the Node.js AWS SDK is optimized for node, and AFAIK they haven't released a version for Deno.
Why would they publish before looking into this first? At best it's incomplete, at worst it's misinforming readers.
Hopefully this guy's post stays up and gets an added disclaimer about not writing shit like this without actually deep diving and learning the software they're comparing.
We are also developing native infrastructure instruments for Deno. We will let you know when we do the first release at https://deno.services
My pet question: Will there be a CLI-local debugger with a good UX like pry+byebug or ipdb?
With the current chrome-based setup, it takes far more click-and-drag in js than in python and ruby to stick a `debugger;` in a function, run a single automated test which calls that function, and see how variables are defined.
That said, the fact that the author failed to bundle shows us one of the hurdles that Deno has to overcome. When the tooling is "batteries included" people need to remember to pull the little plastic tab out(bundling).
Hmm, is it though? Maybe take those extra couple hours to figure it out
Bundling the dependencies would very likely solve the init issue, but I wonder if it would help with the execution of the lambda function?
That seems to be what the author is mainly concerned with.
I wanted to address a few points that were coming up in comments:
* Yes, I failed to bundle the deps with the Deno deploy, causing a long cold-start time as it downloaded deps. I was not concerned about cold-start times. I knew that there were ways to optimize it. I decided not to, as it simplified the tests. I was very careful about making sure duration measurements were running on warm lambdas. I don't believe I covered it in the blog but I stop-watched the dynamodb calls and could see them running around 100-120ms each. I was satisfied that the values I was seeing was largely due to API calls to dynamodb, and not the deno runtime.
* In re: API gateway. Yes, the cold-starts are an issue that I believe could be resolved. But even then, the API calls to dynamodb are STILL substantially slower than Node and would be too slow, imo, to be worth putting behind API Gateway where low response times usually trump a lot of other concerns.
* I encourage anyone that wants to try this for themselves to check out the github repo linked in the article. It contains a CDK module you can easily run and recreate all of this in your own AWS account and see for yourself. If I was interpreting this data wrong, PLEASE tell me. The last thing I want to do is be spreading misinformation. If you find easy optimizations, I would love to see a PR and I'll update the blog with your contribution (with your approval).
Thank you all for reading this. This is the first time a blog post has ever made it on to hackernews (afaik). I'm stoked with all the interest, even if I believe some of it is because of misunderstandings because I explained the tests poorly.
-M