HN Explorer

Show HN: I made a local development, debugging and hotreload tool for AWS Lambda (github.com)

111 points by sozal ↗ HN
MerLoc is a live AWS Lambda function development and debugging tool. MerLoc allows you to run AWS Lambda functions on your local while they are still part of a flow in the AWS cloud remote.

MerLoc makes it possible to - test your function locally without deploy to the AWS Lambda environment (so no wait for build, package and deploy) - debug your function by putting breakpoints from your IDE - hot-reload updated function on your local automatically to apply changes automatically (so again no wait for build, package and deploy) - run the individual function locally while it is still part of flow shown above and use real requests from the AWS Lambda environment

50 comments

[ 2.6 ms ] story [ 70.8 ms ] thread
cube2222
That looks cool! Though it looks like it only works for Java-based Lambdas, unfortunately.

Does it only work with HTTP Lambdas / manual invocations, or does it also play well with others, like SQS subscriptions? I think that's the biggest pain point for local Lambda development with the AWS-provided tooling for me right now, as detailed in a blog post I wrote a while ago[0].

All the official tooling only lets you do HTTP calls and manual invocations for locally running Lambdas, which means you can't really test architectures comprised of multiple Lambdas, or where Lambdas are just a part of the whole setup.

[0]: https://spacelift.io/blog/aws-lambda-migration

sozal
Hi,

Yes it is only for Java runtime but Node.js runtime is also in our near-term roadmap. If you are looking for Node.js runtime, could you please upvote issue here: https://github.com/thundra-io/merloc/issues/1 or create a new one if you have any other request.

It works for *all* Lambda invocation triggers: Manual, HTTP, SQS, SNS, Kinesis etc ...

heisen_berg
Is this project open source? If so, is there any limitation to install any local environment?
sozal
Yes, it is open-source on Github. You can clone repository and install the platform to your own AWS account and run it inside your own network.
aoozdemir
Does the hotreload feature comes out of the box? Is there any limitation regarding what can we reload while debugging? Could you expand that feature a bit?
sozal
Yes, hot-reload features comes out of the box and you don't need install any other tool or service.

There is no limitation for debugging. When you changed any source code, it is compiled and then related functions are reloaded on your local automatically. So when you apply a change or fix for your function, you can check whether it works or not without deploy to AWS Lambda environment. So it increases serverless application development productivity by helping you developing and debugging faster.

somethingAlex
I find the debugging and testing process pretty slow with AWS SAM so I may have to try this out when you have a Node runtime.
sozal
Yes, there any many serverless people looking for Node support. Could you please thumbs-up its feature request here: https://github.com/thundra-io/merloc/issues/1
sozal
[Disclaimer] I am the co-founder and CTO at Thundra which is serverless monitoring and debugging solution.

We also have developed another debugging tool to debug AWS Lambda function on remote AWS Lambda environment from your local IDE: https://www.serverlessdebugger.com/

It doesn't support hot-reload but very useful for debugging functions.

tunesmith
Doesn't something like localstack.cloud help with this need? I've just recently inherited some NodeJS lambdas and am thinking of looking into that to debug/develop locally.
edgyquant
Local stack seems to be invite only
kadoban
I suspect you misunderstood. It's Open Source, at least the free version.

Or maybe I'm missing something?

trwhite
There's a free trial that lasts 2 weeks. After that I think it's paid.
tunesmith
Not exactly. The paid editions have free trials, but there's also a community edition that is free. However, it's pretty difficult to figure out what exactly is supported in the community edition. There's a whole list that mention various AWS services are only supported in "PRO", but I don't know if that means that free support might come later in community.
kadoban
All the basic stuff is in the community one. Certainly lambdas are.
trwhite
It must be API Gateway that isn't
sozal
Hi,

LocalStack is great tool but the difference here is that MerLoc runs your function in local against real cloud services with the same credentials of your real AWS Lambda function. So there is no AWS service simulation, mocking or local implementation here.

gee_totes
This is awesome and something I definitely would use for my docker and node based lambdas. Looking forward to this project adding more labmda backends!
_gabe_
It still amazes me that AWS doesn't provide good local debugging/development support out of the box. I feel like we're slowly going back in time. Iteration cycles are very high, debugging cloud dev is essentially placing a bunch of print statements and trying to trace the code, build times take forever...

Edit: Here's some more conversations I found on the topic of the terrible workflow for cloud dev if anyone is interested https://news.ycombinator.com/item?id=26855037

sozal
You are right, local development and debugging is still one of the biggest pains of serverless world. This was our motivation to develop MerLoc
_gabe_
It looks awesome :)

Unfortunately we have a full Javascript stack at my company, but I'll keep an eye on this in case anybody adds more dev environments!

user3939382
I just got xdebug working in ECS. It took really understanding xdebug config in depth but it’s amazing now that it’s working.
tlarkworthy
Yeah, you are right. I made https://webcode.run also for this reason (also hot code reload and debugging but for JavaScript computational notebooks)
[deleted]
(comment deleted)
scarface74
There is “sam local”.

But honestly, all a Lambda is a script with a function that takes two parameters - an event and a context. You can scaffold testing and development just like you would a controller action in an MVC framework - use a test harness that calls the method with the correct JSON payload and usually a null event context.

kkielhofner
This where Azure and the Microsoft ecosystem is pretty far ahead from a developer experience standpoint.

I’m not an MS fanboy by any stretch but VS Code, Azure Functions, and even local storage emulation with Azurite is pretty slick.

First time I played with it my background was very far from anything in this space (literally first time I used node) and I had a completely locally developed toy function with storage up and running in maybe 30 minutes. Then in a few clicks it was deployed and running in Azure.

All in less time than it took for me to wander through the AWS docs and console to even figure out which product to use.

vasinov
How is MerLoc different from SST?
sozal
Hi,

Current version of MerLoc is similar to SST except some minor details.

The biggest difference so far is that you can use MerLoc with any deployment tool like AWS CDK, AWS CloudFormation, AWS SAM, Serverless Framework, SST, Terraform, etc ...

And we are also working on an improvement which will let you use the same network with the remote function on your local.

kulikalov
You are missing the point. Lambda functions are made for tiny scripts or relaying events that barely need any debugging. Also, they are highly integrated with other cloud services so it doesn’t make much sense to bother about complex dev tools for them.
phailhaus
That's no longer the case. Lambdas have fantastic applications to more complex use cases, so better debugging capabilities are now necessary.
sozal
You are definitely right. Serverless is much more than 5 years before
linsomniac
Thus spake the Master Programmer: "Though a program be but three lines long, someday it will have to be maintained."
icedchai
Some people have gone full serverless, right or wrong (and I happen to think you are correct: Lambdas are best for simple scripts.)
spatley
this looks like a great solution to problems in serverless application development. I would be interested in how this does or does not overlap with the AWS toolkit:

https://aws.amazon.com/intellij/

https://aws.amazon.com/pycharm/

https://aws.amazon.com/visualstudiocode/

I have used the Pycharm in Python Lambda development and I was able to set a breakpoint in my IDE and hit it and see the status of all my variables.

Is MerLoc an alternative to that?

sozal
Hi, As far as I see, yhe plugins you mentioned just runs the function as a standalone app.

But MerLoc runs it on your local as a part of AWS cloud infr This means that function is triggered from real events, runs againts real AWS services and again response is returned to real AWS services

isuckatcoding
Are Lambda functions doing so much now that they they can’t just be run locally?

I get that if you’re trying to access some AWS service, this could be helpful but in that case I thought the general consensus is basically mock things out.

hk1337
Some people cram their entire application into a single lambda. Others are a little better and have their application spread out amongst multiple lambdas.
icedchai
The lambda developer experience is pretty awful. I generally put "main" methods in all my lambdas so I can test them locally, but otherwise hitting actual AWS APIs. I'm not a big fan of mocking. Doing any sort of edit/deploy cycle is way too slow.
phillu
I had great success with unit testing the lambda code and mocking external/aws services.

For serious software you want a proper local/unit test setup anyway. Using this in your dev cycles is just the natural next step. Instead of waiting for the cdk/cfn deploy to finish, which really is way too slow, you just execute 'jest' or whatever test framework you have.

Sure, figuring out how to mock everything takes some upfront time, but it pays off in the long run.

xsmasher
I always write a remote function as just "an async function that takes type A and returns type B" - then I can write tests for it very easily and run it locally, IF it doesn't use any AWS services like databases.

For deployment I have a generic "wrapper" that makes it lambda-compatible by converting a webEvent -> Json and my outputJSON -> web response event.

whummer
Disclaimer: I'm the co-founder and CTO of LocalStack (https://localstack.cloud)

This looks awesome, congrats Serkan and team! Indeed, looks similar to SST, with the extensions you've mentioned.

I can see this becoming very useful if you want the local debuggability of Lambdas in your IDE, with code running remotely. (i.e., not leveraging locally emulated services)

Looking forward to giving it a whirl and see how it can integrate with (or even run against) LocalStack..! :) Keep up the great work!

sozal
Thanks Waldemar. In fact it is already integrated with LocalStack
[deleted]
(comment deleted)
scarface74
What does this do that SAM local can’t do for testing Lambdas locally and “sam sync” for deploying a single function?

https://docs.aws.amazon.com/serverless-application-model/lat...

sozal
Hi,

The difference is that MerLoc runs functions on your local as a part of AWS cloud infa. This means that function is triggered from real events, runs againts real AWS services and again response is returned to real AWS services.

scarface74
Isn’t that slower than just mocking up your event payload?
sozal
Actually not. It is just one time setup and no mocking is needed anymore
yrgulation
This is actually a tool i’d need to use - thank you for creating it.

However i cant help but notice how large corps, such as amazon, build products on top of freely available software, intentionally leave gaps in their offering (aws local dev is horrible, amazon doesnt provide a local env so we wont build an alternative - but is happy to use ours) and then the community steps in and fills in the gaps. I yet have to understand how they get away with such things.

Hopefully amazon will have the decency to sponsor such projects.