I worked as a software engineer in bio and am building a company adjacent to this. There's a subset of popular software – especially in computational science and machine learning – that doesn't package well with pip or conda. For those tools, containers are actually a good solution.
Once things are containerized, it's still annoying to cycle between testing code locally on a laptop vs. running in the cloud. I hit this recently with Stable Diffusion on my laptop vs. running on an AWS GPU – I spent most of my time shifting execution between the two.
Lug is a semi-sane interface for running commands in a container and deploying briefly to the cloud: it reroutes Python system calls (like subprocess.run) to the Docker container. It pickles the function and its dependencies, so it's a single argument to switch between local and remote execution.
1 comment
[ 5.0 ms ] story [ 14.1 ms ] threadOnce things are containerized, it's still annoying to cycle between testing code locally on a laptop vs. running in the cloud. I hit this recently with Stable Diffusion on my laptop vs. running on an AWS GPU – I spent most of my time shifting execution between the two.
Lug is a semi-sane interface for running commands in a container and deploying briefly to the cloud: it reroutes Python system calls (like subprocess.run) to the Docker container. It pickles the function and its dependencies, so it's a single argument to switch between local and remote execution.