Show HN: Depot – fast, remote Docker container builds (depot.dev)

This is how they all scale builds, and hopefully things become more standardized with a reduction in vendor lock in. It is surprising to me how much of the world relies on moby/buildkit, and scary if you've ever looked at the code. Primarily a single developer, not straightforward to understand, comments & docs are basically non-existent. Looks like that might be starting to change with so many companies now building and SaaS'n on top of BuildKit.

That works, but we've considered adding a `depot bake` command that does the same thing as `docker buildx bake`, to make the integration nicer. Ideally docker-compose would have the ability to communicate with buildx drivers (that's what Depot registers as), but to my knowledge it doesn't currently have that option.

I used it, didn’t have any issues but i realize there’s lots of reasons that might not work.

Things where Depot may not be a good fit are when you only ever need Docker images for local development. In those situations, the network becomes the bottleneck because you have to pull down the built image each time. That said, if you need to build multi-architecture images locally, Depot becomes useful again as it can build native images for each architecture, no emulation needed.

If you're entirely on-prem, we don't currently support that so Depot isn't a fit there.

1. Just build the image — in CI that would test that the image built successfully, and the build cache would be ready for future builds. So for instance if you ran a build on a branch, but only pushed the image on the main branch workflow, the second run could just re-use the cache.

2. Build and push the image to a registry — from there you could do anything with the image in the registry (pull it from any workflow, deploy it, etc)

3. Build and pull the image to the local Docker daemon. In CI, that might be ideal for integration testing the container, like you mention.

You can also use option (2) for integration testing the container, which is especially useful with multi-platform images. Docker doesn't (yet) support storing multi-platform images locally, but it will pull the correct platform from the registry.

tl;dr — Depot supports the same options as `docker buildx build`, where you can push, pull, or just build the image depending on your needs

It would be cool to add an integration between Depot's CLI and Dagger's CLI such that Dagger could communicate with the remote builder machine. You could use Dagger with remote compute without needing to manage that compute, effectively. Depot even supports launching builders within your own AWS account, so you'd have that same ability with Dagger as well.

I struggle to think of what I would need a SaaS service to run BuildKit for me unless they're trying to take on GitHub Actions's compute side or whatever

When you start a container build with Depot, it will launch an Intel or an Arm VM running BuildKit (or both for a multi-platform image). It creates persistent SSD cache volumes for each project and attaches those to the build machines at launch. And it automates connecting your CLI to those machines over TLS. Then when the build is done, it shuts down those machines to save cost.

Depot does all that either in our cloud account, or you also have the option of connecting your own AWS account and Depot will act just as the orchestration layer.

So basically you get the remote build machines, but without needing to set them up, manage costs, configure access control, etc. But at its core, it's running the open-source version of BuildKit (the Depot CLI embeds the same Go library as `docker buildx build`).

arm64 builds should even be a more dramatic change, they should drop to at least as fast as your current amd64 builds.

Things that took seconds to build on my local machine were taking double-digit minutes to build in GCP. I ended up building everything with gitlab-ci using kaniko and pushing to GCR, insane speed up.

With Cloud Build, you need to use —-cache-from and other amendments to leverage remote caching. (And I believe that caching multi-stage builds require you to explicitly bush and push every stage.)

With Depot, you should in theory get all of that caching “for free”. You can also take advantage of things like BuildKit cache mounts (local directories on the host machine to cache, e.g., pip install artifacts) which AFAIK can’t be done with Cloud Build.

(No affiliation with Depot but I have played around with it.)

The first thing that comes to mind is multi-platform support. Currently, it's very slow to build multi-platform images in things like Cloud Build because you have to emulate the Arm portion of the build. We don't have timings in Cloud Build for this, but in GitHub Actions a multi-platform image build can take over an hour when building for both Intel & Arm. With Depot, those builds can take less than 5 minutes because we build the image for both architectures in parallel on native CPU architectures.

Other things where Depot might be more streamlined than Cloud Build is that persistent caching is automatic. The dedicated build machines for each architecture have persistent SSD caches attached to them, so you don't need to save/load cache in between runs, it's just there ready to be used.

We currently charge on a per project basis as behind the scenes we are provisioning and maintaining a VM with a persistent SSD cache for each architecture you are building for (i.e. Intel & Arm). We switched to usage based pricing before rolling Depot out to the public so you are only charged for your active projects and used build minutes in a month. So far it has seemed to be working well, but we are always open to feedback and ideas on how we can make pricing better.

We don't currently document this, and we probably should, but it's entirely possible to use a single Depot project for multiple Docker images. There is no limitation that prevents this and the only downside is you may fill up your cache space at a faster rate and get fewer hits if you have different image builds clobbering the cache.

So your pipeline would effectively look like this.

    image: node:10.15.0
    pipelines:
      default:
        - step:
            script:
              - curl https://depot.dev/install-cli.sh | DEPOT_INSTALL_DIR=/usr/local/bin sh
              - npm install
              - npm test
              - docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD
              - depot build -t <my.container.registry.io>/<my_app>:<my_tag> . --push --project <DEPOT_PROJECT_ID> --token <DEPOT_PROJECT_TOKEN>
            services:
              - docker

Edit: Formatting. Also thank you for pointing out this is an integration we are missing in our docs. We will get that added!

Our biggest cost at the moment is persistent disks. Each project gets its own SSD(s) that live for the lifetime of the project, and that's a major part of why Depot builds are so much faster and support BuildKit features that require persistent disks (e.g. RUN --mount=type=cache). We have some ideas for how we might improve this in the future, keeping the same or better performance profile but avoiding the continuously-provisioned disk, we'll see how that goes.

One note on "per project", each Depot project represents a pair of Intel/Arm VMs, each with a 50GB SSD. But there's otherwise no restriction on what you build using those projects, you can use a single project with multiple repos/Dockerfiles/whatever. Projects are the access control boundary, and building multiple disparate images using the same project would cycle the cache faster, but otherwise there are no limits on concurrency or what's building inside those VMs.

You very well could build everything on just the single $25 project, there aren't any restrictions on concurrency or what images you build on what projects.

    steps:
      - uses: depot/setup-action@v1
      - uses: depot/build-push-action@v1
        with:
          platforms: linux/amd64,linux/arm64

I have some ideas I want to explore with Depot, but it really it feels like it should be possible to make cross-compiling and dependency encapsulation easy to use, so "building a container image" is reduced to creating a tarball in the correct OCI format, and you don't have to interact with Dockerfiles as much if you'd prefer not.