Ask HN: A Better Docker Compose?

I'm not really sure what problem you're trying to solve here; if you want to follow DRY, there are YAML tricks you can use[0], but if you want docker-compose to "figure it all out" itself, I don't think there are a safe set of assumptions it could make to reliably meet your expectations.

Also I'm not sure it really matters; again this would help to understand your question better, because I think you're trying to solve a problem that may not exist in your threat model.

[0] https://medium.com/@kinghuang/docker-compose-anchors-aliases...

I find systemd works pretty well

I solve the secrets problem by using ansible to provision the host system and build the compose file, and ansible-vault to handle the secrets.

You can also dynamically build a compose file for example:

export COMPOSE_FILE="$(ls -1 services/*/compose.yml | paste -sd ":" -)"

In general I don’t have many issues with the docker-compose.yml format.

At the risk of sounding clichè/unhelpful.

> “you can solve every problem by adding or removing an abstraction layer”

Have you considered generating your YAML/JSON config with something that composes?

If you are open to it you might be interested in dhall [1] as it’s a config language with variables, functions and imports.

I have used it for pet projects and I could see how it could offer some tidy encapsulation patterns for larger, more complicated production applications.

[1]: https://dhall-lang.org/

I'd like to ask the questioner and to anybody who would like to comment:

As docker compose is only for single node deployments, how is your workflow from dev to production? If your containers are deployed in something like Kubernetes in production, isn't inconvenient to use docker compose for development?

[dead]

I find this really annoying. There's also no concept of a package like Helm Charts which means you can distribute all your images on Docker Hub but then when it comes to compose files and configurations with templating your up @$%^ creek.

I was hoping something like https://dagger.io/ might solve this but they currently don't support compose files, or maybe https://www.acorn.io/ ?

It does feel like there is a legitimate space for a multi-container service application standard that isn't just kubernetes, unless perhaps things like k3d and docker desktop end up so good at running tiny k8s that docker compose dies.

Docker's network 'isolation' isn't as bulletproof as you imagine, especially if you're on Linux. From your host machine you can access every single service IP directly which is sometimes unexpected as you might assume it's firewalled off. I'm pretty sure long ago docker used to warn folks their network isolation isn't to be used in any production security sense, it's strictly for developer convenience. If the config is uneweildly you could just throw everything into one default network, as it's still just as secure as that when you're defining individual networks (i.e. neither is super secure).

Probably not quite what you're looking for but perhaps along the right lines (and it looks like they've added some products and maybe features) https://traefik.io/traefik/

The gist of it is Traefik is a reverse proxy that learns about your containers via configuration in the container only, so you can stand up additional services without rebooting everything. Still config heavy, but maybe less than some other approaches.

Docker compose is a dead end AFAIK, it's not deprecated or anything, but the community power behind it has essentially moved to other things (Kubernetes, but also wasmer, nomad, skaffold).

While Docker Desktop / Podman / Rancher Desktop combined with stuff like Skaffold aren't exactly a drop-in replacement for docker-compose, it does do a much better job at bringing up and tearing down entire compositions while re-using existing packaging and access controls.

If you are running docker-compose for non-development things, it might be a different story; it might be suitable for non-GitOps things, but as posted elsewhere, at that point you're better off using something like systemd.

When composing really small setups I either do this with a shell script (think 10 lines including trapping exits) or a systemd unit. Whenever it needs to be bigger I nearly always end up with an actual scheduler (K8S, Nomad) and GitOps because you can't really deliver something maintainable, available and durable anymore without it (well... I suppose if you have only 1 project to deliver, forever, you could manually manage it).

It does get a whole lot easier when you have a common foundation you can re-use. Spinning up an entire stack with security, monitoring, alerting, automatic rollouts/rollbacks for even the smallest project is just single-digit minutes work now.

Pulling in some other factors: how sharable/collaboratable is something these days if it is not built on similar enough technologies and modules? A solo yolo project might not care much about this, but when was the last time someone asked for software that is risky and not durable?

Could try Nix with Agenix. Modularized and composable, and secrets can be stored encrypted in git.

Personally after using docker compose since it came out I’m excited to see the evolution of local Kubernetes development. I’d scrap even dealing with trying to make something as limited as docker compose do what you want. I’d focus on moving towards local Kubernetes development.

This will bring you closer to the deployment stack if you are deploying to Kubernetes. Then also let you leverage tools like kustomize to dry out your configurations.

There are some great projects like tilt, devspace, skaffold, etc that help facilitate deving on a local or remote cluster.

As far as configuration management that can be as simple as cascading kustomize configs or helm. Then leveraging something like vault. The point really is, if you start with Kubernetes you have way more flexibility with tooling and options to do whatever the heck you want.

Shameless plug I recently started a series on local Kubernetes development. It covers some of this with tilt and more. If you would like a specific thing covered here I can add an installment to it. https://youtube.com/watch?v=Nj55RDVwrIE&si=EnSIkaIECMiOmarE

I’ve been spending a week trying to learn how to deploy a collection of containers (my web app, a Postgres DB, and some microservices) to AWS and I am still so lost.

The first solution I happened upon was serverless. Specifically SST, which is written with AWS CDK, but you must develop on live services and I just can’t justify paying to develop.

Then I found Serverless Framework, which is an abstraction on CloudFormation, but the offline solutions like localstack get a lot of flack for being buggy and localstack charges for some services. I also looked into Architect but the documentation is abysmal.

Then I figured serverful might be the easier way to go. I found that docker compose has a built in integration with AWS ECS where it transforms your yaml into Cloudformation to provision the right services. However, it seems to just be missing key parts like custom domain and SSL certificate provisioning which seems to defeat the IaC ethos.

Then I figured I might go with Terraform and I found some seemingly good starters like https://github.com/aws-ia/terraform-aws-ecs-blueprints https://github.com/cloudposse/terraform-aws-ecs-web-app https://github.com/turnerlabs/terraform-ecs-fargate but the examples are just lacking. They don’t have any examples for multiple containers that can access each others’ resources that I can find. Reading these templates has at least given me a better idea of the resources I need to provision in AWS but the networking and configuration still frighten me. Like do I need to configure nginx with a reverse proxy myself? How do I orchestrate that container with the others? And apparently services can crash and just not restart? And I need to make sure to configure volumes for data that needs to persist. And setting up the CI/CD seems daunting.

I’ve also heard about docker swarm, kubernetes, pulumi, AWS SAM, etc but it’s a lot to learn. When I go on Discords for web frameworks, mostly everyone including the devs of these frameworks use 2nd tier managed providers like Vercel, Fly, Netlify, Supabase, Cloudflare, etc. But many of those are just not as reliable as core cloud providers, the cost is way higher, and now you’re setting up a local stack that probably wildly differs from how it will work in production between those services. Glad to see I’m not alone in a very reasonable expectation of a simple way to orchestrate multiple containers on AWS, what must be the most common use case web developers have

Simple, you pay for a PaaS. I know this is unpopular but it is why platform as a service exists — to abstract away the typical configuration patterns and let somebody else deal with the devops overhead.

I find your example simple and easy to understand and I don't see any issues with defining everything explicitly.

I'm using docker swarm/compose extensively for multiple years and configuration can be a bit verbose sometimes but it's very simple system to maintain and write config for.

I think, splitting relatively small configs into smaller bits and using additional abstraction can actually increase complexity of the system compared to having everything in a single file as it's basically write and forget and not something you modify heavily once system is mature.

I use a monorepo (turbo) that has a directory: ./app/<docker workspace or app name>. In it I have a package.json that runs docker-compose up. I use regular .env files and keep a .env.sample in the repo. The docker-compose brings up the services and volume mounts the .env into it's local environment (same with /data for example postgres, or some other package).

I then disable iptables in /etc/docker/daemon.json: "iptables" : false to prevent docker from poking a hole in my firewall. (This should be on by default..) I also enable User name spaces with this: "userns-remap" : "default" so that the docker image runs isolated. I can disable this in my docker-compose with userns=host Actual instructions are slightly different, but: https://docs.docker.com/engine/security/userns-remap/

I also found I could disable networking all together on some services, and just expose a port. Some services don't need to connect to the net.

Then I run a cloudflare argo tunnel to the application. But you could also use a nginx reverse proxy, some other tunnel, or a one page express.js reverse proxy with pm2. There's a cloudflare argo tunnel docker-compose setup which allows you to type in your tunnel information in the config. I thought that was cool but I did not try it.

You can use environment variables with error messages

https://docs.docker.com/compose/environment-variables/

   - `${VARIABLE:?err}` exits with an error message containing err if `VARIABLE` is unset or empty in the environment.
   - `${VARIABLE?err}` exits with an error message containing err if `VARIABLE` is unset in the environment.

> Is there a way to abstract away these extra configuration steps and keep it simple yet secure by default?

No, sorry.

The way it works in the cloud is, you build some infrastructure, you give that infrastructure the ability to contact a secrets management service, and you either A) configure a container orchestrator to look up a secret when your service starts and inject it at start time (either as an env var or as a file), or B) your service itself looks up the secret using permissions inherited from the environment it has started in (such as an instance metadata service).

You could fake all of that in a crappy way by running an additional service which is just a web server with a complicated random URL for each secret you want, configure each service to declare an environment variable which is the complicated random URL, and have each service curl the URL at start time to retrieve a secret. But that's not less configuration, it's slightly more. The only benefit is that you can kill the secret service after the URLs have been grabbed, leaving the secret only in memory (and assuming the URL was rotated every time this whole thing started, would make them temporary, sort of).

But, as a hacker, that would all only present small challenges which I would eventually work around once I find an attack to read from memory or execute code. So unless you need super duper extra security for some reason, just use what you've got.

> Is there a way to abstract away these extra configuration steps and keep it simple yet secure by default?

Secure from what? What is your threat model? It's hard to imagine 'default' security because everyone is worried about different things.

> I just want them to talk securely and minimize any attack vectors and keep any configuration to a minimum

Your application is more likely to expose a SQL injection vulnerability than someone gaining unauthorized access to your network. I'm not saying you shouldn't secure it, I'm just saying that it is probably pointless since your application has access to :all-the-things: and it is probably the weakest link.

Further, if you're doing this on a developer machine, you're just making it harder to debug issues. As the user of the host machine, you have access to everything by default.

> How do you do (not the configuration management) but manage an inventory of what configuration is needed to run your environment (or a subset for ci)?

Documentation, documentation, documentation. Basically, have a markdown file with each config item, it's value in each environment (not literal values, but ex: 'github token for production, with X,Y,Z scopes'), why it exists, and who is responsible for it.

Arion is a Nix wrapper tool for docker-compose. Nix is handy for use as a template language. It is actually closer to a general-purpose language than to a template engine, so I guess what you want can be implemented manually, since it only requires config file generation.

Arion docs: https://docs.hercules-ci.com/arion/

Nix language: https://nixos.wiki/wiki/Nix_Expression_Language

  ### Databases
  zipcodes = {
    name = "zipcodes"; 
    pkg = customPkgs.zipcodes; 
    dependsOn = {};
    type = "mysql-database";
  };
  ...

  ### Web services

  ZipcodeService = { 
    name = "ZipcodeService"; 
    pkg = customPkgs.ZipcodeService; 
    dependsOn = { 
      inherit zipcodes;
    };
    type = "tomcat-webapplication"; 
  };
  ...

  contextXML = ''
    <Context>
      <Resource name="jdbc/ZipcodeDB" auth="Container" type="javax.sql.DataSource"
                maxActivate="100" maxIdle="30" maxWait="10000"
                username="${zipcodes.target.container.mysqlUsername}" password="${zipcodes.target.container.mysqlPassword}"   driverClassName="com.mysql.jdbc.Driver"
                url="jdbc:mysql://${zipcodes.target.properties.hostname}:${toString  (zipcodes.target.container.mysqlPort)}/${zipcodes.name}?autoReconnect=true" />
    </Context> 3
  '';

Create your own yml based schema and a script converting it to docker compose syntax.

It sounds like a simple straightforward task.

I use Polycrate [1] in conjunction with Ansible to create complex / monolithic systems with Docker / Swarm and to create a simpler interface for the end user than docker compose provides.

[1] https://docs.polycrate.io

Just saw Borg, Omega, Kubernetes: Lessons learned over a decade (2016) (acm.org) https://queue.acm.org/detail.cfm?id=2898444 pass by.

" Dependency Management

Standing up a service typically also means standing up a series of related services. If an application has dependencies on other applications, wouldn't it be nice if those dependencies (and any transitive dependencies they may have) were automatically instantiated by the cluster-management system? "