Hi, I'm the creator of Docker and came across this post. It's always fun to see other engineers think about how you designed a system. I haven't been involved in developing Docker in many years, but it's a fun trip down memory lane.
Oh wow, hi! I think my biggest “gripe” with docker is that everyone grabs it as a hammer and views many problems as a nail. Am I way off base with that? What was your original vision for how docker might be used in the wild?
There are many. But there is an alternative universe where I remained CEO instead of transitioning to CTO, and continued pursuing my product vision obsessively instead of compromising on it because of lack of self-confidence and a desire to be loved. I think I would have avoided a lot of the mistakes that the company made - but on the other hand, I would have made a fresh set of mistakes instead, so who knows what the outcome would have been!
Thanks. I needed to hear this. So many things in life can turn out differently if we can overcome our “lack of self-confidence and a desire to be loved”. It’s never too late!
If you were implementing an operating system (free from compatibility concerns), what would you do differently to better support lightweight containers?
I'm curious what made think of it in the first place? What was your thought process that made you think that these Linux primitives give the opportunity to make it a platform for distribution and deploying package applications and eliminating a lot of the issue around "it works my machine" argument?
You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)
The LXC way of getting an up per container is pretty great - I’m basic bitch with this sort of thing so I assume it’s less of an issue for the experts but really enjoyed the IP per container when I hit it.
This is sometimes easier, but once you have a docker-compose of what you want, it’s hard to beat. You can have running server with all the services and network shares all sorted in minutes.
Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.
The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".
As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
Huh, according to wikpedia, OpenVZ and Solaris Zones (Containers) were both released in 2005. My memory was that Solaris was first and OpenVZ was a copy cat, but I guess not.
“Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough”
Developers are a tough market to please, especially when your business revolves around open source software.
Unless you have an option to do private cloud hosting (like all these databases and other platforms do), it’s incredibly difficult to monetize.
In the case of docker, I think it’s in the nature of the product that it’s difficult to monetize. Best you can do is enterprise support contracts, but those aren’t good enough to justify all that VC funding.
I don't think it's necessarily that difficult to monetize, they just ignored a lot of the needs of enterprise users (rootless, not punching holes in the firewall, not fighting with systemd, private registries) for long enough that other projects (e.g. Podman, Quay) filled the void.
None of those things were obstacles to commercialization. None of them were common requests from enterprise customers except private registries, which Docker did build.
Podman has never been commercialized.
Docker's difficulties in commercializing were self-inflicted. Enterprise and open source teams were culturally siloed and as a result the enterprise products were not "native". There was a lack of focus and a "checkbox" approach to building product. There was no singular "dictator" who could enforce a coherent product design and vision, and it showed. etc.
I was a DotCloud user before Docker, and I remember keenly thinking - “oh, is that how they went from hosting only PHP to any language/stack?”
I always assumed it was the same internal tech, now productized. Then of course, it became THE tech, overtaking vagrant and anything else for local development and deployment. The latter was the true money shot. Server runs exactly what you test with locally.
BTW just for giggles opened hub.docker.com with Network Tools opened:
368 requests, 9.62MB, Finish: 4.50s.
For a thing which should be a static cached copy with everything bundled. Sure, it's a progress from the previous iterations when it could hang for 30 seconds before actually loading... but this is still ridiculous.
> Like they told you 9 months ago, the terminology is exactly right
Oh, well. I guess you ride to your work on wheels, right? Because while you sit in the car/bus/train it's the wheels what actually rides on the road/track, right?
And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
You see, if they just called it as it should, ig:
bridge => hostnatnetwork // or internalbridgenatnetwork or whatever else
macvlan => bridge
then they would be both named correctly and be the technically correct - and 99% of Docker users couldn't tell a difference - because they are not net or sys administrators.
But the last 1% is the people who actually makes the workspaces for these developers where they actually do run their fancy-shmancy containers. And for that 1% it surely pays naming this as they are.
And there is also that dumb decision to silently rewrite 127.0.0.1 to Google DNS without a question. Especially when 98% of Docker users are incapable to diagnose it by themselves.
So it's not only the naming of a network functions.
But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
> RUN command has had heredoc support for 5 years now
So ten years late? Adoption is off the limits, as you can see - that snippet is from the official Docker image.
And if I need to run multiple commands and not just one RUN statement? Like the people are asking for years? And you so knowledgeable person of course do know the reason for that, right?
> And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes learning and communication harder by introducing ambiguity and indirection. "bridge" now means multiple things and not only is it needlessly obfuscated, it's wrong (explained later). I _hate_ projects that do this.
> Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
Rhetorical question, maybe it'll help you find a source of misunderstanding: Why do you expect it to join the broadcast domains of your containers and your LAN? And which LAN, exactly?
> No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN. Therefore you incorrectly expect and demand that term "bridge" accurately describe the logical connection between your container and your LAN.
Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
A Docker 'bridge' network is not necessarily routed to LAN either so your suggestion of calling them "hostnatnetwork" is wrong. Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
> But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
All of this networking terminology is consistent and applied correctly but for some reason it's not clicking with your previous knowledge and you're trying to fit a square peg through a round hole. Take a step back, investigate.
> So ten years late?
Don't shift the goalposts. You claimed it still doesn't exist.
> No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
I think you just proved his point. Docker came onto the scene and hit roadblocks in enterprises where sysadmins lived. These architectural and terminology choices ran head on into well established domains. Yes, docker did change things, but I think his point is it would’ve had more success with paying customers if the authors of docker had a better understanding of established network administration and system administration concepts from the start (though these were the early devops days…). That lack of understanding caused adoption friction in enterprise space, and different choices could’ve prevented that without changing at all the paradigm of what docker is.
The docker mental model isn’t crazy different. Design choices made it unnecessarily less compatible with existing mental models. This probably contributed to the feeling of it being revolutionary, but impeded its adoption with paying customers.
> Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes
Sorry, but you do insist on naming 'docker' bridge a bridge because it's underlying tech is a bridge. Make up your mind.
> A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
And noone absolutely ever cared about what it's a bridge down there. It could be using gronklew and maybe 20 people on the planet would care to learn what it is.
You are quite bent on it, but nobody cares how the containers talk between each other.
> Why do you expect it to join the broadcast domains of your containers and your LAN?
I expect a setting named 'bridge' in some $thing to actually implement a bridge.
Just like you pressing an AC button in your car expect a very specific thing to turn on/off and not 'Artifically flavoured windshield wipers served in a Container'.
Why do I want it is on me. Maybe I need it. Maybe I control my network and freely give out addresses to anything. Maybe I don't run on some cheap VPS somewere with only 1 IP I can't choose nor change. Bazillion of options. None matters because this is not about 'why do you want it'.
> And which LAN, exactly?
The path of a least automagik.
Only one ethernet interface? Then connect to that ethernet interface. Bitch in the logs what automagik was used and this can lead to the problems.
Many interfaces? Just shutdown (least pain for newbies) or start without attaching (harder to troubleshoot for newbies). Bitch in the logs in either case.
> Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN
'Your misunderstanding seems to stem what there are some people assume what they are driving in like c-a-a-r. When even 5y.o. knows it's the wheels what drives you around.'
> Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
Yes, exactly. Don't call it 'bridge' ~20 years ago and we don't have this talk here.
> Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
It's not even a property of a bridge network. For someone claming knowledge superiority this is a quite lame statement.
> links
Really.
> previous experience in an adjacent area where
Ah, yes, irrelevant 'adjacent area' of computer networks. I'm out of words.[0]
> All of this networking terminology is consistent and applied correctly
No. Go drive your wheels home.
> Don't shift the goalposts. You claimed it still doesn't exist.
I'm just mentioned RUN as an example. You decided what multiline is the goalpost.
And yes, if even the official Docker images doesn't use RUN heredocs then... there is something wrong with them. Like being implemented too late.
[0] but perfectly explains your previous 'flawed assumption'
It's quite amusing to see what you are skipping anything what expictly against you. RFTMing is a quite bold move but whatever makes you feel good today.
I'm skipping it because it's a waste of everyone's time, you're incapable of accepting new information, but I'd be happy to be proven wrong:
First you incorrectly claimed that multi-line RUN commands aren't supported still require that ugly legacy syntax. When that was proven wrong, instead of saying "good, I wasn't aware", you went on a tangent about its adoption and wanted to interrogate my knowledge -- Yes, obviously I know that running all commands in one RUN block is beneficial because it reduces layer count and image size.
Your revolutionary idea of "transactions" is already implemented through multi-stage builds. Except those are even better than transactions because individual steps are still cached, independent components can be built in parallel and they don't invalidate each other's cache. Finally the artifacts are copied into the final image, leaving behind unnecessary layers.
I think there are some interesting points there, but your rude, arrogant, and aggrieved tone overwhelms them.
If you decide to edit your tone, I'm happy to discuss more here re: filesystem transactionality tradeoffs, since that's something I've struggled with in both directions WRT containers.
I'm only responding in kind to GP - no less but not more.
I think you are the one who built an alternate layering system? On mobile, don't like to search rn.
If yes the basics are in the first comment- just not limited to the one RUN statement ofc. And if I do remember how the layers are made then making it transaction-like would be a great dedup saver eg for adding a static rarely changed artifacts - there could be essentially a ~one layer for how many you want images.
Implementation wouldn't be so difficult - you just stop committing until TRAN END; but the main is too provide a mental model for a not very experiencied dev how this supports his work and makes it more efficient on the storage and execution.
>I'm only responding in kind to GP - no less but not more
bullshit, you've been cunty right out of the gate since your first reply. I'm just surprised anyone else in the thread had as much patience as they've shown here.
you seem exactly like the type of person who is incapable of realizing they're a complete asshole and wonders why people choose to avoid them.
> BTW just for giggles opened hub.docker.com with Network Tools opened: 368 requests, 9.62MB, Finish: 4.50s.
lol What on god's green earth does this have to do with commercializing Docker? Do you really think better lighthouse scores on hub.docker.com was what was holding them back?
Did you miss 'up to 30 secs' in the previous comment? It's not about lighthouse scores. It's the whole feeling of a hodge-podged project which is run by people who are - maybe - passionate but ultimately lacking in the basics.
The ultimate buyers of their product was the system operators and the system owners - not the developers per se. They failed hard everything that space. Having a dog slow the main content site didn't added anything positive.
> As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
yes, and no. Probably both. Docker would not become such a default standard if it wasnt for all the open sourcing.
Oh gosh, the choice of language is such a context-specific design decision. The first versions of what became Docker were written in Python. The choice of Go was a mix of nerdy excitement ("it's like the best of Python and C!"), and adoption strategy: back then devops tools were segmented by language runtime: ruby, python or Java mostly. Shipping a compiled binary allowed us to be adopted by all tribes.
None of these things can be transposed today, so I guess I can't answer...
For what it's worth, my current project (dagger) is written in Go.
The article's point that Go is a bad fit for some of the low-level interactions with the kernel, is absolutely correct though. We had to do a lot of juggling to get it to work properly. But in the end it was fine. I would say it was the correct choice of language at the time, and it certainly contributed to Docker's early success.
Another explicit goal was to facilitate contribution. We thought Go syntax was "vanilla" enough to attract a broader set of contributors than a more niche, opinionated language like Erlang, Haskell, Clojure etc. I think that also worked out well - Docker was known for attracting many contributors, many of them making their very first open source contributions. We even ran special meetups to help people become contributors, that was fun.
I think that's so true and it was one of the keys to success of the project. I contributed the very first simplistic implementation of referencing linked containers by name. I never would have bothered if the project was written in C or C++ (I'm more than capable, I just don't enjoy torturing myself). Go made it MUCH easier for a fly-by-night contributor such as myself to join the party.
Do you think e.g. students learning docker syntax at a uni is beneficial in 2026 or you think LLM-assisted programming means that Docker will become an abstraction under an abstraction and its setup will be mostly automated by LLMs? I find myself very rarely debugging or handwriting dockerfiles nowadays!
This is a strange question because you could learn Dockerfile syntax in a single night. There are only 18 directives, and the average Dockerfile uses like 6 of them. 90%+ of Dockerfiles is just "run this shell command" and "copy this file into the docker image". The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.
> The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.
And multi stage builds are just a directed graph short hand for having separate intermediate dockerfiles. There’s nothing special about them either.
If I had to pick one thing that is tricky for someone new to docker to understand, it’s how the filesystem state is cached using the previous state, the env, and the raw command text.
Indeed, the one mistake I keep seeing new people make when writing Dockerfiles is writing them in a way that doesn't optimize for caching. For example, they'll copy their code into the container first and then start installing packages and dependencies. It's an easy mistake to make because the end result is still a functional Dockerfile, they're just wasting time and stressing the poor package servers unnecessarily.
I'm gonna be real with you. No one needs to know docker syntax. Most engineers just copy and paste chunks of Dockerfiles around anyways.
Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.
Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.
LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.
edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.
I think the key thing for a university student is to get the big ideas down: what is a container exactly, how does it work, how does it differ from a VM, what assumptions does it make about what you're running (and how do you run something that doesn't fit that pattern), why do images exist in layers and how are layers implemented, what are the performance benefits/tradeoffs, etc. etc.
When I was learning Docker I can't tell you how many people would describe it to me by analogy to a standardized shipping container, and start going off about shipping container dimensions and supersized cargo ships and trucks and railcars... and finish by smiling beatifically, like they had just explained the whole thing in a masterstroke. Let's... uh... not do that. :-D
In a problem set, the student might learn a bit about dockerfiles or docker-compose.yml... but only, I think, so they can try building and running a container of their own!
If you're going to fully delegate something so trivial as Dockerfiles to slopmachines, why go to university at all? You don't need to learn anything if you delegate everything to the slopmachine.
I'm a big fan of docker and mainly just want to say thanks for bringing it about.
I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...
After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.
It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.
I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)
Yes, I still use Docker. Although my way of using it has changed over time, like many others I assume.
My new project, Dagger, although it doesn't directly compete with Docker, does implement its own container runtime and orchestrator (for CI pipelines), and we recently added compatibility with Dockerfiles and Docker Compose files... So increasingly I find myself using Dagger instead of Docker, in the context of my CI pipelines.
On the other hand, I use docker compose on almost every personal project - like a minecraft server for my kids, home automation, AI hacks, etc. These days I just let my AI agent set it up for me, though :)
Yeah I've really been enjoying using AI to write docker/compose config and related ad hoc scripts. It was all pretty understandable anyway, but now you can just set up whatever you can think of in literally minutes, even if you just want a particular configuration as a one off. It's a great era for devops!
> I use docker compose on almost every personal project
Even if you don't need to? Even if there's no benefit?
I was handed a vibe coded project recently, a radio station playout system. For some reason, it's spread across 8 different Docker containers. One container has just Icecast in it, for example.
It's a private system that runs a single station on a dedicated server that we have full control over, but because of all the containers, it's harder to do common tasks like check a log or edit a config file.
OTOH it's a godsend for setting up a local dev machine or something like that, where you need to run different vendor provided packages. but I'm tired of all the mindless over-engineering (which unfortunately the LLMs trained on).
I’d love to hear more about some limitations/tradeoffs of docker that you made at the time that you regret in hindsight. Was there anything that you wish didn’t get baked into the “default norms” for containers, or areas that would benefit from more attention and redesigns?
I’ve been following dagger and it’s been really interesting to see the evolution of the design, and new set of abstractions. There’s also nix which has seen popularity, with their approach of building layered oci containers.
A while back you had this tweet which often gets cited, or at least _i_ personally think about it a lot.
> If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!
Thanks for your innovation and helping developers! If Docker hasn't been your focus in recent years, what have you mainly been working on and what's been your favorite?
Were you around for the big Moby open sourcing? What went down from your perspective? If I recall, it was a fairly dramatic move that was not driven by the interests of Docker corp.
Yes, I was around for that. It was one of my skunkworks projects, as I had lost formal authority over engineering and product by then, and mostly used my "founder aura" to convince small pockets of people within the company to build cool things with me.
However Moby was a poor choice for a skunkworks project, it involved mostly communications and project management, which requires consistency and scale. Overall it was poorly executed because I failed to build enough support internally ("Solomon's at it again, let him do his thing"). Here I am trying to explain it on Twitter after the keynote, using a picture of a hand-drawn diagram, because nobody in marketing or product was assigned to help. https://x.com/solomonstre/status/855918630915133440
I think the goal of Moby was fundamentally correct: we were trying to replicate the Red Hat model, with well-scoped upstream components (linux -> runc, containerd..) integrated into a commercial product (Docker -> RHEL), with a faster-moving community project in between (Fedora -> Moby).
This was a way for us to clarify that Docker was a product, with a community of users, and Moby was an open source project, with a community of contributors. This clarity was very much needed, because different communities were projecting very different expectations on the brand "Docker", and these expectations conflicted with each other.
I feel grateful for having seen from the inside one of those rare "hypergrowth" moments, it's a terrifying and exhausting experience but I would do it all over again without hesitation.
I also feel lucky that people are still using my work, that it endures without me, and that it generates (mostly) positive reactions. There was a time where Docker had a lot of haters, which was hard on me. But with time I've noticed that the negativity gets mostly washed away, and what remains is positive interactions like the ones in this whole thread. So now Docker is this cool experience I share with lots of people around the world, and when we meet we get to talk about this shared interest. I love it.
What made you decide to use a URI as the namespace for images? I like it, but I think it's been very misunderstood which has caused people to dislike it.
I was inspired by Go's use of URI as a namespace for user packages, and short names for the standard library. I thought it was an elegant way to allow everyone to host anywhere, and dodge a lot of complicated namespacing issues.
One design principle that was very important to me, was portability and avoiding fragmentation. I wanted every image name to have the same meaning in every engine, so that when writing a Dockerfile, you could have confidence that it would actually behave the same on every docker installation. In this we disagreed with other platforms (like Red Hat) who sought to "embrace and extend" Docker by changing the meaning of short names. We always fought this to avoid fragmentation of the platform. This did not make sense from Red Hat's perspective, since they were the platform - who cared about fragmenting Docker? That led to some negativity which is probably what you're remembering.
At the same time early Docker users wanted the ability to host their own images without being forced to use our central registry. Using Go's system solved both problems at once.
My only regret is that so many users had to run local docker registries on each node of their cluster, then write `FROM localhost:1234/myimage` in all their Dockerfiles - all to avoid unnecessary re-fetching of remote images. My goal was to make image distribution a native feature of the cluster, so that you wouldn't have to choose between portable Dockerfiles and proper image caching. But Kubernetes ended up dominating the cluster layer, and as far as design goes, we were not invited to that table. So as far as I know, the problem was never solved, at least universally for all Docker users.
Thank you for taking the time to reply. Yes, it was the short names and the way people were trying to run local mirrors that I remember.
I personally never use the short names and have really come to like your solution. I host my own registry and I can glance at a build or compose file and immediately know if I’m depending on any external resources that I shouldn’t be.
I originally disliked it, but it’s been my favorite namespace solution for a long time now.
Thank you for inventing Docker, it truly changed the life of many developers.
I was there when dotCloud announced Docker in 2013 and it was truly pragmatic. I had previously burned my hands on LXC, Juju Charms and Vagrant to containerize the apps, but it was all a house of cards. I remember helping a friend ship his whole Vagrant Box to the interview panel for a job!
It’s incredible to think about how much boilerplate Docker removed right at it's launch. Thank you.
I remember going to a demo of Docker in South Carolina before it hit 1.0. Came back and wrote the “Docker is the Heroku Killer” article on my blog and it was the first thing I wrote that made the front page of HN.
They aren't at all the same thing, but at a high level they both reduce the amount of fragile/tedious setup-specific futzing you need to do to get a few basic services up.
I was in the audience when you gave the 5-minute docker lightning talk at PyCon US Santa Clara in 2013. I remember chatting with a friend/colleague about it (https://nvie.com/about/) and we both agreed it was the most interesting lightning talk that day.
What’s funny, though, is that since it was “just” a lightning talk, it didn’t even make it into the roundup of notes I sent to my team after PyCon. But a mere year later, Docker was how my team (and seemingly everyone else) ran projects locally.
Any interesting memories from that day in 2013? From your perspective, what happened in the year that followed that caused docker to be adopted so widely? Were there other key moments or turning points?
That day is forever engraved in my memory... Off the top of my head:
I signed up for a lightning talk expecting an audience of 10-20 people, for a confidential demo of our unfinished project that we had so far only showed in 1-1 demos. Imagine my surprise when I learned Pycon does lightning talks on the main stage!
The previous speaker didn't show, so I started early. In the recording you can hear me joke that perhaps I should get his time allocation too - the MC was not amused.
I went over time and had to speedtalk through the end.. The MC once again was not amused and practically shoved me off stage :-D
We had a small sponsor booth at the even, where our small marketing team was prepared to pitch our product (DotCloud) but was thoroughly confused when a horde of people showed up after the talk, to ask about "Docker" - which as far as they knew was the founder's side project.
I drove down to Santa Clara on my newly acquired motorcycle - my very first trip on the freeway. I remember telling myself, "today is the day to take all the risks!"
I was there as well, and while I was really impressed; I was also weirdly weirded out that this wasn’t a bigger deal to all the folks in attendance. And so that made me doubt that what I saw was that impressive, as surely other folks in the audience would know better than I would.
I agree with this take, even not riding as much as I'd like these days.
I had a pretty much near death motorcycle accident I happened to mostly walk away from which really made me restructure my perspective on things I'm afraid of.
It also managed to make my then to be wife realize she really couldn't live without me when I sent her a photo of a mangled bike and the text "ouchies", so that accident really changed my life positively and radically.
I was there :-) My business partner at pycon observed that Docker looked like a complicated way to fix easy_install (at the time it looked like to official fix would be to adopt pip). I thought it had some potential :-) but definitely didn't expect it to take over the way it did.
Thanks for sharing this, a really interesting piece of history. It’s always nice to look back at the beginnings of great technology that powers the modern world. Also, thank you, Solomon!
This is wild to see. I remember how exciting this was when I eventually learned about this tech. To see it minimized to a five minute surprise intro like this and then shuffled off stage is amazing. Thanks for sharing the video in this context.
The recording of that lightning talk session is at https://pyvideo.org/pycon-us-2013/friday-evening-lightning-t.... I was there too and was among those who didn't get it immediately. I do recall more and some mentions of that talk during conversations through the remainder of PyCon though.
I like to tell people the story of how I got a pre-release peek and frankly didn't understand what I was looking at. You emailed me in reply to a comment I made around that time[0].
Funny how little has changed except the names, I guess.
Python and Go are both memory-managed languages. That is perhaps the hardest part of programming (outside of getting the code to do what is intended).
Go and Rust are both compiled, but Rust requires the developer to figure out all of the memory management before compilation, which slows down development to a crawl.
Almost no company will rationally choose Rust over Go if they know how the development speed tradeoff looks like.
Hi, question from a sysadmin / network engineer. Why localhost with port forwarding? Why not native mechanism like routing and mdns? I’ve gave Docker and other container managers which do networking the same way a try a few times and I always found the network unnecessarily complicated. Eventually I came across OrbStack which uses the aforementioned mechanisms. Every container gets an IP address in the local bridge and responds to mdsns queries. This plays especially well in “compose” scenario. Network and accessing the containers are finally done the “native” way which is easier to debug.
I definitely enjoy learning other people's opinions on things I didn't realize I had an opinion on until I learned there was something to opine about.
To me the arbitrarily complex docker networking makes perfect sense. Especially in a compose setup. I want my containers to be able to talk to each other, but I don't want them on my local network or even accessible (sometimes) from the local machine. The various internal networks offer isolation of containers from the rest of the network stack. If I have a rails app in a docker compose configuration with PG, Redis, Elasticsearch, etc, in a development environment, most of the time I don't want the service/accessory containers anywhere near anything other than the app's container. And if I want several compose stacks to work together, I just put them on the same named network. I apologize if I misunderstood your point.
> The various internal networks offer isolation of containers from the rest of the network stack. If I have a rails app in a docker compose configuration with PG, Redis, Elasticsearch, etc, in a development environment, most of the time I don't want the service/accessory containers anywhere near anything other than the app's container.
Which can be achieved with normal routed networks with firewall on the “router” just fine, even on IPv4. Each container can have its own IP address, its own hostname and listen on a standard port, all reachable from containers and from the main OS of the workstation.
We acquired a small team of Cambridge-based computer science & systems engineering wizards, called Unikernel. We then worked together to apply systems engineering to improving and expanding Docker in various ways.
The most successful project that we worked on together was "project Piñata" which later became Docker for Mac. It turns out bundling a Linux VM in a native Mac app, and integrating it into the host filesystem, networking stack etc requires lots of magic..
Now, most of the Unikernel team were very active OCaml developers, and created MirageOS among other ocaml projects. While working on Piñata, they simply reached for the tools they were most familiar with.
When acquiring our company, Solomon took us to a remote farm in France for a week for an offsite, fed us fantastic food and wine, and the first prototype of that Docker for Mac VM was working by the end of the week! It took a few months to hack together the shell. There's a video of Solomon and the pinãta launch party on my blog ;-) https://anil.recoil.org/notes/cacm-docker-cover#getting-invo...
There was a whole other slice of work we did for project Piñata too in the form of datakit (https://github.com/moby/datakit) that was well ahead of its time. I was looking at this archived code again just last week while thinking about agentic git workflows!
(Joining the reunion thread) I was just explaining to a new starter yesterday why Docker Desktop used the codename "pinata". Also I reckon some of the more advanced ideas from back then are still relevant today, probably even more now that everything is changing (or can be changed) with agents. Hope all is well!
Prior to the creation of Docker, did you have experience with other containerization technologies like Solaris Zones and FreeBSD Jails?
Did you at some point, either prior to or after making Docker for Linux, consider building something like Docker on top of FreeBSD Jails?
As a user of both Linux and FreeBSD, running Docker containers and Jails on each of them respectively, it would be interesting to see what FreeBSD Jails could be with something like Docker built on top of it while also taking advantage of other features on FreeBSD like fully using ZFS snapshots, ZFS send/recv, ZFS clone, vnet, and so on. Such that spinning up a container would under the hood receive ZFS snapshots from the “Docker registry” equivalent server, and it might clone the snapshot locally before starting up a container instance so that multiple container instances would be using the same locally available ZFS snapshot that was received once from the “Docker registry” equivalent.
Docker is probably the highest-profile software written in Go. Are you happy with that choice? Would you do it in Rust if it was available at the time, or Zug?
Hi Solomon! Great product. How do you feel about the need for a central registry and not a p2p approach for distributing images? I always felt like p2p was the better and more scalable approach. Any reason why you decided on the centralized approach?
I appreciate what docker did in pushing a more accessible UX on top of containerization and getting people to think more about reproducible setups and sandboxing. Its popularity was definitely one of the catalysts in that direction.
I'm curious if when you were originally designing it, what tradeoffs you chose between, for example did you consider a non-daemonized design or was it daemonized from the beginning?
Having had some time now and also seeing how other people have riffed off of the idea and refined it, what would you have done differently? If you have a retrospective at some point already, I'd be interested to read it.
I have some accumulation of personal architectural guidelines that I built up from trial-and-error + reading others' experience and reflecting on how their design could've affected something I made. As a result, I really value those kinds of discussions about tradeoffs.
Back in 2012 we used to give talks about how we ran all our CI builds in LXC containers, that came about because a bunch of us came from the BSD / Solaris world of jails and it just felt natural.
The folks at infracoders / DevOps meetups would think we were a bit "out there", then what do you know - 2013 rolls around and we quickly started to shift to Docker (too early some might say), but not long after shifting from LXC to Docker it seemed many others finally started to come round to containerisation.
Fast forward to 2026 and I've recently done some work with a client who is only just "getting into containers" (a Windows shop).
Thanks for your early work to bring better containerisation to Linux Solomon. It's funny to think we're 13 years from Dockers release. Time flies.
Since we're reminiscing, I was working for Engine Yard when Docker was released. I remember engineering and product leadership huddled around a desk wondering aloud whether we needed to do something about it.
In retrospect, of course, it was earth-shattering for the company, despite a valiant effort. The entire expertise of deploying long lived Linux servers and carefully shepherding them through change management with Chef became a relic. Totally changed the way applications were deployed.
I've been actively using Docker for most of my projects over the past few years and I sincerely thank you for brainstorming and getting it out in the market.
The sole idea of containerising an application, its utils and dependencies all into one container is a real big deal in early 2010s. The product has been so visionary that it has been re-pivoted as secure sandboxes which is most needed right now. Kudos!
The main primitives that underlie Docker were originally intended to facilitate security and isolation. Docker's innovation was combining filesystem-in-a-box with those security primitives, and delivering all those capabilities with highly intelligible UX. That's not meant to minimize; I think those are monumental (and monumentally impactful) achievements. But there was no "re-pivot" to secure sandbox isolation; that's what things like cgroups and namespaces were created for!
Not really a question, but just saying "hi!". You might not remember this, but in 2012 I was about 13 years old, interested in computers, and had no one to really talk to about them. By some miracle my father (who is in no way in tech!) found some connections to YC and organized for us to come out to California to see a bunch of companies. dotCloud was one such company, I still have the t-shirt! I remember chatting with you and you taught me how to use pip, it's a fond memory I look back on often.
These days I work in infrastructure at Discord; lots of fun problems to solve in that space. I'm getting to work on a lot lower level than my job description would imply.
Docker completely changed my life. When I started using it in 2014, I really appreciated how well it solved certain problems I had for web development.
It spawned a near endless stream of contract work and video courses for about 10 years, along with being introduced to the Docker captain's program early on. Met so many great people and had a chance to visit a few countries around the world through Docker.
It's one of those pieces of tech that has followed me around for half my career and will likely continue until the end. It's the best community I've ever been a part of.
As a backend developer, I am hopelessly grateful for docker and containerization in general. The old days with stateful application servers were AWFUL.
In my opinion, Docker is probably the defining computer technology of the 2010s.
I know that containerization on Unix-like systems didn't begin with Docker, but I definitely think Docker was the first accessible interface for it.
I think app containerization is something that has so many benefits and it's the obvious correct approach for its use cases once you wrap your head around it. This was already understood at big tech companies pre-Docker (e.g. Borg, Tupperware, user app sandboxing on iOS and Android), but I think it really took something like docker to make it mainstream.
As far as I'm concerned, you can probably consider yourself like the Ken Thompson or Dennis Ritchie of your era in terms of the impacts of your contribution.
Oh man, the only big struggle I have with docker, is the abandonment of Docker Swarm.
I've struggled for years wanting something that wasn't K3S or K8S. Something good for small cluster management that had a lower level of entry, that I'd feel better handing off to other teams.
Docker Swarm seemed to fight the puzzle, but falls short based on things that needed to be fixed or finalized, that just never happened.
Any reason that whole pathway got abandoned? Other than just "Google uses Kubes so just everyone can use it"?
At some point I emailed Linus to ask for his advice but he never replied. That was a personal initiative though, not on behalf of Docker.
I did speak to Greg several times.
I think over the years there was natural overlap, people who contributed to Docker would also be kernel contributors. Especially when we spun out runc, containerd, which created a more vendor-neutral space for maintainers to focus on the bottom of the userland stack with minimum drama.
To the extent that Red Hat employees overlap with kernel folks, that made it more difficult to collaborate since Red Hat was overtly aggressive towards Docker, and generally made our engineers feel unwelcome on "Red Hat turf". I discovered that they had devolved into a toxic company with a facade of benevolence. This made me sad because Red Hat was an inspiration to me. One of those "don't meet your heroes" situations.
Since 2006 first as part of the Ottawa Linux Symposium and later as part of the Linux Plumbers Conference we had yearly in person meetings.
We had public discussions on the Linux Kernel mailing list and later on the containers list.
It was a collaborative effort from vserver, openvz, IBM, and Linux Networx, and google. I think those were all the original groups, my apologies if I have forgotten someone.
After Docker made a splash I occasionally asked other developers if they had any contacts at Docker. To the best of my knowledge we never received a bug report, or a feature request. No one from Docker ever showed up at any of our meetings.
I know at one point I wanted to let someone know that Docker they were holding the pieces wrong when it came to security.
Eventually I think one of the OpenVZ guys wound up working for Docker for a time. But that was quite late in the timeline.
I was passing through RedHat about the time of the spat you mention. My impression at the time was that Docker never worked well with anyone, and after some frustrations trying to cooperate the RedHat userspace folks decided it was easier to get a real standard without Docker.
At that point I had gotten the unprivileged user namespace support into RedHat's kernel and changed a lot of minds about it's utility.
I don't remember Docker ever picking up unprivileged user namespace support and as a result was a security disaster waiting to happen.
It felt to me like Docker treated the userspace API for containers like most folks treat the POSIX API. Fixed in stone. In fact there was a lot of wiggle room, and opportunity to shape things for anyone who was willing to dive deep into the technical details.
Red Hat was frustrated with us because they were used to calling the shots. They couldn't fathom that we would refuse a pull request from them. Well guess what, every project is entitled to saying no. And Red Hat's contributions were just overall not good, or they didn't bother to consult with the maintainers on our design principles - like independence from the underlying OS distro.
You're reminding me why working with that company was such a pain. Always with the fucking superiority complex and passive aggressive drama.
As for a "security disaster waiting to happen". My view was, and remains, that the only isolation you can trust is the hypervisor. Kernel level isolation is not strong enough, and unprivileged user namespaces never actually mattered in cloud architectures. Your work on that feature was just security theater and ultimately useless, so spare me the lecture.
I do think Docker embraced user namespaces after my time but I couldn't care less.
Not really a question. Just happy that this tool came to life. Mostly because it's so easy to use - to create images and to replicate software from images. I hope it stays around for a long time. Sure it's not easy to keep support and it's lots of complexity under the hood.
Bocker has a drastically smaller inherent software attack surface than mainline Docker, yet its container isolation is practically non-existent, making it way easier to exploit and break out of.
That part about the threading was a huge source of issues back in the early days. Goroutines could hop threads and it's the threads namespace/cgroup configuration that gets inherited when forking..
One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.
I think you're confusing userland threading (also known as green threads) with a regular thread pool implementation (that uses regular OS threads).
Because most web servers use OS threads. Nginx, Apache, litespeed, they all spin up `core count` threads but they can all work single threaded just fine as well.
No, I mean that Tokio, Go, etc., were all built with general-purpose web servers in mind as far as I know, and they decided that lightweight threads were necessary for that purpose because OS threads had too much overhead to be used per-request.
NGINX uses a state machine per task, which as far as I know is kind of what Rust ends up desugaring to, but the explicit state machine is less easy to read.
Tokio's not a lightweight thread/green thread system. It's an event loop, just like Javascript's, optionally with a fixed number of threads running work underneath.
For Go: you're likely right that OS thread overhead was the main reason for Goroutines, but I don't think it was the only one. There is also a ton of ergonomic efficiency to be had via green threads; like async/await, they're kind of a syntactic unlock that allows you to very liberally use goroutines/channels as a means of semantically modelling actors/state/data flow in your program, where doing that with regular OS threads is both dangerous and (given the start/stop overhead of threads on lots of other platforms) feels like kind of using a howitzer for a flyswatter. Golang saying "yeah, create a million of these, we don't care, it's 4 machine instructions per to start one" was pretty liberating for a lot of people.
The goroutine architecture also unlocked some nice performance at the OS level: GC latency improvements became easier, and some IO multiplexing efficiency a la epoll became almost free to implement due to the ability to cheaply move and suspend goroutines.
> Tokio's not a lightweight thread/green thread system
It's not green threads (and maybe not lightweight threads; there are so many names for things that may or may not be considered the same...) in the sense of stackful coroutines, but either way, it's a userspace implementation of multitasking on top of a number of OS threads, which is what was relevant to my question.
Reasonable clarification. My response to your original "never do this" is a is a bit more specific then. There are two broad userspace concurrency models: cooperative and preemptive. Cooperative is a "sometimes the right call, sometimes more trouble than it's worth" situation; I can see what you're getting at there. But you're flat wrong about preemptive.
Cooperative concurrency: it depends. It's sometimes the right move, but has drawbacks. It's usually implemented to enable efficiency or partial parallelism of kernel-space code (IO). There are plenty of platforms that absolutely need it for that purpose: c10k, nodejs's rise to fame, all that. It behaves poorly when mixed with non-cooperative (blocking) code, and is poorly understood by many folks who expect userspace code to be parallelized as well. A common claim is that cooperative concurrency is harder to use than pre-emptive, but that's not true: some common things are harder, but some less common, but more dangerous things like race conditions tend to be much easier. Tokio and other cooperative systems that schedule onto fixed kernel threads can help with the hard parts a bit, but it's a partial mitigation at best.
Preemptive userspace concurrency: firmly disagree with your original point. Green threads are usually combined with userspace-code parallelism (e.g. goroutines scheduled onto multiple threads) and efficient kernelspace interactions (IO multiplexing). In the popular implementations of green threads I know well (Erlang and Go), this is an unalloyed blessing, in that it both removes many of the dangerous edges of threaded programming, facilitates increased efficiency, and lets users semantically model concurrency/parallelism with much less fear of e.g. creating too many threads, torn reads, etc. That's a big benefit: now we can model things like actors or CSP as subprograms written in procedural, synchronous, ordinary-looking code without getting Ph.Ds in operating system internals or having to carefully reason about complex interlocking layers of callbacks and futures to minimize risk of blocking or deadlock. In many ways, this brings the good parts of single-responsibility/what folks call "the UNIX philosophy" into a single process with minimal pain.
Rare cases where the userspace-concurrent runtime fights with your ability to interact directly with the OS, as with Docker's fork/thread/process-global-state challenges, do emerge. But I don't think those are primarily because of userspace concurrency; rather, they're because it's hard to do low-level things in a high-level, managed runtime. You can fork in the JVM, too, and you'll have problems. Different ones, because the runtime's different (e.g. endless RSS burn at idle due to the child GC moving parent objects), but problems all the same. Same for Python: forking in the presence of regular OS threads and/or a running event loop is equivalent to arming a minefield; doable but troublesome, just like it was for the Docker folks with Go.
To be fair, of the same pains as the cooperative model appear with pre-emptive, too, when userspace concurrency systems are retrofitted onto runtimes that do a lot of other kinds of concurrency as well--but that's because of the hybrid/retrofit nature of the product, and not because of green threading as a concept. Java's rocky adoption of virtual threads is an example of that.
This advice comes from threads on Linux being second-class citizens. As TFA itself states, there are some syscalls that really should only be made in a single-threaded process which is frankly insane... but that's the legacy of starting with the processes-only model and patching it years later to have threads as well.
It worked out pretty well for Go. Nothing else besides Erlang cracked the parallelism UX/runtime-performance code as well as Go did, and it didn't have the drawbacks of Erlang.
> rule broken by almost every language in common use
I don't think so. Go/Erlang/Java do break it (Java very recently). C/C++/Rust/C#/JavaScript/Ruby/Python/Perl/TCL don't break it. PHP ... sort of breaks it, (IMO they just hide cooperative nonparallel concurrency in a nice API, but I'm not well-informed on php)?
And not a single mention of the actual weird decision, a client / server model which complicates permission management by at least two orders of magnitude.
I’d argue in 2011, the client / server model is what made docker a success. It allowed for a seamless, and mostly identical, experience for docker on Mac, Windows and Linux for one. It met developers where they were, daily-ing a Mac or a Windows and deploying on Linux headless servers. It had benefit on its native Linux system too that took all the alternative solutions another 8 or 9 years for the entire Linux ecosystem to develop enough solutions around containers to make a daemon unnecessary while still offering docker DX.
Oh yes docker-machine was quite good. You can create a machine on a Mac by using VMware, VirtualBox, or using the cloud such as EC2. Then your docker CLI can work with all of these machines seamlessly without knowing any details about the machine.
That's actually a design decision made to work around the limitation of having to run as root. Since the container had to be started as a privileged user, if you want non-privileged users to be able to start contianers you either need a setuid binary or a client/server interface like docker has. There are pros and cons to each of those options.
Now it's possible to create containers without elevated privileges (assuming user namespaces are allowed), but that wasn't possible until long after docker had established the client/server model.
I spent my whole day dealing with Docker permissions today. Whats interesting and dangerous is how docker just overrides ufw rules like they do dont exist.
This is the primary selling point of podman to me actually: If you run any non-standard firewall rules (in my case I use virt-manager to run a handful of VMs using a bridge so each VM can get an IP directly on my router's subnet), some stuff may suddenly break because Docker takes a wrecking ball to the firewall rules.
Most of them don't seem that weird to me. Except the use of go. It definitely seems strange to pick a language that can't natively call clone, one of the most important syscalls for using namespaces and implementing containers.
>>Images are just folders that Docker puts somewhere and then calls chroot (or pivot_root) to make that folder your container process’s new root directory.
And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?
The high level abstraction is more important than the low level implementation unless the low level implementation is leaking through, or there is a significant advantage conveyed by understanding the low level implementation.
I know a lot about how http works, and it comes in handy, but I can understand how juniors get away with not knowing it as fully as I do.
I prefer Podman most of the time, but Docker is/was amazing and a great DX put on top of the so called "5 different linux kernel features in trenchcoat". That's the selling point, it's super lightweight and takes full advantage of the OS features that were already there (or being developed) but were awkward to use and required tons of specialized knowledge.
The first two are just kind of the nature of containers on linux, and applies to any container technology.
Podman does allow you to run it as a non-root user (with some limitations), that was in fact one of the biggest motivations for making podman. This wasn't possible at all in the early days of docker, it wasn't until linux introduced user namespaces, where the root user inside the namespace mapped to a non-root user id outside the namespace that non-root conatiners were possible.
And podman, unfortunately IMO, is also written in go.
> podman, unfortunately IMO, is also written in go.
Podman, by default, uses crun as the container runtime, which is written completely in C. Podman is basically just a frontend for managing running containers.
I've used crun without root for a lot of practical use cases, if you are only doing limited use of the network and are careful with volume mounts it works pretty well.
That cannot be true. You need to tell the kernel about the namespaces, cgroups, and the new root. Otherwise it's not a container, and there is no reason or mechanism for a child proc to respect it. And if you let any old unprivileged process do this, you open up security problems.
I've done this for practical use cases, like running a container inside another container that didn't have root privileges. It works fine for that use case. I'm not worried about restricting access of the process because it's already within a larger sandbox, I'm usually just trying to execute the command within the child container's filesystem (e.g. some vendor/partner software they ship as a container image).
I don't know why you're so shocked by this? This is all very basic use of the OCI spec.
Linux by default allows unprivileged processes to create user namespaces, without needing root [0] ("Since Linux 3.8, unprivileged processes can create user namespaces, and the other types of namespaces can be created with just the CAP_SYS_ADMIN capability in the caller's user namespace."), and then create mounts, adjust the filesystem root, etc. within those unprivileged namespaces.
It's only in recent years that it's become trendy for distros to disable unprivileged user namespaces in the name of reducing kernel attack surface. So while a privileged daemon is needed for those distros, there is no such inherent restirction on the kernel level.
The only issue I’ve run into with Podman is that, like docker, I can’t have the container’s process be part of my shell’s foreground process group. For docker the daemon (either rootful or rootless) makes that inevitable, but podman has better rootless support, so I’d been hoping.
Unfortunately, there’s an unconditional setsid during podman’s container setup, which as far as I could tell exists so that the container can be detachable.
(This was all so that I could have the same job management with Ctrl-Z, SIGSTOP, SIGTSTP, fg, and bg, regardless of whether the process I’m running starts up a container. The only way I’ve found to have that is by handling chroot/unshare myself.)
UNIX job management is fundamentally broken already in that it simply can't be properly nested, even with PID namespaces. If the "container’s process [is the] part of my shell’s foreground process group", then it can not, literally can not, job manage its children.
Clearly, the solution is to extend the `kill` syscall to accept the PID as a Gaussian integer, with both real and imaginary components. That way, we can preserve the current behavior of `kill(+123, sig)` and `kill(-123, sig)` as sending a signal to a single process or to a process group, respectively, and will then allow `kill(+123*i, sig)` as sending a signal to all processes that are either within a process group have a direct or indirect parent within that process group. I don't know what we'd use the negative imaginary numbers for, but I'm sure we could think of something.
If not for the uncatchable SIGKILL/SIGSTOP, I'd say that any process that calls `setsid` is responsible for ensuring signal propagation across the pgid. But since those are uncatchable, I tend to shrug and figure that I'll avoid setsid wherever possible.
There was a proposal at one point that would have had `unshare(NEW_CGROUP)` place the process into a child cgroup, rather than merely changing the name of the current cgroup. This would have meant that cgroups could actually be used for nested process management. Unfortunately, there's no guarantee that a process can ever make nested cgroups it wouldn't be sound for all supported resource constraints, so that proposal never went in. But that would have been nice to actually have nestable job management.
> I'd say that any process that calls `setsid` is responsible for ensuring signal propagation across the pgid.
How?! After a process calls setsid(2), it becomes a leader of a new process group so it stops receiving signals from its former process group!
> But that would have been nice to actually have nestable job management.
Yes, it would be. Just imagine: a simple, straightforward way to group a bunch of processes (and their potential children, inescapably), signal them all at once, wait for them all at once, etc. Instead, we at best ended up with docker compose and k8s. And they kinda do solve that problem, no question about that but it's just so incredibly heavyweight...
> How?! After a process calls setsid(2), it becomes a leader of a new process group so it stops receiving signals from its former process group!
By limiting its usage to the period between fork and exec, so that the process calling setsid knows that the parent process is propagating signals to it.
So, my phrasing was inaccurate to the point of being entirely incorrect. I should have said that any process that calls `setsid` is responsible for ensuring that signals are being propagated. (And saved! Definitely didn't make a mistake on the internet and then play it like I was right all along.)
> any process that calls `setsid` is responsible for ensuring that signals are being propagated.
Well, again, the process that calls setsid(2) ceases to receive the signals from its previous process group, so there is nothing for it to propagate. The only way this propagation can work is with cooperation from the parent process of the setsid-calling process: it will need to manually pass the group-wide signals to its setsid-calling child — which it can't reliably do for two obvious reasons.
> By limiting its usage to the period between fork and exec, so that the process calling setsid knows that the parent process is propagating signals to it.
Wait. Are you implying a scheme where the process A, that wants to do setsid(2), forks itself, and it's this child process B that does setsid(2) and all future actual work, while process A merely lives in a "while(1) { sig = sigwait(); signal(pidB, sig); }" loop forever? This almost works but doesn't survive SIGSTOP/SIGCONT/etc. and, of course, you can't propagate SIGKILL. And SIGKILL-propagation is arguably one of the main attraction of the process groups — you send SIGKILL to the process group (or a container, for that matter), and all of it dies with no further work on your part.
> And if you know where to look, you can even find that process in /proc.
This is very important for debugging! Not just that process, but the processes running in that Docker can be found in /proc.
Why this is useful is that you can run gdb outside of that docker and point it to a process in the docker, including setting the file system root to the docker one, so that it gdb reads the correct objects from the correct paths, to get the right symbols and addresses.
For as "terrible of a language" Go is alleged to be for a container runtime, its not like crun or youki are exactly faster in many meaningful ways unless you count churning containers running true and exiting.
And implying the system kernel still has to run the container kinda ignores kata containers and gvisor runtimes.
It's about using go to develop container runtime being terrible and not that "if it was C it would be faster". OS runs the container, docker just does the setup and some managmenent.
Docker hasn't required root for 7 years, I wish people would stop repeating that. It uses a root daemon by default because it makes things easier. Also, Containers are only as fake as a "Desktop" in an OS, which isn't real either. It's a user, network & process namespace with a control bus and a bunch of other weird things. A Desktop is a Container with a GUI.
Other weird facts about OSes: a filesystem is a database, the Linux kernel is a monolithic program, "hidden" dot-files were a Unix bug that got carried forward, deleting a file doesn't remove the file from your machine, virtual memory is a lie, your computer can't run multiple programs at once (it's just running one at a time real fast), in 12 years most systems will crash due to one number getting too big, you technically never need to reboot because you can launch a different version of Linux while already running Linux, you can freeze a program to a file and then later resume running that same program in the same state (but nobody uses it), files aren't actually stored in directories, there's a maximum number of files (and size of file) on most filesystems (which you can use as a hacker to lock up a machine, despite file quotas existing and nobody using them), you can lock up most systems by forking too many times / using too much RAM / accessing disk too much, there's a default number of open files and/or TCP ports which is artificially low and you can use to again lock up a machine, pipes are just two file descriptors connected to one memory buffer, an IP address is just one long number split up into bytes, etc.
The rootless Docker experience used to be pretty bad and it was less friction to use something that was rootless by default (like Podman). Has this changed?
it changed and its better now.
rootless dockerd wants newuidmap/newgidmap + subuid range before it start, and cgroups only work with cgroup v2 + systemd.
Johnathan Blow's talk on the bronze age collapse pushed me to simplify my production stack. I wanted 'just processes', so I stopped using docker there. I'm happy with the result.
> Go is kind of a terrible language for a container runtime
I love Go and use it for everything I can but will absolutely concede this point. It's clearly possible to write the most successful container runtimes on earth in Go, but you absolutely have to fight the language/runtime sometimes. Since containers are based on processes, handing off a container spec to a minimal rust runtime seems like what we should all be doing by now. crun has just proven good enough that AFAIK no one has RIIR.
This is a better way to understand it: we can think of a container as an isolated process. When deploying containers, having a clear understanding of the boundaries between containers and the host machine in terms of permissions, resources, and data is extremely helpful for troubleshooting issues.
Today, I would definitely choose rust over go. And I lament that so much container software uses go rather than rust. But back when docker started rust was a lot less mature and probably wouldn't have been a good choice.
Performance is basically irrelevant because you're not really interacting with the running application. You're running as root, so security-critical. You're talking to the internet for things like fetching images. You need rather complicated business logic to deal with all the possible options, protocols, and file formats.
Sure, you could do it in C, but dealing with dependencies is a major PITA and you're permanently one tiny mistake away from getting pwned, while not needing any of C's strengths. Go is safe, easy to write, and ships as a single batteries-included binary: what's not to like?
What's not to like? For example the fact that Go's threads and OS threads are two different things. For example the fact that I can't just use fork in Go. And many other things.
Go is good for writing web servers. For system software? Not so much. 90% of Unix is written with C. It's not that bad.
That isn't really relevant for the tool starting the container. The biggest problem with go here is that it can't natively perform the operations necessary to set up a container, due to the nature of the runtime, so it has to resort to complicated workarounds like a shim in c, and starting spawning from /proc/self/exe instead of forking/cloning.
I started hating Docker after seeing all the cargo culting on Reddit. Whenever someone asks about deploying an app on Linux, there's always at least one comment saying, Just use Docker. We finally ended up building our own deployment automation tool for our entire stack that doesn't use Docker at all.
That comment invariably comes up because for most people it’s the objectively correct and most well supported option. If you’re asking redditors how to deploy something, you’re precisely the person docker is for.
Even when I'm not using Docker to deploy something, I'm _probably_ using oci images at the least, but I'm generally running my local dev stack with Docker compose. Not always, but I use it in a lot of projects.
What made Docker unique IMO is not the container technology itself. I used chroot 20 years ago and it worked just fine. Yes, no network namespaces; no process namespaces, but I don't even think these are that important.
What made Docker unique were there things:
1. Dockerfile concept. Building images layer by layer. Caching layers. Everything in a single file. It made possible to iterate on image quickly. It made possible to implement CI builds efficiently.
2. Docker network registry concept. Putting universal images onto a server and share them with the world.
3. Momentum. A lot of software that was traditionally complicated to deploy, suddenly appeared in docker servers. I remember how fascinated I was, when I could deploy wordpress website with tiny docker-compose. And how contained it was. I could just delete every trace of it from my host. I always felt uneasy installing various things onto my host and docker just solved that uneasiness.
Interesting, I would probably say more-or-less the exact opposite.
The main draw for me was that it made containers absolutely trivial. Sure, you could spend ages figuring out how to DIY some isolation, but in practice an awful lot of people stopped at having Apache fork its workers with a non-root uid. You want isolation and a repeatable setup? Spin up another VM and have Ansible set it up! Docker provided a batteries-included lightweight way to isolate and manage multiple applications at once.
Dockerfile? I would've been happy with a Bash script, or handing it a tarball. Caching and fast iteration? Compute is basically free. A single file? Well, there's already a whole bunch of app source code files sitting next to it, so what's a few more in a `setup/` directory? The registry? A shared base could've also been achieved with, say, a `deb` file providing the right dependencies, files, and post-install scripts.
I do agree on the momentum, though. Having every piece of software reduced from a multi-page setup guide to a single command and not risk screwing over your entire machine is absolutely magical.
And for most people, that's a bad idea. Most people don't have to know what RPATH, ldd, multiversion/simlink library discovery, etc are. And even if they did, Dockerfiles give you an easy way to do something you couldn't easily do before: multi-stage builds and a highly robust way to say "drop the parts of my transitive hull that were only relevant at build time". Dockerfile semantics kind of descend naturally from needing the behavior of stages, parallelization, cache keying for image layers, semantic layering (FROM), and making simple use-cases relatively at-a-glance/contained.
I never said you can’t do that, I’m merely pointing out that GP’s example isn’t an example of doing that.
My point is that the Dockerfile is not a particularly good build system. It doesn’t solve any of the hard parts (how to build the thing, what to build when) and only solves problems it created (how to avoid re-running something on every invocation) and does it poorly (it doesn’t track dependencies, just determines cached-not-cached on every line.) It leads to awful hacks like putting every build step on an enormous chain of `cmd && \` lines to make sure you don’t produce extraneous layers.
A reasonable build system tracks dependencies for what needs to be built and has a target it’s trying to get to. Docker just says “do your build imperatively and we’ll figure out caching” and it’s extremely easy to shoot yourself in the foot leading to overly large images.
An example: there’s no way to say “build container: do this build and put the product here, runtime container: take the product and store it here” without creating a temporary layer in the build container that is not used anywhere else. So if your artifact is 1GB, you will always require 2GB to do a build: one for the runtime container (where you want it) and one for the build container (where you only needed it temporarily.) You just have to do “docker builder prune” every now and then when your disk inevitably gets full, there’s no way to simply not produce the duplicate in the first place.
(My workaround is to keep the artifact in a type=cache mount, store its SHA sum in the build container, then in the runtime container, grab the artifact from the type=cache mount and compare the SHA sum to make sure it didn’t change. Cache mounts can be GC’d at any time so it’s not even guaranteed to work.)
When I used docker at scale back in the day, the thing that worked best was always: bind-mount an artifacts dir into the build container, run the build. Then package up the artifacts dir. Then at runtime, fetch the artifacts, bind mount them into the runtime container. Then you can use a real artifact store on a plain HTTP server instead of messing with docker layers (no more worrying about pushing 100 1KB layers because you didn’t put enough &&’s in your dockerfile) and it was so, so much faster than any docker registry was in practice.
Dockerfiles are great for simple stuff but, if I already have a great build system that produces single binaries, it’s so much more pleasant to just use that.
The magic of the Dockerfile is that it puts the onus of spelling out the environment dependencies on the maintainer, not the user. This was always the case but the difference is now its an easily verifiable, non-binary file.
Layering is as much an abstraction paradigm as a performance one. Its not trivial to verify every loose bootstrap script or .deb that would be needed to build a sandbox... But in a Dockerfile you can see its just some official alpine image base, and a few added files and an entrypoint.
The momentum, I think, matters a lot more than the other two. Makefiles already allowed you to build layered things with caching. Artifact repositories have also long been a thing.
But the argument I used for advocating Docker in environments where people were not fans of it was that the Linux container had become the de facto standard unit of deployment. There is so much tooling written under that assumption, and if you want to use any of that tooling, you need to package your deployments as Linux containers, even if you want nothing to do with Docker otherwise. That's why it had so much momentum.
Then a fourth point I think you're missing is that the concepts used by Docker are relatively simple. Before I understood what Linux containers were, I used to think things like, "Why would I care about Docker when I can isolate my processes with Linux users, cgroups, namespaces, etc." What I didn't know at that time is that "Linux users, cgroups, namespaces, etc." is exactly the thing Docker was giving a friendly UI to.
I agree, that #1 and #2 are what made docker "unique" technologically, but I don't believe Docker they were a docker innovation.
Basically both described in https://www.usenix.org/legacy/event/lisa11/tech/full_papers/... (in detail) and https://www.usenix.org/conference/usenix-atc-10/apiary-easy-... (in a "sexy" use case to try and motivate it) a few years prior. (and I get annoyed when people say "so you're saying you invented containers?" because they equate docker with containers. And I'm like no, containers existed for a long time going back to logical partition on mainframes in the late 70s/early 80s. All I claimed was a novel way of building and distributing file systems for them that made them much more useful and valuable and in apiary a novel desktop setup that made use of it to improve desktop security)
Even the original docker implementation of using a union fs (docker using aufs) was the exact same as described.
Even if docker came up with it independently (though conferences like LISA and Usenix ATC are where the papers were presented and were where companies like dotcloud/docker would advertise their wares (and in fact docker inc was an exhibitor there, so it wasn't hidden), the differences between the uniqueness of docker and what is described in the paper are minor (in my perhaps somewhat biased opinion of course), and i was clearly first.
Honestly it took me so long to pick up containerisation because of how it's marketed. "Orchestrate a zillion fake computers, ultralight VMs, livestock not pets" big whoop.
"It's an idempotent language for OS level dependencies so you can be sure exactly what is running on your server"
This only scratches the tip of the iceberg. People's minds are blown when they realize Linux has no concept of a container.
It's not a vm: It's all namespaces, baby! Unshare one or more of your mnt, pid, net, ipc, uts, user, cgroup, or time namespaces and you have a "container".
Docker question. Who prefers docker for local dev vs. run the thing directly. I usually prefer run the thing for debugging and hot reloading. Sure: build the containers and test them in CI. Dependencies like PG sure run in containers as they change less often and you save the install.
some people do all of their software development in a container. It really only works well if you are comfortable using TUIs instead of desktop apps though.
Run the thing directly. Doing everything with Docker is what leads people to build bloated images with dev dependencies in them like test runners, dev servers etc. Too many people think containers are some magical thing but you can usually get exactly the same isolation using more specific dev tools. For example, in Python we've had virtualenvs forever and now tools like uv make it easier than ever to do the right thing. A docker image is just an artifact that you build for deployment to container-based platforms like Kubernetes.
337 comments
[ 0.17 ms ] story [ 35.8 ms ] threadI'm happy to answer any questions you may have.
You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)
[1] https://wiki.archlinux.org/title/Systemd-nspawn
Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.
The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".
As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
Developers are a tough market to please, especially when your business revolves around open source software.
Unless you have an option to do private cloud hosting (like all these databases and other platforms do), it’s incredibly difficult to monetize.
In the case of docker, I think it’s in the nature of the product that it’s difficult to monetize. Best you can do is enterprise support contracts, but those aren’t good enough to justify all that VC funding.
Podman has never been commercialized.
Docker's difficulties in commercializing were self-inflicted. Enterprise and open source teams were culturally siloed and as a result the enterprise products were not "native". There was a lack of focus and a "checkbox" approach to building product. There was no singular "dictator" who could enforce a coherent product design and vision, and it showed. etc.
I always assumed it was the same internal tech, now productized. Then of course, it became THE tech, overtaking vagrant and anything else for local development and deployment. The latter was the true money shot. Server runs exactly what you test with locally.
That's because there was never a proper sysadmin on your team to slap some sense in your decisions.
You've got too opinionated[0], too tied on yourself[1] monster which required too much resources what you can handle alone.
And your market is not the developers but the companies where whose developers have a daytime jobs: https://news.ycombinator.com/item?id=34377674
BTW just for giggles opened hub.docker.com with Network Tools opened:
368 requests, 9.62MB, Finish: 4.50s.
For a thing which should be a static cached copy with everything bundled. Sure, it's a progress from the previous iterations when it could hang for 30 seconds before actually loading... but this is still ridiculous.
[0] https://news.ycombinator.com/item?id=46739153
[1] Eg: the syntax to point the image to a specific registry is something... bad. So bad I just abandoned running my own registry, it wasn't worth it.
PS there is still (AFAIK) no support for a transaction-like image building in Compose. So instead of something like this:
people need to write these one-liner abominations: EDIT: to add on [1]:Oh, well. I guess you ride to your work on wheels, right? Because while you sit in the car/bus/train it's the wheels what actually rides on the road/track, right?
And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
You see, if they just called it as it should, ig:
then they would be both named correctly and be the technically correct - and 99% of Docker users couldn't tell a difference - because they are not net or sys administrators.But the last 1% is the people who actually makes the workspaces for these developers where they actually do run their fancy-shmancy containers. And for that 1% it surely pays naming this as they are.
And there is also that dumb decision to silently rewrite 127.0.0.1 to Google DNS without a question. Especially when 98% of Docker users are incapable to diagnose it by themselves.
So it's not only the naming of a network functions.
But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
> RUN command has had heredoc support for 5 years now
So ten years late? Adoption is off the limits, as you can see - that snippet is from the official Docker image.
And if I need to run multiple commands and not just one RUN statement? Like the people are asking for years? And you so knowledgeable person of course do know the reason for that, right?
Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes learning and communication harder by introducing ambiguity and indirection. "bridge" now means multiple things and not only is it needlessly obfuscated, it's wrong (explained later). I _hate_ projects that do this.
> Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
Rhetorical question, maybe it'll help you find a source of misunderstanding: Why do you expect it to join the broadcast domains of your containers and your LAN? And which LAN, exactly?
> No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN. Therefore you incorrectly expect and demand that term "bridge" accurately describe the logical connection between your container and your LAN.
Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
A Docker 'bridge' network is not necessarily routed to LAN either so your suggestion of calling them "hostnatnetwork" is wrong. Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
Disabling firewall rules: https://docs.docker.com/engine/network/packet-filtering-fire...
> But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
All of this networking terminology is consistent and applied correctly but for some reason it's not clicking with your previous knowledge and you're trying to fit a square peg through a round hole. Take a step back, investigate.
> So ten years late?
Don't shift the goalposts. You claimed it still doesn't exist.
The docker mental model isn’t crazy different. Design choices made it unnecessarily less compatible with existing mental models. This probably contributed to the feeling of it being revolutionary, but impeded its adoption with paying customers.
Sorry, but you do insist on naming 'docker' bridge a bridge because it's underlying tech is a bridge. Make up your mind.
> A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
And noone absolutely ever cared about what it's a bridge down there. It could be using gronklew and maybe 20 people on the planet would care to learn what it is.
You are quite bent on it, but nobody cares how the containers talk between each other.
> Why do you expect it to join the broadcast domains of your containers and your LAN?
I expect a setting named 'bridge' in some $thing to actually implement a bridge.
Just like you pressing an AC button in your car expect a very specific thing to turn on/off and not 'Artifically flavoured windshield wipers served in a Container'.
Why do I want it is on me. Maybe I need it. Maybe I control my network and freely give out addresses to anything. Maybe I don't run on some cheap VPS somewere with only 1 IP I can't choose nor change. Bazillion of options. None matters because this is not about 'why do you want it'.
> And which LAN, exactly?
The path of a least automagik.
Only one ethernet interface? Then connect to that ethernet interface. Bitch in the logs what automagik was used and this can lead to the problems.
Many interfaces? Just shutdown (least pain for newbies) or start without attaching (harder to troubleshoot for newbies). Bitch in the logs in either case.
> Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN
'Your misunderstanding seems to stem what there are some people assume what they are driving in like c-a-a-r. When even 5y.o. knows it's the wheels what drives you around.'
> Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
Yes, exactly. Don't call it 'bridge' ~20 years ago and we don't have this talk here.
> Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
It's not even a property of a bridge network. For someone claming knowledge superiority this is a quite lame statement.
> links
Really.
> previous experience in an adjacent area where
Ah, yes, irrelevant 'adjacent area' of computer networks. I'm out of words.[0]
> All of this networking terminology is consistent and applied correctly
No. Go drive your wheels home.
> Don't shift the goalposts. You claimed it still doesn't exist.
I'm just mentioned RUN as an example. You decided what multiline is the goalpost.
And yes, if even the official Docker images doesn't use RUN heredocs then... there is something wrong with them. Like being implemented too late.
[0] but perfectly explains your previous 'flawed assumption'
pat pat
First you incorrectly claimed that multi-line RUN commands aren't supported still require that ugly legacy syntax. When that was proven wrong, instead of saying "good, I wasn't aware", you went on a tangent about its adoption and wanted to interrogate my knowledge -- Yes, obviously I know that running all commands in one RUN block is beneficial because it reduces layer count and image size.
Your revolutionary idea of "transactions" is already implemented through multi-stage builds. Except those are even better than transactions because individual steps are still cached, independent components can be built in parallel and they don't invalidate each other's cache. Finally the artifacts are copied into the final image, leaving behind unnecessary layers.
dns_snek: and macvlan is not called a bridge because it uses a Linux bridge you stupid
dns_snek: I'm a good boy and always right!
^ that's you
If you decide to edit your tone, I'm happy to discuss more here re: filesystem transactionality tradeoffs, since that's something I've struggled with in both directions WRT containers.
I think you are the one who built an alternate layering system? On mobile, don't like to search rn.
If yes the basics are in the first comment- just not limited to the one RUN statement ofc. And if I do remember how the layers are made then making it transaction-like would be a great dedup saver eg for adding a static rarely changed artifacts - there could be essentially a ~one layer for how many you want images.
Implementation wouldn't be so difficult - you just stop committing until TRAN END; but the main is too provide a mental model for a not very experiencied dev how this supports his work and makes it more efficient on the storage and execution.
bullshit, you've been cunty right out of the gate since your first reply. I'm just surprised anyone else in the thread had as much patience as they've shown here.
you seem exactly like the type of person who is incapable of realizing they're a complete asshole and wonders why people choose to avoid them.
Now, considering you don't even know me: go in the bathroom, look atvthe mirror and repeat your comment.
Come back when the enlightment hit you.
lol What on god's green earth does this have to do with commercializing Docker? Do you really think better lighthouse scores on hub.docker.com was what was holding them back?
> what was holding them back?
Did you miss 'up to 30 secs' in the previous comment? It's not about lighthouse scores. It's the whole feeling of a hodge-podged project which is run by people who are - maybe - passionate but ultimately lacking in the basics.
The ultimate buyers of their product was the system operators and the system owners - not the developers per se. They failed hard everything that space. Having a dog slow the main content site didn't added anything positive.
Right, which is why bringing upva meaningless stat like how many assets their web site downloads feels like satire.
yes, and no. Probably both. Docker would not become such a default standard if it wasnt for all the open sourcing.
None of these things can be transposed today, so I guess I can't answer...
For what it's worth, my current project (dagger) is written in Go.
Another explicit goal was to facilitate contribution. We thought Go syntax was "vanilla" enough to attract a broader set of contributors than a more niche, opinionated language like Erlang, Haskell, Clojure etc. I think that also worked out well - Docker was known for attracting many contributors, many of them making their very first open source contributions. We even ran special meetups to help people become contributors, that was fun.
Thank you.
Actually there are some longer unedited versions I might release.
https://docs.docker.com/reference/dockerfile/
And multi stage builds are just a directed graph short hand for having separate intermediate dockerfiles. There’s nothing special about them either.
If I had to pick one thing that is tricky for someone new to docker to understand, it’s how the filesystem state is cached using the previous state, the env, and the raw command text.
Docker is much more than memorising a handful of Dockerfile directives.
Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.
Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.
LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.
edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.
When I was learning Docker I can't tell you how many people would describe it to me by analogy to a standardized shipping container, and start going off about shipping container dimensions and supersized cargo ships and trucks and railcars... and finish by smiling beatifically, like they had just explained the whole thing in a masterstroke. Let's... uh... not do that. :-D
In a problem set, the student might learn a bit about dockerfiles or docker-compose.yml... but only, I think, so they can try building and running a container of their own!
I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...
After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.
It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.
I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)
Yes, I still use Docker. Although my way of using it has changed over time, like many others I assume.
My new project, Dagger, although it doesn't directly compete with Docker, does implement its own container runtime and orchestrator (for CI pipelines), and we recently added compatibility with Dockerfiles and Docker Compose files... So increasingly I find myself using Dagger instead of Docker, in the context of my CI pipelines.
On the other hand, I use docker compose on almost every personal project - like a minecraft server for my kids, home automation, AI hacks, etc. These days I just let my AI agent set it up for me, though :)
Yeah I've really been enjoying using AI to write docker/compose config and related ad hoc scripts. It was all pretty understandable anyway, but now you can just set up whatever you can think of in literally minutes, even if you just want a particular configuration as a one off. It's a great era for devops!
I take it vito talked you into adopting Garden? </injoke>
“You’re overpaying for CI” (https://www.youtube.com/watch?v=tsH3eTo2GLI) from KubeCon 2024 seems especially pertinent these days with AI.
Was one of my favorite talks that year and really made Dagger click for me.
Hope to see you this year in SLC!
Even if you don't need to? Even if there's no benefit?
I was handed a vibe coded project recently, a radio station playout system. For some reason, it's spread across 8 different Docker containers. One container has just Icecast in it, for example.
It's a private system that runs a single station on a dedicated server that we have full control over, but because of all the containers, it's harder to do common tasks like check a log or edit a config file.
OTOH it's a godsend for setting up a local dev machine or something like that, where you need to run different vendor provided packages. but I'm tired of all the mindless over-engineering (which unfortunately the LLMs trained on).
For one, did you get rich beyond all comprehension and did you start a band with your wealth?
Second, do you still program ?
I was lucky enough to get in this industry when Docker was new. Sure beats trying to figure out why X happens on machine A and doesn’t on machine B.
I’ve been following dagger and it’s been really interesting to see the evolution of the design, and new set of abstractions. There’s also nix which has seen popularity, with their approach of building layered oci containers.
> If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!
How are you feeling about WASI today?
However Moby was a poor choice for a skunkworks project, it involved mostly communications and project management, which requires consistency and scale. Overall it was poorly executed because I failed to build enough support internally ("Solomon's at it again, let him do his thing"). Here I am trying to explain it on Twitter after the keynote, using a picture of a hand-drawn diagram, because nobody in marketing or product was assigned to help. https://x.com/solomonstre/status/855918630915133440
I think the goal of Moby was fundamentally correct: we were trying to replicate the Red Hat model, with well-scoped upstream components (linux -> runc, containerd..) integrated into a commercial product (Docker -> RHEL), with a faster-moving community project in between (Fedora -> Moby).
This was a way for us to clarify that Docker was a product, with a community of users, and Moby was an open source project, with a community of contributors. This clarity was very much needed, because different communities were projecting very different expectations on the brand "Docker", and these expectations conflicted with each other.
(And as a result that, with ansible, continues to be my preferred way to deploy while the rest of the world passed me by…)
I also feel lucky that people are still using my work, that it endures without me, and that it generates (mostly) positive reactions. There was a time where Docker had a lot of haters, which was hard on me. But with time I've noticed that the negativity gets mostly washed away, and what remains is positive interactions like the ones in this whole thread. So now Docker is this cool experience I share with lots of people around the world, and when we meet we get to talk about this shared interest. I love it.
People are underestimating now big of a tech shift it was back in the day.
What was your imagined use case when you first came up with it?
Did you think Docker as it is used in the industry right now is where you imagined it would be used?
Any use cases for it that the industry missed but you think would have made things even better?
One design principle that was very important to me, was portability and avoiding fragmentation. I wanted every image name to have the same meaning in every engine, so that when writing a Dockerfile, you could have confidence that it would actually behave the same on every docker installation. In this we disagreed with other platforms (like Red Hat) who sought to "embrace and extend" Docker by changing the meaning of short names. We always fought this to avoid fragmentation of the platform. This did not make sense from Red Hat's perspective, since they were the platform - who cared about fragmenting Docker? That led to some negativity which is probably what you're remembering.
At the same time early Docker users wanted the ability to host their own images without being forced to use our central registry. Using Go's system solved both problems at once.
I personally never use the short names and have really come to like your solution. I host my own registry and I can glance at a build or compose file and immediately know if I’m depending on any external resources that I shouldn’t be.
I originally disliked it, but it’s been my favorite namespace solution for a long time now.
I was there when dotCloud announced Docker in 2013 and it was truly pragmatic. I had previously burned my hands on LXC, Juju Charms and Vagrant to containerize the apps, but it was all a house of cards. I remember helping a friend ship his whole Vagrant Box to the interview panel for a job!
It’s incredible to think about how much boilerplate Docker removed right at it's launch. Thank you.
That's for sure, especially for local dev.
Java App Servers with their WAR file deployments across a cluster were the closest equivalent.
Docker was the enabler for alternatives.
What’s funny, though, is that since it was “just” a lightning talk, it didn’t even make it into the roundup of notes I sent to my team after PyCon. But a mere year later, Docker was how my team (and seemingly everyone else) ran projects locally.
Any interesting memories from that day in 2013? From your perspective, what happened in the year that followed that caused docker to be adopted so widely? Were there other key moments or turning points?
That day is forever engraved in my memory... Off the top of my head:
I signed up for a lightning talk expecting an audience of 10-20 people, for a confidential demo of our unfinished project that we had so far only showed in 1-1 demos. Imagine my surprise when I learned Pycon does lightning talks on the main stage!
The previous speaker didn't show, so I started early. In the recording you can hear me joke that perhaps I should get his time allocation too - the MC was not amused.
I went over time and had to speedtalk through the end.. The MC once again was not amused and practically shoved me off stage :-D
We had a small sponsor booth at the even, where our small marketing team was prepared to pitch our product (DotCloud) but was thoroughly confused when a horde of people showed up after the talk, to ask about "Docker" - which as far as they knew was the founder's side project.
I drove down to Santa Clara on my newly acquired motorcycle - my very first trip on the freeway. I remember telling myself, "today is the day to take all the risks!"
I had a pretty much near death motorcycle accident I happened to mostly walk away from which really made me restructure my perspective on things I'm afraid of.
It also managed to make my then to be wife realize she really couldn't live without me when I sent her a photo of a mangled bike and the text "ouchies", so that accident really changed my life positively and radically.
So yeah. Motorcycles. They'll change your life.
Funny how little has changed except the names, I guess.
[0] https://news.ycombinator.com/item?id=5347255
Am I on the right track?
Python and Go are both memory-managed languages. That is perhaps the hardest part of programming (outside of getting the code to do what is intended).
Go and Rust are both compiled, but Rust requires the developer to figure out all of the memory management before compilation, which slows down development to a crawl.
Almost no company will rationally choose Rust over Go if they know how the development speed tradeoff looks like.
To me the arbitrarily complex docker networking makes perfect sense. Especially in a compose setup. I want my containers to be able to talk to each other, but I don't want them on my local network or even accessible (sometimes) from the local machine. The various internal networks offer isolation of containers from the rest of the network stack. If I have a rails app in a docker compose configuration with PG, Redis, Elasticsearch, etc, in a development environment, most of the time I don't want the service/accessory containers anywhere near anything other than the app's container. And if I want several compose stacks to work together, I just put them on the same named network. I apologize if I misunderstood your point.
Which can be achieved with normal routed networks with firewall on the “router” just fine, even on IPv4. Each container can have its own IP address, its own hostname and listen on a standard port, all reachable from containers and from the main OS of the workstation.
The most successful project that we worked on together was "project Piñata" which later became Docker for Mac. It turns out bundling a Linux VM in a native Mac app, and integrating it into the host filesystem, networking stack etc requires lots of magic..
Now, most of the Unikernel team were very active OCaml developers, and created MirageOS among other ocaml projects. While working on Piñata, they simply reached for the tools they were most familiar with.
When acquiring our company, Solomon took us to a remote farm in France for a week for an offsite, fed us fantastic food and wine, and the first prototype of that Docker for Mac VM was working by the end of the week! It took a few months to hack together the shell. There's a video of Solomon and the pinãta launch party on my blog ;-) https://anil.recoil.org/notes/cacm-docker-cover#getting-invo...
There was a whole other slice of work we did for project Piñata too in the form of datakit (https://github.com/moby/datakit) that was well ahead of its time. I was looking at this archived code again just last week while thinking about agentic git workflows!
Prior to the creation of Docker, did you have experience with other containerization technologies like Solaris Zones and FreeBSD Jails?
Did you at some point, either prior to or after making Docker for Linux, consider building something like Docker on top of FreeBSD Jails?
As a user of both Linux and FreeBSD, running Docker containers and Jails on each of them respectively, it would be interesting to see what FreeBSD Jails could be with something like Docker built on top of it while also taking advantage of other features on FreeBSD like fully using ZFS snapshots, ZFS send/recv, ZFS clone, vnet, and so on. Such that spinning up a container would under the hood receive ZFS snapshots from the “Docker registry” equivalent server, and it might clone the snapshot locally before starting up a container instance so that multiple container instances would be using the same locally available ZFS snapshot that was received once from the “Docker registry” equivalent.
You're a genius well done making containerization actually happen.
Went from the good old days of vagrant (xen/kvm) to lxc/lxd to docker, and docker truly clicked all the way from development to production.
But honestly, thanks to Docker hub even more. The CLI is only one part of the puzzle, the worldwide registry is an immense undertaking.
Docker is probably the highest-profile software written in Go. Are you happy with that choice? Would you do it in Rust if it was available at the time, or Zug?
Why doesn't docker respect unix filesystem permissions? I use docker on a shared server, i can delete docker images of other users. This is bonkers.
Is it a defect of a particular docker setup (following standard instructions on ubuntu), or it's an intrinsic feature of docker itself?
I'm curious if when you were originally designing it, what tradeoffs you chose between, for example did you consider a non-daemonized design or was it daemonized from the beginning?
Having had some time now and also seeing how other people have riffed off of the idea and refined it, what would you have done differently? If you have a retrospective at some point already, I'd be interested to read it.
I have some accumulation of personal architectural guidelines that I built up from trial-and-error + reading others' experience and reflecting on how their design could've affected something I made. As a result, I really value those kinds of discussions about tradeoffs.
The folks at infracoders / DevOps meetups would think we were a bit "out there", then what do you know - 2013 rolls around and we quickly started to shift to Docker (too early some might say), but not long after shifting from LXC to Docker it seemed many others finally started to come round to containerisation.
Fast forward to 2026 and I've recently done some work with a client who is only just "getting into containers" (a Windows shop).
Thanks for your early work to bring better containerisation to Linux Solomon. It's funny to think we're 13 years from Dockers release. Time flies.
In retrospect, of course, it was earth-shattering for the company, despite a valiant effort. The entire expertise of deploying long lived Linux servers and carefully shepherding them through change management with Chef became a relic. Totally changed the way applications were deployed.
Was an exciting time!
In recent years I have shifted over to Podman, primarily because it defaults to rootless without a daemon.
Is that something you considered for docker early on , and if so why did you decide on running as root?
I think docker is not weird, it is very linear.
Docker has the merit to take a bunch of tools and to create an organized view of it.
I want to point you to Misterio https://github.com/daitangio/misterio which is a very thin layer on top of docker compose.
It simplify my homelab and works like a charm, and it is a waaaaay easier to use than k8s.
Thank you again, and let me know if you plan a trip in Milan :) for a meeting
The sole idea of containerising an application, its utils and dependencies all into one container is a real big deal in early 2010s. The product has been so visionary that it has been re-pivoted as secure sandboxes which is most needed right now. Kudos!
What are you up to these days?
Hope things are going well with you at Dagger(?)!
Was it all for the `docker.sock`?
Docker completely changed my life. When I started using it in 2014, I really appreciated how well it solved certain problems I had for web development.
It spawned a near endless stream of contract work and video courses for about 10 years, along with being introduced to the Docker captain's program early on. Met so many great people and had a chance to visit a few countries around the world through Docker.
It's one of those pieces of tech that has followed me around for half my career and will likely continue until the end. It's the best community I've ever been a part of.
And yes the community has always been the best part of Docker...
I know that containerization on Unix-like systems didn't begin with Docker, but I definitely think Docker was the first accessible interface for it.
I think app containerization is something that has so many benefits and it's the obvious correct approach for its use cases once you wrap your head around it. This was already understood at big tech companies pre-Docker (e.g. Borg, Tupperware, user app sandboxing on iOS and Android), but I think it really took something like docker to make it mainstream.
As far as I'm concerned, you can probably consider yourself like the Ken Thompson or Dennis Ritchie of your era in terms of the impacts of your contribution.
I hope the plug is appropriate in this setting - apologies if not.
https://dagger.io/blog/dagger-10-beta/
I've struggled for years wanting something that wasn't K3S or K8S. Something good for small cluster management that had a lower level of entry, that I'd feel better handing off to other teams.
Docker Swarm seemed to fight the puzzle, but falls short based on things that needed to be fixed or finalized, that just never happened.
Any reason that whole pathway got abandoned? Other than just "Google uses Kubes so just everyone can use it"?
My memory from that era when Docker was getting big was our biggest user completely ignored us. Which I thought was quite odd at the time.
I did speak to Greg several times.
I think over the years there was natural overlap, people who contributed to Docker would also be kernel contributors. Especially when we spun out runc, containerd, which created a more vendor-neutral space for maintainers to focus on the bottom of the userland stack with minimum drama.
To the extent that Red Hat employees overlap with kernel folks, that made it more difficult to collaborate since Red Hat was overtly aggressive towards Docker, and generally made our engineers feel unwelcome on "Red Hat turf". I discovered that they had devolved into a toxic company with a facade of benevolence. This made me sad because Red Hat was an inspiration to me. One of those "don't meet your heroes" situations.
Not sure what you meant by the "ignoring" part.
We had public discussions on the Linux Kernel mailing list and later on the containers list.
It was a collaborative effort from vserver, openvz, IBM, and Linux Networx, and google. I think those were all the original groups, my apologies if I have forgotten someone.
After Docker made a splash I occasionally asked other developers if they had any contacts at Docker. To the best of my knowledge we never received a bug report, or a feature request. No one from Docker ever showed up at any of our meetings.
I know at one point I wanted to let someone know that Docker they were holding the pieces wrong when it came to security.
Eventually I think one of the OpenVZ guys wound up working for Docker for a time. But that was quite late in the timeline.
I was passing through RedHat about the time of the spat you mention. My impression at the time was that Docker never worked well with anyone, and after some frustrations trying to cooperate the RedHat userspace folks decided it was easier to get a real standard without Docker.
At that point I had gotten the unprivileged user namespace support into RedHat's kernel and changed a lot of minds about it's utility.
I don't remember Docker ever picking up unprivileged user namespace support and as a result was a security disaster waiting to happen.
It felt to me like Docker treated the userspace API for containers like most folks treat the POSIX API. Fixed in stone. In fact there was a lot of wiggle room, and opportunity to shape things for anyone who was willing to dive deep into the technical details.
Red Hat was frustrated with us because they were used to calling the shots. They couldn't fathom that we would refuse a pull request from them. Well guess what, every project is entitled to saying no. And Red Hat's contributions were just overall not good, or they didn't bother to consult with the maintainers on our design principles - like independence from the underlying OS distro.
You're reminding me why working with that company was such a pain. Always with the fucking superiority complex and passive aggressive drama.
As for a "security disaster waiting to happen". My view was, and remains, that the only isolation you can trust is the hypervisor. Kernel level isolation is not strong enough, and unprivileged user namespaces never actually mattered in cloud architectures. Your work on that feature was just security theater and ultimately useless, so spare me the lecture.
I do think Docker embraced user namespaces after my time but I couldn't care less.
When it comes to the author calling Docker weird, after looking at some of their other postings, something about glass houses comes to mind.
Containers From Scratch • Liz Rice • GOTO 2018
https://www.youtube.com/watch?v=8fi7uSYlOdc
Run containers on bare metal already! - Bryan Cantrill - Velocity NYC 2015
https://www.youtube.com/watch?v=coFIEH3vXPw
[1] "Docker implemented in around 100 lines of bash" https://github.com/p8952/bocker
One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.
(rule broken by almost every language in common use, unfortunately)
Because most web servers use OS threads. Nginx, Apache, litespeed, they all spin up `core count` threads but they can all work single threaded just fine as well.
NGINX uses a state machine per task, which as far as I know is kind of what Rust ends up desugaring to, but the explicit state machine is less easy to read.
For Go: you're likely right that OS thread overhead was the main reason for Goroutines, but I don't think it was the only one. There is also a ton of ergonomic efficiency to be had via green threads; like async/await, they're kind of a syntactic unlock that allows you to very liberally use goroutines/channels as a means of semantically modelling actors/state/data flow in your program, where doing that with regular OS threads is both dangerous and (given the start/stop overhead of threads on lots of other platforms) feels like kind of using a howitzer for a flyswatter. Golang saying "yeah, create a million of these, we don't care, it's 4 machine instructions per to start one" was pretty liberating for a lot of people.
The goroutine architecture also unlocked some nice performance at the OS level: GC latency improvements became easier, and some IO multiplexing efficiency a la epoll became almost free to implement due to the ability to cheaply move and suspend goroutines.
It's not green threads (and maybe not lightweight threads; there are so many names for things that may or may not be considered the same...) in the sense of stackful coroutines, but either way, it's a userspace implementation of multitasking on top of a number of OS threads, which is what was relevant to my question.
Cooperative concurrency: it depends. It's sometimes the right move, but has drawbacks. It's usually implemented to enable efficiency or partial parallelism of kernel-space code (IO). There are plenty of platforms that absolutely need it for that purpose: c10k, nodejs's rise to fame, all that. It behaves poorly when mixed with non-cooperative (blocking) code, and is poorly understood by many folks who expect userspace code to be parallelized as well. A common claim is that cooperative concurrency is harder to use than pre-emptive, but that's not true: some common things are harder, but some less common, but more dangerous things like race conditions tend to be much easier. Tokio and other cooperative systems that schedule onto fixed kernel threads can help with the hard parts a bit, but it's a partial mitigation at best.
Preemptive userspace concurrency: firmly disagree with your original point. Green threads are usually combined with userspace-code parallelism (e.g. goroutines scheduled onto multiple threads) and efficient kernelspace interactions (IO multiplexing). In the popular implementations of green threads I know well (Erlang and Go), this is an unalloyed blessing, in that it both removes many of the dangerous edges of threaded programming, facilitates increased efficiency, and lets users semantically model concurrency/parallelism with much less fear of e.g. creating too many threads, torn reads, etc. That's a big benefit: now we can model things like actors or CSP as subprograms written in procedural, synchronous, ordinary-looking code without getting Ph.Ds in operating system internals or having to carefully reason about complex interlocking layers of callbacks and futures to minimize risk of blocking or deadlock. In many ways, this brings the good parts of single-responsibility/what folks call "the UNIX philosophy" into a single process with minimal pain.
Rare cases where the userspace-concurrent runtime fights with your ability to interact directly with the OS, as with Docker's fork/thread/process-global-state challenges, do emerge. But I don't think those are primarily because of userspace concurrency; rather, they're because it's hard to do low-level things in a high-level, managed runtime. You can fork in the JVM, too, and you'll have problems. Different ones, because the runtime's different (e.g. endless RSS burn at idle due to the child GC moving parent objects), but problems all the same. Same for Python: forking in the presence of regular OS threads and/or a running event loop is equivalent to arming a minefield; doable but troublesome, just like it was for the Docker folks with Go.
To be fair, of the same pains as the cooperative model appear with pre-emptive, too, when userspace concurrency systems are retrofitted onto runtimes that do a lot of other kinds of concurrency as well--but that's because of the hybrid/retrofit nature of the product, and not because of green threading as a concept. Java's rocky adoption of virtual threads is an example of that.
> rule broken by almost every language in common use
I don't think so. Go/Erlang/Java do break it (Java very recently). C/C++/Rust/C#/JavaScript/Ruby/Python/Perl/TCL don't break it. PHP ... sort of breaks it, (IMO they just hide cooperative nonparallel concurrency in a nice API, but I'm not well-informed on php)?
Everything in that list feel natural when you know how things evolves over many years. Maybe it is just me getting older and less curious.
rkt (2013) was CoreOS's alternative to Docker and was daemon-less and failed.
Podman came along relatively late (2018) and benefited from a lot of Linux kernel and systemd improvements since Docker was designed.
So while arguably the client/server model is outdated for container management tools, I'm not sure it's weird. This is my workstation for example:
Daemons are maybe the least weird paradigm you can choose for a Unix program.Now it's possible to create containers without elevated privileges (assuming user namespaces are allowed), but that wasn't possible until long after docker had established the client/server model.
And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?
I know a lot about how http works, and it comes in handy, but I can understand how juniors get away with not knowing it as fully as I do.
The first two are just kind of the nature of containers on linux, and applies to any container technology.
Podman does allow you to run it as a non-root user (with some limitations), that was in fact one of the biggest motivations for making podman. This wasn't possible at all in the early days of docker, it wasn't until linux introduced user namespaces, where the root user inside the namespace mapped to a non-root user id outside the namespace that non-root conatiners were possible.
And podman, unfortunately IMO, is also written in go.
Podman, by default, uses crun as the container runtime, which is written completely in C. Podman is basically just a frontend for managing running containers.
Not anymore, they switched to runc (but it can use any OCI compliant runtime)
:D
A lot of the more important syscalls involved do require root.
Podman also has a fully rootless mode which can work for many uses: https://github.com/podman-container-tools/podman/blob/main/d...
Or a setuid binary
I don't know why you're so shocked by this? This is all very basic use of the OCI spec.
> just trying to execute the command within the child container's filesystem
There is no such thing as changing your root filesystem or your uid from an an unprivileged process. There is no "just do it in user mode" either.
So they must have a root process somewhere doing it for you.
It's only in recent years that it's become trendy for distros to disable unprivileged user namespaces in the name of reducing kernel attack surface. So while a privileged daemon is needed for those distros, there is no such inherent restirction on the kernel level.
[0] https://man7.org/linux/man-pages/man7/user_namespaces.7.html
Otherwise it’s basically the same and you can continue using your Dockerfiles and Docker Compose files.
Unless you’re doing something really fancy it’s basically a drop-in replacement.
Unfortunately, there’s an unconditional setsid during podman’s container setup, which as far as I could tell exists so that the container can be detachable.
(This was all so that I could have the same job management with Ctrl-Z, SIGSTOP, SIGTSTP, fg, and bg, regardless of whether the process I’m running starts up a container. The only way I’ve found to have that is by handling chroot/unshare myself.)
If not for the uncatchable SIGKILL/SIGSTOP, I'd say that any process that calls `setsid` is responsible for ensuring signal propagation across the pgid. But since those are uncatchable, I tend to shrug and figure that I'll avoid setsid wherever possible.
There was a proposal at one point that would have had `unshare(NEW_CGROUP)` place the process into a child cgroup, rather than merely changing the name of the current cgroup. This would have meant that cgroups could actually be used for nested process management. Unfortunately, there's no guarantee that a process can ever make nested cgroups it wouldn't be sound for all supported resource constraints, so that proposal never went in. But that would have been nice to actually have nestable job management.
How?! After a process calls setsid(2), it becomes a leader of a new process group so it stops receiving signals from its former process group!
> But that would have been nice to actually have nestable job management.
Yes, it would be. Just imagine: a simple, straightforward way to group a bunch of processes (and their potential children, inescapably), signal them all at once, wait for them all at once, etc. Instead, we at best ended up with docker compose and k8s. And they kinda do solve that problem, no question about that but it's just so incredibly heavyweight...
By limiting its usage to the period between fork and exec, so that the process calling setsid knows that the parent process is propagating signals to it.
So, my phrasing was inaccurate to the point of being entirely incorrect. I should have said that any process that calls `setsid` is responsible for ensuring that signals are being propagated. (And saved! Definitely didn't make a mistake on the internet and then play it like I was right all along.)
Well, again, the process that calls setsid(2) ceases to receive the signals from its previous process group, so there is nothing for it to propagate. The only way this propagation can work is with cooperation from the parent process of the setsid-calling process: it will need to manually pass the group-wide signals to its setsid-calling child — which it can't reliably do for two obvious reasons.
> By limiting its usage to the period between fork and exec, so that the process calling setsid knows that the parent process is propagating signals to it.
Wait. Are you implying a scheme where the process A, that wants to do setsid(2), forks itself, and it's this child process B that does setsid(2) and all future actual work, while process A merely lives in a "while(1) { sig = sigwait(); signal(pidB, sig); }" loop forever? This almost works but doesn't survive SIGSTOP/SIGCONT/etc. and, of course, you can't propagate SIGKILL. And SIGKILL-propagation is arguably one of the main attraction of the process groups — you send SIGKILL to the process group (or a container, for that matter), and all of it dies with no further work on your part.
This is very important for debugging! Not just that process, but the processes running in that Docker can be found in /proc.
Why this is useful is that you can run gdb outside of that docker and point it to a process in the docker, including setting the file system root to the docker one, so that it gdb reads the correct objects from the correct paths, to get the right symbols and addresses.
And implying the system kernel still has to run the container kinda ignores kata containers and gvisor runtimes.
Other weird facts about OSes: a filesystem is a database, the Linux kernel is a monolithic program, "hidden" dot-files were a Unix bug that got carried forward, deleting a file doesn't remove the file from your machine, virtual memory is a lie, your computer can't run multiple programs at once (it's just running one at a time real fast), in 12 years most systems will crash due to one number getting too big, you technically never need to reboot because you can launch a different version of Linux while already running Linux, you can freeze a program to a file and then later resume running that same program in the same state (but nobody uses it), files aren't actually stored in directories, there's a maximum number of files (and size of file) on most filesystems (which you can use as a hacker to lock up a machine, despite file quotas existing and nobody using them), you can lock up most systems by forking too many times / using too much RAM / accessing disk too much, there's a default number of open files and/or TCP ports which is artificially low and you can use to again lock up a machine, pipes are just two file descriptors connected to one memory buffer, an IP address is just one long number split up into bytes, etc.
Of course it can.
Try to make your own VM runtime next, you might be surprised!
Johnathan Blow's talk on the bronze age collapse pushed me to simplify my production stack. I wanted 'just processes', so I stopped using docker there. I'm happy with the result.
I love Go and use it for everything I can but will absolutely concede this point. It's clearly possible to write the most successful container runtimes on earth in Go, but you absolutely have to fight the language/runtime sometimes. Since containers are based on processes, handing off a container spec to a minimal rust runtime seems like what we should all be doing by now. crun has just proven good enough that AFAIK no one has RIIR.
You'll be glad to know of youki then: https://github.com/youki-dev/youki
Performance is basically irrelevant because you're not really interacting with the running application. You're running as root, so security-critical. You're talking to the internet for things like fetching images. You need rather complicated business logic to deal with all the possible options, protocols, and file formats.
Sure, you could do it in C, but dealing with dependencies is a major PITA and you're permanently one tiny mistake away from getting pwned, while not needing any of C's strengths. Go is safe, easy to write, and ships as a single batteries-included binary: what's not to like?
Go is good for writing web servers. For system software? Not so much. 90% of Unix is written with C. It's not that bad.
Side note almost nothing about Docker is weird. This article chooses rather boring design decisions and picks on them.
What made Docker unique were there things:
1. Dockerfile concept. Building images layer by layer. Caching layers. Everything in a single file. It made possible to iterate on image quickly. It made possible to implement CI builds efficiently.
2. Docker network registry concept. Putting universal images onto a server and share them with the world.
3. Momentum. A lot of software that was traditionally complicated to deploy, suddenly appeared in docker servers. I remember how fascinated I was, when I could deploy wordpress website with tiny docker-compose. And how contained it was. I could just delete every trace of it from my host. I always felt uneasy installing various things onto my host and docker just solved that uneasiness.
The main draw for me was that it made containers absolutely trivial. Sure, you could spend ages figuring out how to DIY some isolation, but in practice an awful lot of people stopped at having Apache fork its workers with a non-root uid. You want isolation and a repeatable setup? Spin up another VM and have Ansible set it up! Docker provided a batteries-included lightweight way to isolate and manage multiple applications at once.
Dockerfile? I would've been happy with a Bash script, or handing it a tarball. Caching and fast iteration? Compute is basically free. A single file? Well, there's already a whole bunch of app source code files sitting next to it, so what's a few more in a `setup/` directory? The registry? A shared base could've also been achieved with, say, a `deb` file providing the right dependencies, files, and post-install scripts.
I do agree on the momentum, though. Having every piece of software reduced from a multi-page setup guide to a single command and not risk screwing over your entire machine is absolutely magical.
You can absolutely build images with a bash script, here's an example (its vibe slop, but it works): https://github.com/andi0b/vibe-todo/blob/main/docker-build.s...
And for most people, that's a bad idea. Most people don't have to know what RPATH, ldd, multiversion/simlink library discovery, etc are. And even if they did, Dockerfiles give you an easy way to do something you couldn't easily do before: multi-stage builds and a highly robust way to say "drop the parts of my transitive hull that were only relevant at build time". Dockerfile semantics kind of descend naturally from needing the behavior of stages, parallelization, cache keying for image layers, semantic layering (FROM), and making simple use-cases relatively at-a-glance/contained.
My point is that the Dockerfile is not a particularly good build system. It doesn’t solve any of the hard parts (how to build the thing, what to build when) and only solves problems it created (how to avoid re-running something on every invocation) and does it poorly (it doesn’t track dependencies, just determines cached-not-cached on every line.) It leads to awful hacks like putting every build step on an enormous chain of `cmd && \` lines to make sure you don’t produce extraneous layers.
A reasonable build system tracks dependencies for what needs to be built and has a target it’s trying to get to. Docker just says “do your build imperatively and we’ll figure out caching” and it’s extremely easy to shoot yourself in the foot leading to overly large images.
An example: there’s no way to say “build container: do this build and put the product here, runtime container: take the product and store it here” without creating a temporary layer in the build container that is not used anywhere else. So if your artifact is 1GB, you will always require 2GB to do a build: one for the runtime container (where you want it) and one for the build container (where you only needed it temporarily.) You just have to do “docker builder prune” every now and then when your disk inevitably gets full, there’s no way to simply not produce the duplicate in the first place.
(My workaround is to keep the artifact in a type=cache mount, store its SHA sum in the build container, then in the runtime container, grab the artifact from the type=cache mount and compare the SHA sum to make sure it didn’t change. Cache mounts can be GC’d at any time so it’s not even guaranteed to work.)
When I used docker at scale back in the day, the thing that worked best was always: bind-mount an artifacts dir into the build container, run the build. Then package up the artifacts dir. Then at runtime, fetch the artifacts, bind mount them into the runtime container. Then you can use a real artifact store on a plain HTTP server instead of messing with docker layers (no more worrying about pushing 100 1KB layers because you didn’t put enough &&’s in your dockerfile) and it was so, so much faster than any docker registry was in practice.
Dockerfiles are great for simple stuff but, if I already have a great build system that produces single binaries, it’s so much more pleasant to just use that.
Layering is as much an abstraction paradigm as a performance one. Its not trivial to verify every loose bootstrap script or .deb that would be needed to build a sandbox... But in a Dockerfile you can see its just some official alpine image base, and a few added files and an entrypoint.
But the argument I used for advocating Docker in environments where people were not fans of it was that the Linux container had become the de facto standard unit of deployment. There is so much tooling written under that assumption, and if you want to use any of that tooling, you need to package your deployments as Linux containers, even if you want nothing to do with Docker otherwise. That's why it had so much momentum.
Then a fourth point I think you're missing is that the concepts used by Docker are relatively simple. Before I understood what Linux containers were, I used to think things like, "Why would I care about Docker when I can isolate my processes with Linux users, cgroups, namespaces, etc." What I didn't know at that time is that "Linux users, cgroups, namespaces, etc." is exactly the thing Docker was giving a friendly UI to.
Basically both described in https://www.usenix.org/legacy/event/lisa11/tech/full_papers/... (in detail) and https://www.usenix.org/conference/usenix-atc-10/apiary-easy-... (in a "sexy" use case to try and motivate it) a few years prior. (and I get annoyed when people say "so you're saying you invented containers?" because they equate docker with containers. And I'm like no, containers existed for a long time going back to logical partition on mainframes in the late 70s/early 80s. All I claimed was a novel way of building and distributing file systems for them that made them much more useful and valuable and in apiary a novel desktop setup that made use of it to improve desktop security)
Even the original docker implementation of using a union fs (docker using aufs) was the exact same as described.
Even if docker came up with it independently (though conferences like LISA and Usenix ATC are where the papers were presented and were where companies like dotcloud/docker would advertise their wares (and in fact docker inc was an exhibitor there, so it wasn't hidden), the differences between the uniqueness of docker and what is described in the paper are minor (in my perhaps somewhat biased opinion of course), and i was clearly first.
"It's an idempotent language for OS level dependencies so you can be sure exactly what is running on your server"
Gimme gimme gimme!
It's not a vm: It's all namespaces, baby! Unshare one or more of your mnt, pid, net, ipc, uts, user, cgroup, or time namespaces and you have a "container".
A few cool tricks:
`range shell python:3:12` - opens a shell image without pulling the entire image.
`range shell ubuntu --ubuntu` - you are in Ubuntu. Type any command Ubuntu doesn't have and it comes from the image that has it. On the fly.
`range run --network none ubuntu` - network-isolated sandbox
range run ebs://snap-... -- cat /etc/os-release - opens an EBS snapshot with no EC2 instance
Another cool trick:
`range run python:3.12 --mount hf://moonshotai/Kimi-K2-Instruct:/model -- du -sh /model` says 959G instantly :)
Check it out here - https://getrange.sh
Range always reads only the bytes your program touches. Nothing else. You can prob shell into a 1TB image from a Raspberry Pi :)