Great news. I'm not a fan of Docker's new monolithic approach to containerization. Things like orchestration and networking should not be included in docker, but rather pluggable.
I think this was the original model proposed by Docker. What we have now is (as other posters have mentioned), a Docker organization reasonably bent towards creating value for their investors, which means they need to start building things that, you know, make money.
To clarify, I don't think there's anything inherently wrong with what Docker's doing, but it is at odds with an entirely open, pluggable system. It doesn't make any sense for their business model to truly make it easy to just use their containers and none of the revenue-generating offerings.
I've not been following the discussions but if it's such a critical piece of the whole puzzle and it's in everybody's interest that it remains open, wouldn't a foundation, rather than a single private company, be the best venue for leading the project forward?
Then how do you fund that foundation? Good developers cost a ton of money. Marketing, organizing events, organizing conferences etc also costs a ton of money. I think something like Docker, especially given its growth and adoption rate, never would have been possible without VC funding. VCs wouldn't invest in a non-profit foundation.
I prefer the Unix model - many programs that work together. That might not be practical for networking (a natural plug-in, probably), but feels like it should be the way for orchestration.
The Docker image registry and image management should really be a separate program as well - that is a huge pain point that Rocket seems more likely to get right.
Interestingly enough, with flannel, docker's advanced networking capabilities become pretty trivial, and communication across hosts is also pretty trivial.
I think all in all, CoreOS has built out a ton of tools to make using Docker easier, and they're all very well defined, and compossible. I'd even say that a lot of docker's features could be completely removed by using some of these tools.
Links? Nah just use ips/dns + etcd for service discovery.
Networking? Need very basic bridged networking, and flannel will handle communication on a single host, or multihost.
Deployment? Use fleet.
Not that all these are 100% perfect like I've made them out to be, but any individual component could be swapped out if you want.
I think this is probably more indicative of the issue that Future Docker would like to be a CoreOS-competing platform, and has been edging towards that state. This is CoreOS' natural bounceback from that.
The thing I like about the link model is that they hide your containers from other containers and only expose the connections you want (I think using iptables?)
I'd like a tool that makes this linking easier outside of Docker, but for now this is one of the features I like about it (although holy moly do Docker links have a lot of baggage you have to bring along for the ride, like giving everything names).
From the docs, it looks like that has a dependency on Docker, which kind of defeats the purpose. If I'm stuck with Docker, I'm better off just sticking with links: I'm looking for something that could work with systemd-nspawn, etc.
The problem with the CoreOS tools is that they're pretty tightly coupled. We looked into using fleet to manage our deployments. Unfortunately, it relies on a minor feature of etcd and cannot work with Consul, our corporate standard. Flannel? Yep, again, tightly coupled with etcd.
I like some of the ideas behind the CoreOS tools, but until they start playing well with others, they're a non-starter for me. I'm not interested in tools that try to lock me into other, inferior, tools.
The Unix model works great at the network later. Otherwise I couldn't be building a complete, multi tenant, docker container as a service / infrastructure as a service, cloud. Built on top of an end to end SDN.
Would you consider open-sourcing / documenting / blogging how you do that (or even providing some pointers to help get me started)? I'm playing with kubernetes and AWS, and it isn't clear what the best networking solution is (rudder, weave, IPv6, SDN); it would be helpful to have some pointers on the OpenVswitch front.
(I only ask because it doesn't look like it is your core business)
This looks very interesting - it'll be really useful to have something like Docker that isn't so monolithic - it should be much more composable in new ways.
it's a very exciting time for Linux Containers. it's been a fun to watch the evolution from BSD jails to lxc to docker, but the rate of innovation and usefulness is certainly accelerating. it sure seems like rocket's approach will be much less of a black box than docker images/registry, which should make it much more approachable to people trying to understand what linux containers are all about.
Yes, we have prototyped doing socket activation with rocket already but the patches haven't been merged. So, yes, the intention is to make socket activation work.
I'm all for a new container runtime if it lets me start containers as a non-root user. Allowing non-root users to start containers would open up a whole new level of applications, particularly on multi-tenant HPC-style clusters.
Yes, finally! I've been working around this by making users inside the containers that people launch jobs on, but it would be much easier if they could do it individually in their own namespace (not using a docker group either).
I'm not saying that Rocket will support this, I just hope it does! I really want users to be able to spawn a container themselves without requiring special privileges.
* Despite Brandon Philips (CoreOS CTO) serving on the Docker governance board, Docker has aggressively expanded their scope well beyond their original container manifesto.
* CoreOS believes the Docker runtime is now too unwieldy and "fundamentally flawed"; the unwritten word that really sprung to mind was that Docker was getting "greedy."
* CoreOS reaffirms their original operating model of being capable of running their infrastructure on and with Docker.
* Rocket is CoreOS's answer to stay true to the "simple composable building block" mantra.
> CoreOS believes the Docker runtime is now too unwieldy and "fundamentally flawed"; the unwritten word that really sprung to mind was that Docker was getting "greedy."
I wonder if that comes from the partnership with Microsoft.
Regardless of whether your comment was lightly sarcastic or not, I agree that the Docker VMWare[1] & Microsoft partnership announcements may have been conditional upon a committed Docker roadmap outlining some or all of the features that others (such as CoreOS) may feel should be broken out. Typically larger ecosystem players want to be assured that your offering will have a clearly defined role within their existing ecosystem that plays to your core brand and technical competency.
They raised $55 million [1], so you have to believe their ambitions are to extract as much rent from the container ecosystem as possible. That's not a bad thing, but it's behind a lot of their moves.
Docker's MO is to become "that thing that is on all servers" so that when they flip the switch and start monetizing off support and tertiary services, people will be more-or-less locked in.
It has indeed surprised me how quickly a normally-slow-to-accept-new-things community has adopted Docker (even well before it was considered "stable").
> how quickly a normally-slow-to-accept-new-things community
I think you're referring to the sysadmin community - but I think the driver for this has been the search for deployment nirvana. Deployment is a much more fragmented field, so it makes sense that a good solution would find fertile ground.
Absolutely. Not only does it simplify deployment, you also get the ability to quickly spin up a new development environment. That means it's easy to dip a toe in and slowly increase how much you use it.
> You are not "locked in" by Docker Inc if you are using Docker just like you aren't locked in by Github if you are using git.
A much more accurate analogy would be you are not "locked in" by Oracle if you are using MySQL. It may be true today, but no guarantee that will always be the case.
Despite the attempt of some to move goal posts, you're still guaranteed that you won't be locked in by Oracle even tomorrow. You still have the source code for the version you're running right?
If people think the software moves in the wrong direction it will be forked (see MariaDB). Nothing world changing will happen.
Docker Inc. seems to make a lot of effort to ensure Docker is a truely open project. I get the feeling that people think that handing your project to Apache is the only way to prevent vendor lock in these days.
This is great news, particularly for Enterprise customers adopting containers. IMO, Docker's 'new' direction completely ignored the tremendous amount of support they had from the sysadmin and devops communities.
But crucially, they also crossed the business models of many startups (including CoreOS, Weave, Flocker, etc.) that rely on Docker maintaining an Open Platform. So this is an entirely logical response.
I'll be surprised if now Docker in response doesn't unveil an 'enterprise' Docker version that basically just strips away the unnecessary features and has more security by default. The enterprise market is just too valuable to let it just slip away like this. Your move...
I attended Docker Global Hack Day #2 on Oct 30 from Austin. A talk was given on an active Docker project for host clustering and container management, which was non-pluggable, and made no reference to and used none of the code from CoreOS's etcd/fleet/flannel projects.
This was where I first started worrying about CoreOS and Docker divergence.
But since the hack day there has been a pretty reasonable (IMO) GitHub discussion about the tradeoffs between out-of-box ease of use and customizability.
I saw that same presentation at the same event, but came away with a very different impression: the container management they showed was implemented completely outside of docker itself, with no patches to the docker codebase needed. Also, IIRC it actually did use significant code from etcd for coordination.
It had no etcd in it and the POC was implemented as part of the Docker API/CLI, as best I recall. There were significant questions in the discussion about etcd not being there.
Docker's 'new' direction is to direct its attention towards solving the orchestration and management problems involved in actually running infrastructure on Docker.
A number of third parties had begun work on various (sometimes proprietary) orchestration and management systems for creating a reliable/scalable/easily manageable cluster with Docker as a building block. CoreOS is one. But Docker is pushing towards an official, open-source orchestration/management system that threatens to make all of those companies irrelevant.
I think it is a great stand for Docker. Very recently (IMHO in 1.3), it merged the functionality of Fig into Docker.
I think Docker orchestration and coreos can coexist - if I had to use COREOS to use the goodness of Docker, then systemd-nspawn would come and eat Docker's lunch.
I wish that Docker bless one of Ansible/Chef as the official orchestration base and take it forward. I really don't want to earn something Docker specific.
I agree that Chef and Ansible are different than container orchestration today - especially, when you look at low level stuff like networking, mounts, etc... But I guess what I was saying is that it is not hard to add these features to them.
They already have a specification format that works well and check for idempotency at its core. Unless you mean something like etcd is fundamental to container orchestration, which I don't believe it is (we run a couple of containers in production using Fig)
> Docker's 'new' direction is to direct its attention towards solving the orchestration and management problems involved in actually running infrastructure on Docker.
IME examining Docker, this is actually the hard problem.
I believe the two (Docker & CoreOS) might have rather similar strategies and / or product roadmaps.
What seemingly gets mixed up by quite a few commentators on this topic:
Docker is an orchestration, deployment, management, etc solution - the "container" is created by LXC, jails, libVirt or other OS features and now also libContainer.
This discussion also shows how far away / how early we are with "containerization" or container that are exchangeable / movable between different (OS) environment - we are discussing the companies that are building cranes to load and unload the boxes before we even have an understanding how the boxes will really look like.
Woohoo! Docker is going off the rails trying to roll container orchestration into it. Kubernetes is arguably a "larger" project than docker, and docker is out of their depth thinking they can just roll such a feature in.
That may be a little overly dramatic. There have been failed fragments/forks/derivatives, but there have also been some sweeping successes.
It's too early to foretell the fate of Rocket. Containers are getting lots of attention, so I'm actually pretty happy to look at this as a potentially rewarding experiment. Worst case, it fails and we keep using Docker (or whatever else springs up).
There were many VM engines, now there are a few. I imagine the same thing will happen with container technology. Generally 1 technology stands out, then things fragment, then things coalesce into a tiny handful of solid solutions.
Docker may or may not be the container engine that lasts a long time. There is a reason they raised a bunch of money. Clearly containers are going to be big, but is Docker the one that goes on to be dominant? Docker is trying through building features & biz dev, but it's far from over.
These aren't really containers. They're giant statically linked binaries, more or less. The actual operating system is now just a VM host for running containerized giant WIMPs (weakly interacting massive programs). Fast-forward a few years and the host can wither and die and be replaced with a proprietary or custom/fragmented management layer. Linux survives only as an internal pseudo-OS within each mega-binary "container."
Edit: what I was really getting at was that these technologies are patches for the inadequacy of the OS. The fact that we need containers at all stems from the difficulty of managing software installations, configuration, etc on the actual operating system.
> The fact that we need containers at all stems from the difficulty of managing software installations, configuration, etc on the actual operating system.
Great post. However, containers are an easy concept to grasp. Even if the actual OS could be fixed, you'd still want some similar concepts even if the names were different.
Fragments? Certainly.
Dies? Linux has been fragmented from its inception. If you include the world's Android phones, Linux probably runs on more computers than any other kernel or OS. Rocket will not kill Linux, containers, or docker. In the worst case, it will kill CoreOS, and even that's unlikely.
Likely not even close. Just about every single washing machine, refrigerator, microwave, digital stove, etc, runs a variant of an open source operating system called Tron, or the more common ITron variant.
I dunno if linux fragments is causing linux to die... BSD have many fragments and they're doing fine.
I think competition is good, this will give us an option that's not monolithic.
I didn't realize docker direction was to encompass orchestration until this thread. This isn't something I want to use docker for and also I'm glad the competition is address the security issue where there is a need for more security.
And with a rival option I'm happy to choose rocket as an option when it's stablized and there aren't any other options out there.
I have been concerned that Docker's scope was expanding too far for a while now, so I'm glad to see an alternative that might work appear on the horizon. That said, I am somewhat concerned that CoreOS has a suspiciously similar business model to where Docker would probably like to be.
It's in a business's best interest, and exceedingly common practice, to "land and expand" with something clear and compelling, and following that add features to compete with alternative solutions. I don't think there's anything inherently altruistic about CoreOS that would keep Rocket lean in the long-run, especially as they begin migrating their various tools away from Docker containers.
I had the same initial reaction, but I think there's good reason to trust the CoreOS folks to remain faithful to the project's goals. Containerization (although foundational) is one part of CoreOS's platform. It's easy to see where the boundaries fall, e.g. I expect systemd and fleetd to keep their respective functionality and not overlap with Rocket.
It become pretty clear once dotCloud became Docker Inc. that they intended to capitalize on the "Docker" brand to sell an integrated orchestration platform. CoreOS already has enterprise customers for their operating system and related components. They seem like the perfect team to take this challenge on.
I hope you can understand that it's frustrating when, after hard work pitching an API to dozens of ecosystem players, spending weeks trying to wrangle a working implementation which makes as many of them as happy as possible, without compromising integrity of design - after all that, in the end, all it takes is one unhappy camper to write a blog post and that immediately tramples everything else.
It's even more discouraging in this particular case, because after this blog post, Alexis and I have discussed this topic extensively, and as a result he has since joined the effort. In fact I will be hacking with him in person on integrating Weave as a native networking plugin in 2 days in Amsterdam.
So, sorry for the insta-snark. But it can be frustrating to see so much good will and hard work be crushed in a second.
Hey, regardless of the technical sides of anything, I'm sure this is not a fun day for you. I think you're handling the situation terribly, but still, not a fun day. Anyway, docker is awesome, and thanks for building it. I know it's made my devops life a lot more enjoyable of late.
Concerning Weave that's quite a good news, the weave point of view for docker networking is good and can be easily setup in many (not all of them of course) infrastructure.
The difference here would be, IMO, that they have clearly made openness one of Rocket's goals: the formats should be well-specified and maintained separately so that other implementations can run them.
As a heavy user of CoreOS and docker, I'm interested to see how this plays out.
My problems with docker have been the security model, for which the only recourse I've had is to use the USER keyword in my Dockerfiles. Furthermore, networking has been a pain point, which I've had to resolve by using host networking to access interfaces.
Let's see how rocket deals with these issues and others. I pay for CoreOS support, so I'm glad to see that they're addressing this.
Docker's main focus is to "get people agree on something". And they are doing great in getting traction and adoption. But if everyone starts to create their own flavor of containers, we still don't get portability across servers and clouds. It would be better IMHO if Rocket implements the Docker API, or if they collaborate together in creating a minimal standard. Then everyone would benefit. I'm really curious how Solomon will respond to this...
FWIW, part of the design difference is that rocket doesn't implement an API. When you do `rkt run` it is actually executing under that PID hierarchy; there is no rktd that forks the process.
This is a design goal so that you can launch a container under the control of your init system or other process management system.
Improving the security model of docker is mentioned. Docker is known to be currently unsafe to run untrusted containers. Does anyone know yet if Rocket plans to support running untrusted containers safely, ala sandstorm.io?
Unlikely. Doing that requires a willingness to break things (disabling vast swaths of the kernel API in order to reduce attack surface). Sandstorm is fine with breaking things because Sandstorm is all about rethinking the platform and that means apps already need to be tweaked in a number of ways (see: https://blog.sandstorm.io/news/2014-08-19-why-not-run-docker...). Docker and Rocket are very much designed to provide "Standard Linux" inside their containers, and be able to run standard Linux applications.
It looks like Rocket actually intends to be more conservative than Docker:
"Additionally, in the past few weeks Docker has demonstrated that it is on a path to include many facilities beyond basic container management, turning it into a complex platform. Our primary users have existing platforms that they want to integrate containers with. We need to fill the gap for companies that just want a way to securely and portably run a container."
So it's actually moving in the opposite direction, compared to Sandstorm.
(You of course know this already, but disclosure for others reading: I'm the lead dev of Sandstorm.)
It isn't tied to systemd. The stage1 that is in the current prototype uses systemd to monitor and fork processes but we would love to see other stage1's that configure other process runners. For example configure and run a qemu-kvm filesystem as the container.
Also, even though it is using systemd to monitor and fork processes a design goal is to run on all Linux's that have a modern Kernel.
What about non-Linux platforms (FreeBSD, Mac OS X with a kext)?
One thing that I believe Docker has failed at is in taking a purely declarative approach to image definition; rather than specifying the packages that are assembled/inserted to create the container, Docker ships around non-portable Linux binaries.
I second that. At the begining Docker people were mentioning adding FreeBSD Jails support, what seemed to me an awesome thing, a platform independent containerization middleware, but recently they just seemt to forget about it and they're doing only linux-centric things - what a shame.
Yes, but the Docker Remote API allows for a great deal of implementation freedom -- including running on a different OS substrate. We're doing this with sdc-docker[1] to run Docker on top of SmartOS and in a SmartOS container, and the Docker folks have been incredibly supportive. Despite the rhetoric, Rocket appears to be much more bound to the OS platform than Docker -- and given @philips' comment that "part of the design difference is that rocket doesn't implement an API"[2], this binding appears to be deliberate.
It depends on how you look at it. The Docker Remote API provides an abstraction on the OS substrate and definitely binds you a lot more to Docker and their model. The "rocket doesn't implement an API" means all that it isn't really doing much more than kicking off the container and using the existing OS substrate to manage everything else.
I can see where for SmartOS & Windows the Docker approach is more flexible. If someone has already settled on Linux, but they have their own ideas about how to manage containers within Linux that have nothing to do with CoreOS, the Rocket model is going to leave them much more flexibility.
The app container specification has socket activation in it. This is going to essentially tie it to systemd. Otherwise you will need another daemon running to do the socket activation, but then that would seem to be a "fundamentally flawed" execution model.
You may be shocked to discover that socket activation is actually a pretty old idea, and can even be found on non-Linux platforms. systemd embraces it in a pretty big way, but I can't see a problem getting the app containers to work with one of the other socket activation models.
And you may be surprised that I ran socket activation well over 15 years ago, so yes I'm well aware of the approach. The comment is more around the fact that in CoreOS's post they seems to harp on the security of a daemon process running as root that is responsible for spawning containers. What I'm saying is that with socket activation you will essentially have that again. Rocket can only work around it today because they have systemd as PID 1 running as root doing the socket activation.
Socket activation isn't mandated and completely optional. If an application detects it didn't get its sockets it can certainly just start listening instead if it wishes.
Socket activation does not need systemd in any way. You can launch any socket-activated daemon with https://github.com/LEW21/sdlaunch - without any dependence on systemd.
The post mentions not having a daemon running as root, but then you have to run `rkt` as root anyway. Won't this just mean that instead of having a single implementation of a Rocket daemon running as root, there is now one custom one every time it needs to be automated?
It's great to see this problem broken up into reusable pieces though. It totally makes sense to function without a daemon, especially out of the box.
There actually is a significant difference between having 'rkt' as a setuid-root process that's invoked from the command line, and having a docker server always running waiting for commands. There are more ways for a potential attacker to get at the server. So, Rocket at least looks like they're trying to shrink the attack surface.
> There actually is a significant difference between having 'rkt' as a setuid-root process that's invoked from the command line, and having a docker server always running waiting for commands. There are more ways for a potential attacker to get at the server.
Wrong. With a server, the only thing an attacker has control over is its input. With a setuid-root binary, they still have control over its input, but they also have control over the entire environment under which it executes, including many things that developers generally assume an attacker can't control. Setuid binaries are incredibly scary from a security perspective and much harder to get right than servers.
"Looks like" is usually about as far as people get when they start down this road. If they make it down the road, they arrive at mess. Just look at what OpenStack has been through :)
So then, I guess docker could just run the two servers, one internal as root and one public as not? That's a pretty quick fix.
> There are more ways for a potential attacker to get at the server. So, Rocket at least looks like they're trying to shrink the attack surface.
hm.. I don't think that's a given at all! There's been many issues with setuid-root programs. And I've seen that the OpenBSD guys favor privilege separation by breaking breaking up daemons into several parts that communicate using a very strict set of commands. For example a dockerd that does most of the work, but talks to another daemon (dockerd-root) when it needs to do anything privileged.
How will App Container Images be built? I'm guessing that unlike Docker, the standard App Container build tool(s), if any, will be separate from Rocket.
Right now there is a `actool build` subcommand that will build an ACI given a root filesystem. That tool is used to build the validation ACI's and the etcd ACI. It is rough right now and we will make it simpler to use overtime; and as rkt gets better people can run the build tool from inside of a container given source code.
Nice. It occurs to me that since an ACI is just a tarball, the build process is decoupled from the runtime engine, unlike in Docker. I've found the Docker build process to be unsuitable for creating minimal images (though I've read that nested builds plus layer squashing will fix this). It'll be interesting to watch the exploration of different build tools and processes that Rocket's decoupled approach will enable, if it catches on.
Docker already supports alternative build systems via docker import.
Realistically, if the stack is broken into a dozen pieces then somebody will create a bundle with sensible defaults (let's call it "CoreOS") and then we'll be back in the same situation.
I actually need gpu, not the ui. I need it to do scientific computation. Video streaming service is another case. gpu has better video encoding capabilities.
I previously heard that docker has trouble loading device drivers.
Not the parent poster, but needing GPU isn't necessarily the same as having UI. You can use GPU for a variety of general purpose math (Example: mining bitcoins, or doing stuff like Folding@Home), or for offline rendering.
yes, I understand offline rendering. I'm looking into egl off screen rendering. But due to historical reason, the current gpu drivers (NVIDIA) need x server.
Certainly. The kernel can simply pass through the device, although you lose some of the security of containerization that way. There may be issues with multiple containers sharing the same GPU though.
Have a look at this container [1] I put together for accessing GPU instances on AWS via Docker. Runs various compute tasks including multiple containers against a single GPU without issue.
From the looks of your other comments in this tangent it might be exactly what you need or a starting point at least.
It's a base for these BOINC [2] and F@H [3] containers.
Thank you very much! this is really useful information. Aside from cuda, I also want to make EGL/opengl work with docker, hopefully I can find examples for that.
Interesting what the CoreOS team is building. If the code becomes as neat as some of the main parts of CoreOS, then this alone merits attention, we cannot have to much security.
294 comments
[ 4.2 ms ] story [ 190 ms ] threadTo clarify, I don't think there's anything inherently wrong with what Docker's doing, but it is at odds with an entirely open, pluggable system. It doesn't make any sense for their business model to truly make it easy to just use their containers and none of the revenue-generating offerings.
The Docker image registry and image management should really be a separate program as well - that is a huge pain point that Rocket seems more likely to get right.
I think all in all, CoreOS has built out a ton of tools to make using Docker easier, and they're all very well defined, and compossible. I'd even say that a lot of docker's features could be completely removed by using some of these tools.
Links? Nah just use ips/dns + etcd for service discovery.
Networking? Need very basic bridged networking, and flannel will handle communication on a single host, or multihost.
Deployment? Use fleet.
Not that all these are 100% perfect like I've made them out to be, but any individual component could be swapped out if you want.
I'd like a tool that makes this linking easier outside of Docker, but for now this is one of the features I like about it (although holy moly do Docker links have a lot of baggage you have to bring along for the ride, like giving everything names).
https://github.com/vishvananda/wormhole
I don't like the sound of locking into one vendor for everything.
I like some of the ideas behind the CoreOS tools, but until they start playing well with others, they're a non-starter for me. I'm not interested in tools that try to lock me into other, inferior, tools.
(I only ask because it doesn't look like it is your core business)
* Despite Brandon Philips (CoreOS CTO) serving on the Docker governance board, Docker has aggressively expanded their scope well beyond their original container manifesto.
* CoreOS believes the Docker runtime is now too unwieldy and "fundamentally flawed"; the unwritten word that really sprung to mind was that Docker was getting "greedy."
* CoreOS reaffirms their original operating model of being capable of running their infrastructure on and with Docker.
* Rocket is CoreOS's answer to stay true to the "simple composable building block" mantra.
I wonder if that comes from the partnership with Microsoft.
[1] http://www.forbes.com/sites/benkepes/2014/08/25/vmware-gets-...
[1] http://www.crunchbase.com/organization/docker
It has indeed surprised me how quickly a normally-slow-to-accept-new-things community has adopted Docker (even well before it was considered "stable").
I think you're referring to the sysadmin community - but I think the driver for this has been the search for deployment nirvana. Deployment is a much more fragmented field, so it makes sense that a good solution would find fertile ground.
You are not "locked in" by Docker Inc if you are using Docker just like you aren't locked in by Github if you are using git.
A much more accurate analogy would be you are not "locked in" by Oracle if you are using MySQL. It may be true today, but no guarantee that will always be the case.
Docker Inc. seems to make a lot of effort to ensure Docker is a truely open project. I get the feeling that people think that handing your project to Apache is the only way to prevent vendor lock in these days.
Yes but I don't have the skill, time, resources and will to maintain MySQL if and when Oracle goes evil (I mean more evil than now ;-)
This is why I choose carefully what companies / groups I depend on for my future computing needs.
Docker seems decent but I don't think I want them to do ochestration..
But crucially, they also crossed the business models of many startups (including CoreOS, Weave, Flocker, etc.) that rely on Docker maintaining an Open Platform. So this is an entirely logical response.
I'll be surprised if now Docker in response doesn't unveil an 'enterprise' Docker version that basically just strips away the unnecessary features and has more security by default. The enterprise market is just too valuable to let it just slip away like this. Your move...
This was where I first started worrying about CoreOS and Docker divergence.
https://github.com/docker/docker/pull/8859
It had no etcd in it and the POC was implemented as part of the Docker API/CLI, as best I recall. There were significant questions in the discussion about etcd not being there.
A number of third parties had begun work on various (sometimes proprietary) orchestration and management systems for creating a reliable/scalable/easily manageable cluster with Docker as a building block. CoreOS is one. But Docker is pushing towards an official, open-source orchestration/management system that threatens to make all of those companies irrelevant.
I think Docker orchestration and coreos can coexist - if I had to use COREOS to use the goodness of Docker, then systemd-nspawn would come and eat Docker's lunch.
I wish that Docker bless one of Ansible/Chef as the official orchestration base and take it forward. I really don't want to earn something Docker specific.
Ansible/Chef orchestration IMHO solves a very different problem than container orchestration.
I agree that Chef and Ansible are different than container orchestration today - especially, when you look at low level stuff like networking, mounts, etc... But I guess what I was saying is that it is not hard to add these features to them.
They already have a specification format that works well and check for idempotency at its core. Unless you mean something like etcd is fundamental to container orchestration, which I don't believe it is (we run a couple of containers in production using Fig)
IME examining Docker, this is actually the hard problem.
What seemingly gets mixed up by quite a few commentators on this topic:
Docker is an orchestration, deployment, management, etc solution - the "container" is created by LXC, jails, libVirt or other OS features and now also libContainer.
This discussion also shows how far away / how early we are with "containerization" or container that are exchangeable / movable between different (OS) environment - we are discussing the companies that are building cranes to load and unload the boxes before we even have an understanding how the boxes will really look like.
I'm not really making a value judgement, just an observation.
It's too early to foretell the fate of Rocket. Containers are getting lots of attention, so I'm actually pretty happy to look at this as a potentially rewarding experiment. Worst case, it fails and we keep using Docker (or whatever else springs up).
Docker may or may not be the container engine that lasts a long time. There is a reason they raised a bunch of money. Clearly containers are going to be big, but is Docker the one that goes on to be dominant? Docker is trying through building features & biz dev, but it's far from over.
These aren't really containers. They're giant statically linked binaries, more or less. The actual operating system is now just a VM host for running containerized giant WIMPs (weakly interacting massive programs). Fast-forward a few years and the host can wither and die and be replaced with a proprietary or custom/fragmented management layer. Linux survives only as an internal pseudo-OS within each mega-binary "container."
Edit: what I was really getting at was that these technologies are patches for the inadequacy of the OS. The fact that we need containers at all stems from the difficulty of managing software installations, configuration, etc on the actual operating system.
Great post. However, containers are an easy concept to grasp. Even if the actual OS could be fixed, you'd still want some similar concepts even if the names were different.
http://en.wikipedia.org/wiki/TRON_project
Tron has been around since the mid 80s I believe and Linux was first released in the early 90s.
I think competition is good, this will give us an option that's not monolithic.
I didn't realize docker direction was to encompass orchestration until this thread. This isn't something I want to use docker for and also I'm glad the competition is address the security issue where there is a need for more security.
And with a rival option I'm happy to choose rocket as an option when it's stablized and there aren't any other options out there.
It's in a business's best interest, and exceedingly common practice, to "land and expand" with something clear and compelling, and following that add features to compete with alternative solutions. I don't think there's anything inherently altruistic about CoreOS that would keep Rocket lean in the long-run, especially as they begin migrating their various tools away from Docker containers.
It become pretty clear once dotCloud became Docker Inc. that they intended to capitalize on the "Docker" brand to sell an integrated orchestration platform. CoreOS already has enterprise customers for their operating system and related components. They seem like the perfect team to take this challenge on.
What features were recently introduced that it increased Docker's scope?
I hope you can understand that it's frustrating when, after hard work pitching an API to dozens of ecosystem players, spending weeks trying to wrangle a working implementation which makes as many of them as happy as possible, without compromising integrity of design - after all that, in the end, all it takes is one unhappy camper to write a blog post and that immediately tramples everything else.
It's even more discouraging in this particular case, because after this blog post, Alexis and I have discussed this topic extensively, and as a result he has since joined the effort. In fact I will be hacking with him in person on integrating Weave as a native networking plugin in 2 days in Amsterdam.
So, sorry for the insta-snark. But it can be frustrating to see so much good will and hard work be crushed in a second.
No malice, just a friendly tip :)
My problems with docker have been the security model, for which the only recourse I've had is to use the USER keyword in my Dockerfiles. Furthermore, networking has been a pain point, which I've had to resolve by using host networking to access interfaces.
Let's see how rocket deals with these issues and others. I pay for CoreOS support, so I'm glad to see that they're addressing this.
This is a design goal so that you can launch a container under the control of your init system or other process management system.
It looks like Rocket actually intends to be more conservative than Docker:
"Additionally, in the past few weeks Docker has demonstrated that it is on a path to include many facilities beyond basic container management, turning it into a complex platform. Our primary users have existing platforms that they want to integrate containers with. We need to fill the gap for companies that just want a way to securely and portably run a container."
So it's actually moving in the opposite direction, compared to Sandstorm.
(You of course know this already, but disclosure for others reading: I'm the lead dev of Sandstorm.)
Also, even though it is using systemd to monitor and fork processes a design goal is to run on all Linux's that have a modern Kernel.
One thing that I believe Docker has failed at is in taking a purely declarative approach to image definition; rather than specifying the packages that are assembled/inserted to create the container, Docker ships around non-portable Linux binaries.
[1] https://github.com/joyent/sdc-docker
[2] https://news.ycombinator.com/item?id=8682798
I can see where for SmartOS & Windows the Docker approach is more flexible. If someone has already settled on Linux, but they have their own ideas about how to manage containers within Linux that have nothing to do with CoreOS, the Rocket model is going to leave them much more flexibility.
Disclaimer: while I work on CF, I'm not that close to the Warden nitty-gritty.
It's great to see this problem broken up into reusable pieces though. It totally makes sense to function without a daemon, especially out of the box.
Wrong. With a server, the only thing an attacker has control over is its input. With a setuid-root binary, they still have control over its input, but they also have control over the entire environment under which it executes, including many things that developers generally assume an attacker can't control. Setuid binaries are incredibly scary from a security perspective and much harder to get right than servers.
So then, I guess docker could just run the two servers, one internal as root and one public as not? That's a pretty quick fix.
hm.. I don't think that's a given at all! There's been many issues with setuid-root programs. And I've seen that the OpenBSD guys favor privilege separation by breaking breaking up daemons into several parts that communicate using a very strict set of commands. For example a dockerd that does most of the work, but talks to another daemon (dockerd-root) when it needs to do anything privileged.
OpenSMTPD example: https://www.opensmtpd.org/presentations/asiabsdcon2013-smtpd...
OpenSSH: Initial efort, 2002: http://www.citi.umich.edu/u/provos/ssh/privsep.html
http://www.openbsd.org/papers/openssh-measures-asiabsdcon200... - Page 16 ->
Yep, this is _exactly_ one of our design goals. ACIs are trivially buildable and inspectable with standard Unix tools.
Dockerfiles/`docker build` is an implementation of a build system which uses the docker engine to make said rootfs.
Realistically, if the stack is broken into a dozen pieces then somebody will create a bundle with sensible defaults (let's call it "CoreOS") and then we'll be back in the same situation.
I'm looking into using containers for ui applications. I need to access GPU within the application. is this doable with Rocket or Docker?
Also does Rocket have to be used with CoreOS?
I previously heard that docker has trouble loading device drivers.
From the looks of your other comments in this tangent it might be exactly what you need or a starting point at least.
It's a base for these BOINC [2] and F@H [3] containers.
1: https://registry.hub.docker.com/u/ozzyjohnson/cuda/
2: https://registry.hub.docker.com/u/ozzyjohnson/boinc-gpu/
3: https://registry.hub.docker.com/u/ozzyjohnson/cuda-fah/
http://blog.docker.com/2014/12/initial-thoughts-on-the-rocke...
[1]: https://github.com/coreos/rocket/blob/9ae5a199cce878f35a3be4...
[2]: http://lwn.net/Articles/572957/
[1] https://github.com/docker/libcontainer