That's actually a very helpful piece of information, as I hadn't noticed the tiny RedHat line at the bottom--thank you! But it still leaves open the original question about advantages/disadvantages vs. CoreOS.
If by 'obtuse' you mean 'clear to anyone who has used iptables', sure. Sure, IPv6 would be nice, and it's probably not hard to add support for. Portability? You're talking about a Fedora-based Linux distribution and complaining that it might not be as portable as CoreOS, a completely custom distribution, running a Linux-only container system?! And "daemon-centric architecture", what the hell is that? Anything that sits in the background is a daemon. Considering it supports SELinux out of the box, and geard supports HTTP instead of unix socket controls, root isn't necessary or a safety concern. And what exactly is the problem with SSH access?
By obtuse I meant arguing that local port forwarding was somehow adding something elegant here. Arbitrary protocols would be ideal. I didn't mention portability to where, but we agree there's a clear limitation. By daemon-centric architecture I mean that state is maintained and exchanged between the guest and host rather than the host inflicting its version of truth upon the guest (shut down, start again with new settings), which is arguably cleaner for disposable service containers (even those requiring persistent storage). Linux has supported SELinux out of the box for 15 years, but it doesn't mean that it is easy to configure or gets used properly. Manipulating live services via SSH is an anti-pattern because it removes repeatability/predictability by causing undocumented state to accrue in guests, ultimately resulting in wasted time chasing down bugs that someone introduced by accident. To put this another way - the whole attraction of containers is disposable environments: SSH in its conventional systems administration utilization is their antithesis.
SSH is an encrypted protocol for communication with remote hosts. You don't have to allow the use of 'bash', but can still take advantage of all the networking, encryption, authentication, etc capabilities afforded by SSH. Nobody's stopping you from scp'ing files and then calling puppet using ssh. Seeing as virtually all modern network appliances support using SSH for transferring configs as well as a maintenance and rescue channel, it's probably not harmful to allow its use in a virtual container, and each org can decide how it manages its use.
I for one don't really care much about root vs non-root in terms of host security. Even if you patch your kernel, without using grsec/SELinux/etc somebody'll just find another priv escalation hole next year and own your host with an unprivd user. If you actually focus on securing the host and applications, userland exploits have the same effect: little.
I don't think there's such a thing as a disposable service container. If it's a multi-user service it's got certain intrinsic properties you can't avoid. You have to be able to configure it, run it, use it, maintain it, troubleshoot/debug it, monitor it, etc. Who's "inflicting truth" doesn't matter as long as you can do all of the normal administration tasks required of multi-user network services. This, by the way, is where SSH becomes incredibly useful. Configuration management is not a panacea.
Things like Docker already do weird custom things to your service very similar to iptables-mangling rules, and nobody seems to have complained about it yet. It was only recently that they stopped mangling stock binaries like init in your image!
I think you missed, misunderstood or disagree with the phrase in its conventional systems administration utilization with regards to ssh's evilness in production.
I think its one of the latter two because of your third paragraph there, where you slam the model under proposal (the disposable service container) as a nonexistant dream. Well, let me tell you, it exists and it works. I think most large companies use something similar (Google, etc.) because scaling any other way is too expensive.
Disposable service containers are 100% predictable copies of well tested system images that never, ever get logged in to or altered... only started, communicated with, or destroyed in very well defined ways. They are not allowed to accrue undocumented state, because there be dragons.
Is dynamically-allocated memory considered documented state? How about temp files? Network connections to new peers? Consensus changes in the network? What defines documentation, or state?
When you say "100% predictable", I think you might be discounting the fact that most services change as they run, and while they may have a finite set of operations, those operations' outcomes vary depending on input. Determinism does not work so well in network services. On top of that, most services are sufficiently complex that there does not exist enough tools to support them in a completely automated way.
Or, maybe I should modify my original statements to say that disposable services do exist, but are only useful short-lived with very specific use cases. I don't see the point in handicapping yourself for some theoretical ideal of predicable operations. Even with unpredictable services in an undocumented state, they can be managed to work quite well in the real world and happen to run gigantic pieces of infrastructure without issue.
I don't find containers to be the cheapest way to scale. The old model of just running a bunch of random apps on a single host is really cheap scaling. Containers just reduce the size and scope of the host and take process-level resource management to the host level, which has been done decades ago in the OS, but it didn't really catch on too well.
In order to dynamically adjust the load on your JBOA (Just a Bunch of Applications) platform to make it as efficient as possible, you adjust the distribution of requests to your services, versus containers where you adjust the distribution of your containers. Of course it depends on whether your app stack/services is designed properly, and since they seldom are it's not always very efficient in practice. What you consider cheap will probably change based on what kind of crap you have to manage.
In any case, how you use SSH is totally up to you, and it does not have to result in evilness or undocumented state. Just don't use it wrongly.
CoreOS uses a "dual root" setup to stage upgrades. Atomic uses rpm-ostree to stage upgrades. Both require reboots and the bootloader to roll back changes.
Both will let you run Docker containers.
CoreOS doesn't seem to support SELinux natively, but Atomic does. (Probably related to the fact that Atomic is a normal-ish linux distro and CoreOS is totally custom)
Atomic gives you a web interface to manage running services. I am not aware of a similar tool [Cockpit] on CoreOS.
Atomic uses geard as an interface to installing Docker containers into systemd and running them. It's a part of OpenShift. It seems to be abstraction that you can use to control more parts of the "orchestration" of large systems, plugging in zookeeper/etcd/serf for the backend configs. Seems like they have a strong focus on the real-world security of their application clusters. https://github.com/openshift/geard/blob/master/docs/orchestr...
Unfortunately, what it works at isnt presenting complex technical information in a way that allows engineers to concretely evaluate the implementation and claims.
Quite the opposite: bereft implementations and retread ideas are pasted over by fancy marketing.
The rise of buzzword marketing shine has been bad for our industry.
15 comments
[ 3.1 ms ] story [ 32.0 ms ] threadHow many of these also hold for CoreOS? Probably less, but definitely some.
I for one don't really care much about root vs non-root in terms of host security. Even if you patch your kernel, without using grsec/SELinux/etc somebody'll just find another priv escalation hole next year and own your host with an unprivd user. If you actually focus on securing the host and applications, userland exploits have the same effect: little.
I don't think there's such a thing as a disposable service container. If it's a multi-user service it's got certain intrinsic properties you can't avoid. You have to be able to configure it, run it, use it, maintain it, troubleshoot/debug it, monitor it, etc. Who's "inflicting truth" doesn't matter as long as you can do all of the normal administration tasks required of multi-user network services. This, by the way, is where SSH becomes incredibly useful. Configuration management is not a panacea.
Things like Docker already do weird custom things to your service very similar to iptables-mangling rules, and nobody seems to have complained about it yet. It was only recently that they stopped mangling stock binaries like init in your image!
I think its one of the latter two because of your third paragraph there, where you slam the model under proposal (the disposable service container) as a nonexistant dream. Well, let me tell you, it exists and it works. I think most large companies use something similar (Google, etc.) because scaling any other way is too expensive.
Disposable service containers are 100% predictable copies of well tested system images that never, ever get logged in to or altered... only started, communicated with, or destroyed in very well defined ways. They are not allowed to accrue undocumented state, because there be dragons.
When you say "100% predictable", I think you might be discounting the fact that most services change as they run, and while they may have a finite set of operations, those operations' outcomes vary depending on input. Determinism does not work so well in network services. On top of that, most services are sufficiently complex that there does not exist enough tools to support them in a completely automated way.
Or, maybe I should modify my original statements to say that disposable services do exist, but are only useful short-lived with very specific use cases. I don't see the point in handicapping yourself for some theoretical ideal of predicable operations. Even with unpredictable services in an undocumented state, they can be managed to work quite well in the real world and happen to run gigantic pieces of infrastructure without issue.
I don't find containers to be the cheapest way to scale. The old model of just running a bunch of random apps on a single host is really cheap scaling. Containers just reduce the size and scope of the host and take process-level resource management to the host level, which has been done decades ago in the OS, but it didn't really catch on too well.
In order to dynamically adjust the load on your JBOA (Just a Bunch of Applications) platform to make it as efficient as possible, you adjust the distribution of requests to your services, versus containers where you adjust the distribution of your containers. Of course it depends on whether your app stack/services is designed properly, and since they seldom are it's not always very efficient in practice. What you consider cheap will probably change based on what kind of crap you have to manage.
In any case, how you use SSH is totally up to you, and it does not have to result in evilness or undocumented state. Just don't use it wrongly.
Both will let you run Docker containers.
CoreOS doesn't seem to support SELinux natively, but Atomic does. (Probably related to the fact that Atomic is a normal-ish linux distro and CoreOS is totally custom)
Atomic gives you a web interface to manage running services. I am not aware of a similar tool [Cockpit] on CoreOS.
Atomic uses geard as an interface to installing Docker containers into systemd and running them. It's a part of OpenShift. It seems to be abstraction that you can use to control more parts of the "orchestration" of large systems, plugging in zookeeper/etcd/serf for the backend configs. Seems like they have a strong focus on the real-world security of their application clusters. https://github.com/openshift/geard/blob/master/docs/orchestr...
Project page: https://wiki.gnome.org/Projects/OSTree
Relevant HN discussion: https://news.ycombinator.com/item?id=6277518
Quite the opposite: bereft implementations and retread ideas are pasted over by fancy marketing.
The rise of buzzword marketing shine has been bad for our industry.