> Container hype is running high. Developers have flocked to the technology for more easily packaging applications and running them across various disparate environments.
It's a hype that should die in a fire bigger than the one reserved for Adobe Flash.
Many people will think they can just deploy a shitload of containers and be done... well, each and every single one of them has to be maintained, updated for security fixes...
Not to mention the likely possibility of breaking API changes in the container version. vagrant/puppet I'm looking at you, nothing is worse than having to figure out which ancient version of vagrant and puppet was used at creation in order to get the system running again. Or, well, look five years in the future, and hope today's hyped container solution will still be present then.
Fuck containers, get a properly managed ordinary Debian server and save yourself a lot of headaches.
I've tried so hard as an infrastructure person to explain this to people. It's better just to ignore the hype or move to sane orgs that aren't on the container hype train (so you're not wasting valuable time on tech that's going to get gutted eventually).
There is no magic tech that makes problems go away. Just abstractions (sometimes they work, sometimes they don't).
It's hard to take your criticism seriously when Google, Netflix, Heroku, Shopify, Paypal, Uber, Ebay, Yelp, Spotify don't qualify as "sane" orgs according to your criteria.
Meh large companies have large numbers of devs with varied opinions and power to apply those opinions. Lots of things we think of as shit now and have since dropped like a heavy coat has no doubt been implemented at one time by at least 1 of the companies you listed. It's whether they stand the test of time.
The number of man hours invested (wasted imo) into pointless layers of container stacks is mind blowing and I'm yet to see the return on investment for any of it in real world scenarios. The cynic in me sometimes thinks you get some new abstraction to beta and the set up your own con and get some time off work to talk about it, and having been a primary speaker at a conference gets your CV gravitas. Theres a con for every goddam library any Big Name releases before you can even properly evaluate it in a prod stack.
As as side point that doesn't entirely fit with my argument the most important lesson I took away from dev conferences when I still went to them is that devs wax lyrical about all kinds of things in public, but often the things they talk about are their pet projects or sidelined to some small part of the business that doesn't really affect the main workflow, yet because they work at a Big Name we attach importance to it.
All the companies you name have significant engineering manpower to dedicate whole teams to maintain all the tooling so that the devteams don't need to worry about maintaining their containers, VMs and whatnot.
I dare say that all that microservice stuff only works with 300+ employees if you keep everything containerized.
Where are these magically difficult to maintain containers? I regularly deploy stuff via containers and moving my github repo to a container to being deployed is very simple.
Containers turn app developers/deployers into distro maintainers. This is far outside the kind of thing they are likely to be competent at, and because they get their own enviroment very grevious problems can and do go unaddressed, because there is no apparent effect on other things.
I'm glad I'm not the only one. I'm trying to not be the old dog who can't learn a new trick, but the recent hype around massive containerization in production, and micro services, etc... just baffle me. As someone who's been doing application and infrastructure architecture for almost 20 years, for most application environments the cons seem to out-weigh the pros by a large margin...
> each and every single one of them has to be maintained, updated for security fixes...
No, of course not. No single container should have a lifetime greater than a day or two. "Maintenance" is deploying the latest, integration- and end-to-end-tested version of the software, as part of a continuous build pipeline.
So you depend on every application company to update their containers for every library vulnerability that emerges that they embed? Good luck with that.
Build your own images, problem solved. (Albeit creating a whole host of other problems in place of it, but those are no worse than a world without containers.)
No one relying on third party images beyond an OS can claim that they _really_ care about security or vulnerabilities anyway.
Glad someone here has half a fucking brain! All this shit about keeping them up to date. These are the morons that insist on running ssh in the container and run an init inside it too.
I've gotten so much utility out of docker, it feels absurd to read this comment. Containers made the sysadmin part significantly easier, whether it was testing new Node releases, trying out random databases or learning new languages. Without containers I would be spending a lot of time configuring servers/VMs and failing at that. So strange.
No matter the possible productivity gains, I won't use anything needing more than two hours just to get started (i.e. running a Hello World).
well that certainly helps frame your posts. In this case docker has a webpage that you can play around with. Maybe 10 minutes to get the basic idea.
I'm struggling to find out where your hostility comes from. It doesn't make your job disposable, which it sounds like where your fear comes from. Just because people can't come to you to get the ideal Debian server setup doesn't mean you are less valued.
> I'm struggling to find out where your hostility comes from.
I like to be in absolute control over my environment - and my time. If a tool author cannot manage to provide a Debian package with reasonably current and extensive documentation, I won't waste my time.
This thread has been stormed by people who work in the sys admin space and either don't care to learn the technology, have used it unsuccessfully while it was young, or see it as harmful to their career.
HN tech hivemind is often both conservative and arrogant: node is an aberration, atom is a travesty, docker is for the lazy, front end frameworks are overused, etc etc.
Very true. The fact is, even very intelligent people tend to stick with ideas that were popular as they 'grew' into their position.
Programming is young, but if you look at the history of more developed fields such as physics and mathematics, there is a very long history of extremely respected scientists out right rejecting new theories for being outlandish and ridiculous that we now accept as the bedrock of scientific theory.
The truth is, just as with physics and mathematics, in computer science new ideas do form, but older generations rarely adapt, and it's only with death of the old, and the growth of a new generation do ideas become accepted.
I don't think anybody is suggesting containers are bad for ephemeral exploration, development, the things you mention. The argument is what you use it for when it comes to production. For example if you're modeling machines with docker you're already using it wrong. Even docker considers that an anti-pattern.
I've invested some time to learn about best practices and have been using docker in production for many months with completely satisfactory results. I am not seeing a rational discussion here.
It's not so emotionally satisfying to talk about "how does The Thing help you concretely and what concrete difficulties have you run into," and way more fun to talk about either "it's all just hype, everyone's a stupid newbie" or "The Thing is total amazeballs and totally teh future you guys."
Me neither, all I see is an avalanche of FUD by what seems to be "old-timers" in this forum. Also mind the fact they haven't been down-voted. This tells me the people with down-voting abilities and this FUD-spreading old-timer group overlap greatly. The main commenters spreading FUD have karmas of ~1500, ~2500, ~12400, ~2900.
> Many people will think they can just deploy a shitload of containers and be done... well, each and every single one of them has to be maintained, updated for security fixes...
Do you apply the same kind of reasoning to virtual machines, and reach the conclusion that a single massive mainframe is the optimal configuration?
Something I think a lot of people struggle with, both on the Dev side and the Ops side, is to think of VMs and containers as completely disposable. Does it add value, or help you, for you to have an emotional attachment to this server?
Instead of doing OS updates, application updates, code updates, and configuration updates to 50 identical webservers that live behind load balancers, it might be faster to upgrade one. Then that one can be templated, cloned and deployed and replace the currently operating ones. There's still server name, IP address, and other configuration that has to be automated, and sometimes automating this is harder than automating all that patching, sometimes not.
This obviates server histories. True - you can't say "This VM has always performed better than the others" since you get a new farm of VMs every month. While many see that as a downside, it's actually a benefit that you don't need to think about that data.
Puppet/Chef and general admin automation scripts handle the updates/config changes/etc... very easily without the added complexity of container management/orchestration.
Ha! I worked at [major tech company] and our Puppet runs took about 3 hours. Developing manifests was surreal - punch card era code/compile/test cycles. The worst and most frequent production outages were seemingly innocuous Puppet changes that made it through code review. Our puppet manifest was in the 10kloc range, most of them generated.
Containerization is now letting individual teams manage app-specific configuration they actually understand and can test and iterate. A central org maintains the base image and can redeploy our containers with OS-level fixes as necessary. Our "self-service" deployment infrastructure was a behemoth that no one understood, documented by tribal knowledge. At less I can Google how to use the Docker API.
If your Puppet runs take three hours and range into the thousands of lines, that's a disastrously poorly built Puppet environment. Blame your people, not your tools.
(And I say this as no fan of Puppet; I'm a Chef user. Where I've never seen anything remotely that deranged, even in dysfunctional environments.)
Wow dude. Clearly you have not touched Docker. And if you have ever run a VM that is a container. Or I guess you are only using a mainframe? It's just a component like any component to any application. And there is plenty of tooling to help with updates and compliance, like CoreOS open source of Clair. Your point seems to come from another place because technically it's totally misguided.
DevOps guy who has the pain of working with Docker. I absolutely love getting to work with a product that breaks at point releases.
Oh, right. I'm supposed to have no life as a tech worker and spend countless nights dealing with container runtime issues in production because "It's The Wave Of The Future".
These are great point, but as someone who is beginning to think what I can do with containers for my current infrastructure setup, so far my impression is container is not to change the infrastructure, but rather how compute is used.
The meaning and the life cycle of container are no different from having a big virtual machine image, but container technology such as docker has been made blazing quick to start and kill, and I think that's a step forward. Containers in general allows multiple applications live in its own "pretty good" sandbox environment (read about Docker security, as there are many security implications most people don't pay attention to). This is great for truly stateless applications, or ETL applications, for which there is no need for a pool of machines dedicated to just host one specific application. This is not something we can do with big virtual machine unless now you want to invest your time to manage creating different users on the server, carefully writing down UNIX permission, as opposed to understand the security control around using containers (and linking containers). And many people don't pay attention to the size of the base image. Most people probably just use the vanilla Linux (ubuntu/redhat/centos, etc) image, but that kind of Docker image is too big for scaling and testing.
An example of ETL or long running program would be downloading huge files from some on-premise, and upload files to S3 in chunks. If your files are large, you should chunks to your compute. To keep track of progress of thousands of chunks, you need queue and worker nodes. now you've got a fleet of computes, and you want to run everything (worker and primary web applications handling the requests) in parallels, as quickly as possible, as stateless as possible on the compute side, you can leverage containers, plus the use of queues, and a RMDBS for transaction logs. Here container technology such as Docker is useful.
You are right, don't blindly follow the "everything should be containerized" hype, I don't think most people would dare to containerized Cassandra database and fit a big box with Cassandra database and web applications running in production.
Regardless of approach, when it comes to self-healing, highly automated, reproducible, and secure infrastructure:
* service discovery
* access control
* monitoring and insight analysis
* credential management
* version control and management (containers, security patches, container and image life cycle)
are problems we cannot solve with containers without integrating with third-party softwares.
Guess, what, there is no one solution solves these. They are just buzz words and goals we haven't achieved well as a community... even OSS platforms like CloudFoundry and Kubernetes are only able to do the application-side of life cycle to the point of easing deployment, not so much everything else.
I don't know much about CloudFoundry -- but with Kubernetes, are you saying that it's service discovery, access control, monitoring and insight analysis, credential management, version control and management are not addressed directly by Kubernetes, or perhaps, not mature yet? Just trying to get clarity on what you are saying here.
I can answer for CF. Service discovery, access control, monitoring, credentials and management are part of the core feature set -- because that's what distinguishes a platform from a single tool.
Version control properly lives outside the runtime environment, in my opinion; though these days CF has Gitlab integration points.
It's no longer building some servers that run and putting somebody else's code on it. Instead, it's working closely with the team producing code and helping build a mature deployment and management capability. It's about being a responsible expert on the bits outside of the application process.
Containers help immensely here because they recognise that a product is composed of more than just an application and they have lowered the barrier of entry to making the system of parts as a whole cooperate.
As part of this change there will of course be quality issues. Classically trained developers will produce poor quality container images, just as classically trained sysadmins produce poor quality apps.
This initial hump will pass as new tools are introduced to assist with this and as people learn the skills they need to produce higher quality images.
If you have the skills, invest your time in helping this process along rather that raging at the fact you work in an industry where change is disruptive and constant.
You may find containers to be useful in this process. With rare exceptions, I don't, and the way you write this post with the implication that containerization is an unmitigated good grinds my gears. Disruptive and constant change leads plenty often down real bad roads, and unthinking embrace of every fad that comes down the pike has led to many more disasters than conservative thought has.
I do find containers to be good although I'll admit I'm biased - I have several hundred thousand apps in production in containers, running in an efficient and scalable way that would not have been possible using prior technology.
My experience is that using containers effectivelly requires a substantial rethink of the infrastructure and how the teams using it work. The payoff though is a step change in productivity.
Over time, I believe that companies which embrace the change will deliver more quickly than those who do not. They will get products to market in a shorter timespan and generally be nicer places to work.
Will that mean the death of other ways of doing infrastructure? No probably not. After all, there are still mainframes and cobol programmers out there.
> well, each and every single one of them has to be maintained, updated for security fixes...
Then you patch it by redeploying it.
> Not to mention the likely possibility of breaking API changes in the container version. vagrant/puppet I'm looking at you, nothing is worse than having to figure out which ancient version of vagrant and puppet was used at creation in order to get the system running again
Is this a real issue that people run into? I've never had any issues with this.
> Or, well, look five years in the future, and hope today's hyped container solution will still be present then.
Good thing the real work essentially boils down to some scripts which can be easily translated into any new technology
> Fuck containers, get a properly managed ordinary Debian server and save yourself a lot of headaches.
> One of the novel things about the platform is that it is hardware agnostic, meaning that it can control physical bare metal machines, virtualized environments, or even public cloud infrastructure
Cloud Foundry already has this capability (runs on AWS, vSphere, OpenStack, Vagrant(!) and now Azure); Otto is not far behind.
Anything you can supply a Garden backend for, Cloud Foundry can use.
> Pricing has not yet been set, but it will be a subscription, freemium model; A beta version of the software will be available soon.
Buzzwords abound. How do they know a freemium model will be possible without losing money? Why will they put out a beta rather than starting with a "gold" version of the software?
Articles like these do not bode well for their subjects. They read like symptoms rather than anything that actually increases hype in my view.
How do they know a freemium model will be possible without losing money?
They don't, just like they also don't know if a non-freemium pricing model will be possible. They presumably wrote a business model that it'll be implemented and then reevaluated.
Why will they put out a beta rather than starting with a "gold" version of the software?
Probably because they want the benefits of having early feedback before investing too much work into it.
what i'd like to see is containers used for Steam games with slow loading times (or having to launch another launcher), i'd like to be able to hibernate a game and resume at a later time.
Containers are not usually suspendable like VMs. What you need is a process checkpointing tool like CRIU[1], which can suspend regular processes to the disk.
I'm still not sold on Docker. The shift towards microservices makes it redundant.
Docker is a tool which was built for platform-as-a-service/DevOps people who had to manage complex systems made up of many different technologies which those DevOps people didn't want to think about. It provides a relatively clean encapsulation of complex logic.
With microservices, the idea is to break up your complex system into independent services which each perform a specific task. I think an implied goal of microservices is that each individual service should be made up of relatively few technologies - If each microservice is made up of few technologies, it's really easy and fast to just git pull the relevant changes (or update from your package manager) and relaunch your service.
I would like it if someone could explain it to me why I need to pull something from DockerHub instead of GitHub (assuming that you have a simple microservice made up of a single, cohesive technology stack which you actually understand and builds relatively quickly)?
58 comments
[ 2.4 ms ] story [ 107 ms ] threadIt's a hype that should die in a fire bigger than the one reserved for Adobe Flash.
Many people will think they can just deploy a shitload of containers and be done... well, each and every single one of them has to be maintained, updated for security fixes...
Not to mention the likely possibility of breaking API changes in the container version. vagrant/puppet I'm looking at you, nothing is worse than having to figure out which ancient version of vagrant and puppet was used at creation in order to get the system running again. Or, well, look five years in the future, and hope today's hyped container solution will still be present then.
Fuck containers, get a properly managed ordinary Debian server and save yourself a lot of headaches.
There is no magic tech that makes problems go away. Just abstractions (sometimes they work, sometimes they don't).
The number of man hours invested (wasted imo) into pointless layers of container stacks is mind blowing and I'm yet to see the return on investment for any of it in real world scenarios. The cynic in me sometimes thinks you get some new abstraction to beta and the set up your own con and get some time off work to talk about it, and having been a primary speaker at a conference gets your CV gravitas. Theres a con for every goddam library any Big Name releases before you can even properly evaluate it in a prod stack.
As as side point that doesn't entirely fit with my argument the most important lesson I took away from dev conferences when I still went to them is that devs wax lyrical about all kinds of things in public, but often the things they talk about are their pet projects or sidelined to some small part of the business that doesn't really affect the main workflow, yet because they work at a Big Name we attach importance to it.
I dare say that all that microservice stuff only works with 300+ employees if you keep everything containerized.
Until of course, everything explodes.
No, of course not. No single container should have a lifetime greater than a day or two. "Maintenance" is deploying the latest, integration- and end-to-end-tested version of the software, as part of a continuous build pipeline.
Build your own images, problem solved. (Albeit creating a whole host of other problems in place of it, but those are no worse than a world without containers.)
No one relying on third party images beyond an OS can claim that they _really_ care about security or vulnerabilities anyway.
In this case the application containers themselves can perfectly live less than a day.
If it turns out during evaluation that setting up a simple Debian VM with the requirements of the tool, the tool ends up on the "black list" for me.
No matter the possible productivity gains, I won't use anything needing more than two hours just to get started (i.e. running a Hello World).
well that certainly helps frame your posts. In this case docker has a webpage that you can play around with. Maybe 10 minutes to get the basic idea.
I'm struggling to find out where your hostility comes from. It doesn't make your job disposable, which it sounds like where your fear comes from. Just because people can't come to you to get the ideal Debian server setup doesn't mean you are less valued.
I like to be in absolute control over my environment - and my time. If a tool author cannot manage to provide a Debian package with reasonably current and extensive documentation, I won't waste my time.
Programming is young, but if you look at the history of more developed fields such as physics and mathematics, there is a very long history of extremely respected scientists out right rejecting new theories for being outlandish and ridiculous that we now accept as the bedrock of scientific theory.
The truth is, just as with physics and mathematics, in computer science new ideas do form, but older generations rarely adapt, and it's only with death of the old, and the growth of a new generation do ideas become accepted.
Do you apply the same kind of reasoning to virtual machines, and reach the conclusion that a single massive mainframe is the optimal configuration?
Instead of doing OS updates, application updates, code updates, and configuration updates to 50 identical webservers that live behind load balancers, it might be faster to upgrade one. Then that one can be templated, cloned and deployed and replace the currently operating ones. There's still server name, IP address, and other configuration that has to be automated, and sometimes automating this is harder than automating all that patching, sometimes not.
This obviates server histories. True - you can't say "This VM has always performed better than the others" since you get a new farm of VMs every month. While many see that as a downside, it's actually a benefit that you don't need to think about that data.
Containerization is now letting individual teams manage app-specific configuration they actually understand and can test and iterate. A central org maintains the base image and can redeploy our containers with OS-level fixes as necessary. Our "self-service" deployment infrastructure was a behemoth that no one understood, documented by tribal knowledge. At less I can Google how to use the Docker API.
(And I say this as no fan of Puppet; I'm a Chef user. Where I've never seen anything remotely that deranged, even in dysfunctional environments.)
Oh, right. I'm supposed to have no life as a tech worker and spend countless nights dealing with container runtime issues in production because "It's The Wave Of The Future".
The meaning and the life cycle of container are no different from having a big virtual machine image, but container technology such as docker has been made blazing quick to start and kill, and I think that's a step forward. Containers in general allows multiple applications live in its own "pretty good" sandbox environment (read about Docker security, as there are many security implications most people don't pay attention to). This is great for truly stateless applications, or ETL applications, for which there is no need for a pool of machines dedicated to just host one specific application. This is not something we can do with big virtual machine unless now you want to invest your time to manage creating different users on the server, carefully writing down UNIX permission, as opposed to understand the security control around using containers (and linking containers). And many people don't pay attention to the size of the base image. Most people probably just use the vanilla Linux (ubuntu/redhat/centos, etc) image, but that kind of Docker image is too big for scaling and testing.
An example of ETL or long running program would be downloading huge files from some on-premise, and upload files to S3 in chunks. If your files are large, you should chunks to your compute. To keep track of progress of thousands of chunks, you need queue and worker nodes. now you've got a fleet of computes, and you want to run everything (worker and primary web applications handling the requests) in parallels, as quickly as possible, as stateless as possible on the compute side, you can leverage containers, plus the use of queues, and a RMDBS for transaction logs. Here container technology such as Docker is useful.
You are right, don't blindly follow the "everything should be containerized" hype, I don't think most people would dare to containerized Cassandra database and fit a big box with Cassandra database and web applications running in production.
Regardless of approach, when it comes to self-healing, highly automated, reproducible, and secure infrastructure:
* service discovery
* access control
* monitoring and insight analysis
* credential management
* version control and management (containers, security patches, container and image life cycle)
are problems we cannot solve with containers without integrating with third-party softwares.
Guess, what, there is no one solution solves these. They are just buzz words and goals we haven't achieved well as a community... even OSS platforms like CloudFoundry and Kubernetes are only able to do the application-side of life cycle to the point of easing deployment, not so much everything else.
Version control properly lives outside the runtime environment, in my opinion; though these days CF has Gitlab integration points.
It's no longer building some servers that run and putting somebody else's code on it. Instead, it's working closely with the team producing code and helping build a mature deployment and management capability. It's about being a responsible expert on the bits outside of the application process.
Containers help immensely here because they recognise that a product is composed of more than just an application and they have lowered the barrier of entry to making the system of parts as a whole cooperate.
As part of this change there will of course be quality issues. Classically trained developers will produce poor quality container images, just as classically trained sysadmins produce poor quality apps.
This initial hump will pass as new tools are introduced to assist with this and as people learn the skills they need to produce higher quality images.
If you have the skills, invest your time in helping this process along rather that raging at the fact you work in an industry where change is disruptive and constant.
You may find containers to be useful in this process. With rare exceptions, I don't, and the way you write this post with the implication that containerization is an unmitigated good grinds my gears. Disruptive and constant change leads plenty often down real bad roads, and unthinking embrace of every fad that comes down the pike has led to many more disasters than conservative thought has.
My experience is that using containers effectivelly requires a substantial rethink of the infrastructure and how the teams using it work. The payoff though is a step change in productivity.
Over time, I believe that companies which embrace the change will deliver more quickly than those who do not. They will get products to market in a shorter timespan and generally be nicer places to work.
Will that mean the death of other ways of doing infrastructure? No probably not. After all, there are still mainframes and cobol programmers out there.
Then you patch it by redeploying it.
> Not to mention the likely possibility of breaking API changes in the container version. vagrant/puppet I'm looking at you, nothing is worse than having to figure out which ancient version of vagrant and puppet was used at creation in order to get the system running again
Is this a real issue that people run into? I've never had any issues with this.
> Or, well, look five years in the future, and hope today's hyped container solution will still be present then.
Good thing the real work essentially boils down to some scripts which can be easily translated into any new technology
> Fuck containers, get a properly managed ordinary Debian server and save yourself a lot of headaches.
Spotted the Debian sys admin!
Cloud Foundry already has this capability (runs on AWS, vSphere, OpenStack, Vagrant(!) and now Azure); Otto is not far behind.
Anything you can supply a Garden backend for, Cloud Foundry can use.
Buzzwords abound. How do they know a freemium model will be possible without losing money? Why will they put out a beta rather than starting with a "gold" version of the software?
Articles like these do not bode well for their subjects. They read like symptoms rather than anything that actually increases hype in my view.
They don't, just like they also don't know if a non-freemium pricing model will be possible. They presumably wrote a business model that it'll be implemented and then reevaluated.
Why will they put out a beta rather than starting with a "gold" version of the software?
Probably because they want the benefits of having early feedback before investing too much work into it.
[1] http://criu.org/
Docker is a tool which was built for platform-as-a-service/DevOps people who had to manage complex systems made up of many different technologies which those DevOps people didn't want to think about. It provides a relatively clean encapsulation of complex logic.
With microservices, the idea is to break up your complex system into independent services which each perform a specific task. I think an implied goal of microservices is that each individual service should be made up of relatively few technologies - If each microservice is made up of few technologies, it's really easy and fast to just git pull the relevant changes (or update from your package manager) and relaunch your service.
I would like it if someone could explain it to me why I need to pull something from DockerHub instead of GitHub (assuming that you have a simple microservice made up of a single, cohesive technology stack which you actually understand and builds relatively quickly)?