HN Explorer

Ask HN: As a web developer, do I have to know what Docker and Kubernetes are?

25 points by dprophecyguy ↗ HN
Hey, everyone, i am a javascript web developer, I have worked on both Node and Angular (Back and Front) but right now on Front. I know docker and kubernets are all the rage but i was busy learning Node and Angular i had one and a half year of experience still getting better in both. So what i want to know is Why do we need Docker and Kubernetes as in simple term (Explain like i am 5) and i am also working on my side project so whether knowledge of them will help me to make a good decision regarding my projects. SORRY FOR BAD ENGLISH

47 comments

[ 1.7 ms ] story [ 105 ms ] thread
sd8dgf8ds8g8dsg
Using docker can greatly improve your development situation. For example, you can use docker to define the full environment needed to run your web app, meaning you can have another dev running it with "docker run", and not needing to install any local dependencies, as everything (web server, node etc) is contained inside the docker container. This turns out also be useful for your future self, when you in 2 years from now are having troubles re-creating the environment needed for running your service.

Docker orchestrate can further improve your development situation, in which you can use it to define and test out multiple virtual servers on your laptop, such as the database, the file server and the web server.

stephenr
Quite honestly Vagrant will do all of that for development.
sumedh
Setting up containers is ridiculously fast compared to VMs.
stephenr
What do you mean by "setting up". Are you talking about the work to provision a working target environment (i.e. what the VM/Container runs) or are you talking about the literal time for the environment to start?
viraptor
Containers help with both. No full-system boot delay helps. Cached layers help. Not dealing with vm-specific kernel modules helps. Fast non-nfs volumes shared with the host help.

There are speed benefits for containers all over the place - both in time spent dealing with the environment and in actual execution.

stephenr
> No full-system boot delay helps

I just cold-booted a Vagrant VM, which is configured to check for new base-box versions (i.e. a network delay that can be disabled).

    time vagrant up
    ...
    real    0m39.698s
    user    0m11.989s
    sys     0m5.490s
If your argument about tooling is around performance and the "slow" one takes just 40 seconds to start, I think you need to clarify how often you expect to be starting this tool in a day.

> Cached layers help.

Helps what, exactly?

> Not dealing with vm-specific kernel modules helps.

I don't even know what you're talking about here. Do you mean you need to have a hypervisor kernel module? What exactly do you need to "deal with"?

> Fast non-nfs volumes shared with the host help.

I've never once used an NFS volume in a Vagrant guest. In case you weren't aware, Vagrant is not limited to using VirtualBox.

> There are speed benefits for containers all over the place - both in time spent dealing with the environment and in actual execution.

You haven't actually identify any environment issues you need to "deal with", just said "X helps".

viraptor
Some context then. The happy path startup cost (environment is already prepared) matters to devs who never deal with the environment itself. If you're dealing with frontend only - maybe that's you most of the time, and you only see the 40s maybe once a day.

But there are other costs. Someone deals with updates, with adding new functionality, with debugging issues. I spend most of my day recently waiting for puppet runs in vagrant/vbox. It's 10+ min for a full run. Then I make more changes - sometimes an incremental provision is enough, sometimes I need a full rebuild. On the other hand, if the environment was built with docker layers, I'd have much less waiting to do, because the caching of partial build/layer is integrated. Over a whole day, that's not insignificant time saving.

And sure, vagrant has multiple backend. I'm using vbox, and so does almost every developer using vagrant. It's the only free one. (Apart from the docker backend)

stephenr
That is literally what the concept of base boxes is for.

Build your own base box with whatever prerequisites you have baked in. It's still a 10 minute puppet run if that's how long your provisioning takes, but that happens once, and then you distribute the box. While you can just use a base distro box, for large/complex setups it's not always the best approach.

sumedh
How long does it take to setup postgres 9.4 with test datbase and one user for that DB in vagrant?

What is the command/process to do that?

stephenr
What's your config management tool of choice? I assume they will have recipes/etc for basic tasks like that.

Personally I'd just put the call to `apt-get` and then `psql` into a shell script.

Either way, set the provisioner. Either do it in vagrant provisioning, or if that takes to long, build a base box with Packer and put the provisioner there.

Edit: or for a simpler use, look on Vagrantcloud.com and find a base box that has what you want, and run that.

turtlebits
For docker w/ docker-compose, I can get a postgres container in <2 seconds with this definition

  version: "2"
  services:
    database:
      image: postgres
      environment:
        - POSTGRES_PASSWORD=123456
      ports:
        - "5432:5432"
docker-compose up -d 0.59s user 0.14s system 50% cpu 1.462 total

And restarting it takes < 1 second.

sumedh
Since you did not give a simple one line command I am going to assume its difficult to do that in vagrant.

You dont need to mess with a config management tool when you have docker.

stephenr
If a box provides Postgres it's just `vagrant init vendor/boxname && vagrant up`.

That's what you're relying on with a '1 line command' in docker to give you Postgres: an image that someone else has created.

Whether a Postgres-only box exists in Vagrantcloud is irrelevant, unless your argument is purely on the merits of the pre-made boxes/images third parties have contributed.

sumedh
You did not answer how long does it take to do that.
stephenr
Most boxes start in ~30s on reasonable hardware. I don't know how long the box will take to download because I don't know what box you might use, what vagrant provider (e.g. parallels boxes are usually a little bigger than vbox ones), what your internet connection speed is.

If your entire decision making process is "I only run things that take two seconds" then I don't think we can have a conversation because honestly I don't even think about the time vagrant takes to start.

My computer does multiple things all at once. It takes longer for my IDE to start and be usable than it does for Vagrant, and funnily enough computers can run multiple processes at once, so this whole "omg 30 seconds is too long to wait" argument doesn't fly for me.

sumedh
The setup time is one of the argument, in docker the whole thing takes couple of seconds, you might not appreciate the quick boot up time but lot of others care about that. If you are happy with slow boot up time then you should not migrate to docker but if you want something quick and easy then consider docker.
stephenr
> in docker the whole thing takes couple of seconds,

Once you have a running Linux Kernel. I don't use a Linux host OS, so I would still have a delay starting a VM to run Linux, and running a Hypervisor to run containers for a dev environment is too many levels of abstraction for my liking.

> you might not appreciate the quick boot up time but lot of others care about that. If you are happy with slow boot up

Is Docker quicker? Sure. Does that make Vagrant slow? No. Slower does not mean slow.

> but if you want something quick and easy then consider docker.

I prefer things I actually have control over rather than just "quick and easy".

sumedh
> Is Docker quicker? Sure.

I rest my case, feel free to yell at kids to get off your lawn.

> Slower does not mean slow.

Whatever makes you happy. If you are happy with vagrant stick to it, nobody is forcing you to use something better.

viraptor
The idea of the base boxes is there, sure. But the ability to configure repositories, make base box of that, install keys, make base box of that, install required told, make base box of that, ... etc. with the mechanism for transparent reuse of the latest cached box you care about is not available on vagrant. And when you're making changes and debugging, it makes all the difference. If vagrant integrated a nice way to do that, the time saving would be amazing.
stephenr
You kind of can do that, but it's not a Vagrant feature, so much as a Packer feature.

Most of the Packer builders (e.g. virtualbox, parallels, VMware) support building from scratch (i.e. start with ISO install media and a blank VM disk) OR building from an existing VM.

So you could have a multi-stage Packer build pipeline that iterates step by step.

But I'm still not really sure what your workflow is. My company builds a series of Debian (and for a client, Ubuntu) base boxes. When debugging issues, 99% of time required is checking/fixing inside a running box. Once a solution is found, applying that back to the base box provisioning scripts (or preseed or even the packer config) and testing a build doesn't take that long.

mickeyp
Yes, with the caveat that if you already use a hypervisor for your main dev environment then running another inside it with Vagrant's default VBox setup is likely to either be very slow or not work at all.

Docker will also layer and cache each command greatly speeding up the "build&test" cycle of writing and running Docker containers. With Vagrant you're re-running everything every time.

Furthermore, you can actually use Docker with Vagrant also nowadays.

stephenr
This sounds like a rare scenario, much more common these days I believe would be a non-Linux host, with a Linux "guest" server environment.

If you want to use Docker, you're then automatically using a hypervisor anyway.

> With Vagrant you're re-running everything every time.

Are you talking about the time to provision a non-created environment?

On a clean install, sure, that might take a while if your base-box doesn't have the dependencies you require - but most developers wouldn't often need to re-build their Vagrant environment for a project from scratch (i.e. vagrant destroy is not likely to be a common task). And that whole situation is moot if you use or build a more appropriate base box.

mickeyp
Is it that uncommon? Mac users using a hypervisor? Windows using running Linux in a Virtualbox? That's pretty common in corporate environments...

I am talking about rebuilding the Vagrant image every time you change it - as you are wont to do during the initial step of creating it. Sure, once it's written (by someone who has to spend a fair amount of elapsed time doing trial-and-error tweaks) then you don't have to re-cycle the image. On the other hand, that too can result in people using stale images over time as the configuration drifts.

stephenr
> Mac users using a hypervisor? Windows using running Linux in a Virtualbox?

Vagrant works natively on Macs and Windows, unlike Docker. There is zero reason to run it inside an already virtualised environment on those machines.

> I am talking about rebuilding the Vagrant image every time you change it

You know you don't need to destroy the vagrant environment to re-run the provisioning scripts, if something changes.. right?

> Sure, once it's written (by someone who has to spend a fair amount of elapsed time doing trial-and-error tweaks)

With the minor exception of a few Vagrant specific optimisations (a `vagrant` user with the predefined SSH key & passwordless sudo, for example) during base-box building, literally nothing in a Vagrant provisioning environment needs to be specific to Vagrant.

Also - if you're building base-boxes, it's probably worthwhile forking an existing project that targets your Distro/OS of choice and customising from there.

> On the other hand, that too can result in people using stale images over time as the configuration drifts.

Why? If it's a small change, just set the provisioner to `run: "always"` and it will run on every up. Assuming the changes (i.e. new/updated deps, or config files being copied) are small, it'll be almost instantaneous.

From everything you've said, I'm very curious how you use Vagrant (if at all). It isn't intended to be used like Docker, where you package up an entire image completely ready to use with 0 cycles needed to configure anything, and an update means a new image.

sixhobbits
At the risk of displeasing the Docker crowd, I would say you can get pretty far without them.

In general, I see a lot of people learning and using BuzzWord solutions for problems that they don't have.

In most cases, you'll have some problem and look for a solution to it. If your problem is spending a lot of time messing around with dependencies and deployments and "but it works on my machine" type issues, then Docker may well be the solution.

If you learn a solution and then see what problems it can solve (Hey, everyone is talking about Docker. Let me learn it too), you'll end up frustrated and confused.

Of course, it's a balance -- often you can learn about problems that you didn't even know you had by learning about new technology. But so far I see more people erring towards "Let's fix these problems I don't have" than "I'll just keep doing things this slow old way because I don't want to keep up with tech movements"

mickeyp
Having worked for for about 18-20 different clients I can count on my left hand the number of places that had turn-key, repeatable dev environments. Repeatable prod environments are usually better but not always.

Quite honestly to most lazy developers the idea of repeatable and turn-key environments is anathema to them. "It's time consuming". "It's boring." "It's Somebody Else's Job." It is now my number one way to determine actual seniority and skill level in a developer during interviews.

Docker is not the panacea, but it's a game changer for many people as it lowers the barrier to entry for these things. We should embrace containers for what they are -- a very useful tool.

stephenr
> turn-key, repeatable dev environments

> We should embrace containers for what they are

Um.. relatively unrelated to the problem you described?

Containers are about isolating one or more processes, memory, network, disk etc from the rest of the host system. This can be used to achieve security benefits, and/or to resolve dependency differences.

The problem of repeatable dev environments is solved almost always by a single concept: script it, either using shell scripts, or a configuration tool like Chef, Puppet, etc.

In Vagrant, this means one or more provisioning scripts executed in the guest environment (usually a VM, might be LXC or Docker). In Packer, this peans one or more provisioning scripts executed in the guest environment (usually a VM, might be LXC or Docker). In Docker, surprise fucking surprise, it's a script executed in the guest environment.

michaelmior
It's fairly common to use Docker to run other services (database, cache, etc.) needed when developing a web app.
stephenr
I assume you're talking about running multiple images concurrently, each one focussed on it's own specific role.

Sure, you can do that too with Vagrant. Vagrant isn't limited to running a single machine for a given Vagrantfile.

mickeyp
No, they are not unrelated. Isolation yes, and through that, you get repeatability. We can all agree that declaratively describing your environment (Salt, Chef) is preferable to the rather simplistic Dockerfiles currently in use, but both aim to accomplish the same time. I remember the pre-container days of re-running Salt/Puppet scripts over and over again in a VM until it works. Nothing wrong with that - they're great tools. I also remember the pre-Salt/Puppet days of using shell scripts. None of the above are great: conf. management + docker is a better solution long-term for building a container image.
stephenr
My point is that Docker itself brings nothing to that 'repeatability' table. You have that with shell scripts, salt/chef/puppet/what have you, all of which can be run on any environment that meets the baseline requirements for your project.
flukus
On what technology? Working with .net + nuget the environment is generally pretty repeatable, the only big outside dependencies to worry about are the .net framework itself and sql server. Others like java and rust are pretty similar AFAICT.
stephenr
> I see a lot of people learning and using BuzzWord solutions for problems that they don't have.

THIS more than anything is the biggest issue I have when dealing with clients.

"Hey we need your expertise, this is our problem and we want to use <Docker/Kubernetes/AWS/MongoDB/<$NAME YOU COOL KID FLAVOUR OF THE MONTH>".

Thankfully, most people will listen to reason when you point out how $X won't actually solve their problem but $Y (which they've likely never heard of or dismissed because it's not being hyped by VC funded startups trying to defend their own poor choices) could.

sumedh
You can worry about Kubernetes later for now just understand docker basics. Docker really is a game changer.

Want to try something in Postgres 9.4, you can do it with just one command, want Rabbit MQ with web stomp enabled, get it with just one command.

Setting up Environments with Docker is ridiculously easy, Docker files are pretty intuitive and easy to understand.

mankash666
Docker & kubernetes are modern Dev ops tools - the former abstracts away the host specific quirks, and the latter allows managing many (docker) containers in production.

Should you learn this - it depends. If you plan on building microservices whose devops you'll need to manage, docker is highly recommended. You can get away not learning kubernetes by relying on AWS ECS equivalents, and that's probably what you should do as a beginner wanting to deploy.

If your app architecture is event-driven, and can be rest based AWS lambda is a great option where you don't have to learn either. Beware of costs if invoking in the millions, but if your service ever scales to that, you've probably already found a lucrative opportunity that can afford experienced help

mickeyp
Sorry, but you don't need to be building microservices to even consider using tools like Docker and Kubernetes.

The former (or containers in general) is an excellent way to build repeatable and re-usable environments for development, then your test and CI environment(s), and then finally production, as you -- in a greenfield project, for instance -- crystallise your requirements as you go from the trappings of a blank project to a final one.

To say that either tool is aimed at microservices only is very misleading.

I will also take some exception to the notion that using Lambda with event sourcing means you don't have to learn how Lambda or, indeed, event sourcing works (pros and cons).

OP, your main goal is to ask yourself:

  a) Can these two tools make my life easier (Docker? Definitely. K8s? Maybe.)
  b) Do they improve the quality of the work I do so my client/boss is happy?
mankash666
Repeatable, reusable environments for development doesn't need docker. And my statement on AWS Lambda is valid - while learning is always a net positive, knowing the innards of lambda's container scheduling isn't necessary for using it. One needn't learn chip design to program a computer, but you probably will write better assembly code of you designed the processor
mickeyp
They don't need it - but they sure do make it easier.
nickjj
I would say it's worth learning Docker.

I've been using it since 2014 and it's one of the best software related moves I've made.

I wouldn't say I'm a super fan boy of anything, and I typically only use tools when they make my life way better. Docker is one of those tools because it helps me set up development environments for the web applications I develop, as well as help me move those apps to production seamlessly.

I'm a freelance dev so I have a lot of projects on my machine, and it's amazing to be able to spin up and shut down each app in seconds and also hand everything over to a client and have them run 1 command to get everything up and running regardless of whether they are running Mac, Windows or Linux.

When it comes to production, it's as simple as installing Docker and then running a few commands to have everything up and running with total confidence that it works. That's what Docker can give you in the end with not too much effort.

The reason it's a lot better than a VM is how fast and efficient it is. There's no massive disk space overhead and containers start in milliseconds, not minutes. Honestly it's not even comparable to a VM. There's just too many wins that make it an entirely different beast.

If you don't want to put in the 50 or 100+ hour grind of learning it on your own, I've assembled a 5 hour premium video course that will teach you how to dockerize your own web apps. You'll walk away fully able to use Docker in the real world. There are node examples too. This course is everything I've learned in the last 3 years of using Docker and is filled with best practices from real world usage and feedback from hundreds of people.

You can find that course at https://diveintodocker.com/.

If you have any questions let me know. I'd be glad to help. I'm also a Docker Captain (Docker reached out to me to become a trusted content provider) so I'm not just pitching you on buying something (if I didn't fully believe in Docker I wouldn't be spending my time personally using it).

I also have plenty of free Docker related material on my blog[0] and Youtube[1]. The first hour of my premium course is available on Youtube, no strings attached.

[0]: https://nickjanetakis.com/blog/tag/docker

[1]: https://www.youtube.com/watch?v=XeSD17YRijk&list=PL-v3vdeWVE...

machinemob
In the amount of time it took for you to write this question and parse the subsequent opinions, rants and religious arguments, you could have followed a basic Docker tutorial, written a simple Dockerfile and answered the more important questions: Is this useful to me? Can I be more productive with these tools?

:)

togusa2017
I was running my side project without docker and trust me it was making my life hell due to the difference in production and development environment and also a fear that if I drop this server how the hell am I gonna setup another server with a small amount of downtime. I started moving my side project to docker and in the process learned a lot about it. Now I can't work without a docker . It's not perfect but God it's so much better
turtlebits
You don't have to, but docker + docker-compose is excellent for getting a development stack setup with extremely little effort (and other tooling you may need) - such as postgres/mysql/redis/mongodb/couchdb/jenkins/etc...

You don't need to learn how to build containers. Just the basics of how docker works.

turtlebits
Also it allows you to run a bunch of services with little resources and minimal config. I have a OVH $7/mo VPS running

- reverse proxy (traefik) - 2 web apps - jenkins - postgres - riemann - thumbor (thumbnailing service)

indescions_2017
Yes. It's ubiquitous now. It'll be a bit like everyone around you talking about the "Upside Down". And you have no idea because you never watch tv except for Emily Chang. Plus, you can understand it in under 1 hour:

https://training.docker.com/introduction-to-docker

If you are into Node, I'd also get up to speed with Serverless Deployments and Lambda Functions. Buzzwords, yes. But also modern, powerful and game changing. And most importantly, you won't have to worry about any devops internals ;)

https://cloud.google.com/functions/

[dead] joelbondurant
Docker for Linux != Vagrant/Virtual Box

Docker for MacOSX (Unix for idiots) == Vagrant/Virtual Box