Am I the only one getting a bit fatigued by all of this now? I think when we hit containers I just got a bit tired of going even further down the rabbit hole. Yes I might get a little more performance or stability or whatever from a container vs a full VM or a unikernel vs a container but it just gets more frustrating to manage in my experience. Anybody else feel the same?
When I just had a full VM with a full stack running on some hypervisor I could treat it just like a physical machine so management tools worked pretty much the same (with a few little exceptions). When containers came into the mix it just hit me with so much more work to admin. Unikernels seem like a lot of work too.
I'm thinking that for my smallish deployment setup I'm just going to do things the tried and true way for a while until the practices surrounding containers stabilize a bit and I don't feel like I would be playing catchup every week. The amount of time I would have to spend setting this stuff up is huge compared to the benefit. It's all still fascinating though to keep up with from an observer's standpoint.
Seems pretty weird that we think it's normal to run a computer that simulates several physical computers, each running an operating system built for the mainframe in a 70s university CS department closet that then only ever runs one user and one application. You just think it's easier because you're used to it. I'm glad that this development is happening. In a few years when it's mature, we'll all have a saner setup to work with.
>In a few years when it's mature, we'll all have a saner setup to work with.
I'm happy you get to work in an environment with no legacy applications that require a specific old operating system but that's not the case for a massive portion of sysadmin jobs.
From a recent post, here's a talk[1] about containers where they use SmartOS (Solaris/IllumOS derivative) to containerize Linux by just implementing the syscall table. The future of hosting legacy apps might be providing legacy interfaces on modern OS's (IMO).
Well the legacy app is a solved problem - we have VMs - let them live there and be done with it. Expose the API trough network and call it from anything you want.
Yeah I certainly see the contradiction(?) in things. That is the world we live in though. I think the biggest issue is with the tools we use to manage things. Looking at things as "a computer" with storage/network/cpu seen as one unit is pretty natural now as it has always been that way pretty much.
With one being on same PDP-11 as UNIX and more portable. The more I learn, the more I want to give modern computing stacks the Double Middle Finger. :)
I don't think containers are related to unikernels. The whole point of containers is to sacrifice a bit of performance in order to gain convenience.
Whereas for unikernels, it's the opposite
> Unikernels leverage the advantages of virtualisation to create an operating system that's as specialised and optimised as possible.
It makes no sense to craft a hyper efficient kernel and then run Ruby on it. If your requirements are such that Linux is too slow, you are probably an HFT. Or Google.
I am exceptionally sceptical that "specialised" and "secure" are anything but opposites.
"One hundred novel implementations of /dev/random" is a security nightmare. What you want is one implementation of /dev/random that everybody uses. Like uh, Linux.
Same goes for every other element of the kernel. A secure kernel does not come from using (I'm sorry to say) a hipster programming language, it comes from decades of abuse that is thrown at mainstream kernels.
It wouldn't be 100 implementations of /dev/random, but one implementation per language.
Security with a C / C++ based system that has to be everything to everyone is pretty much intractable. If you have a java unikernel server for example, you only have one programming language which you can understand from top to bottom with a lot of well written open source libraries.
Then you have a hypervisor, from what I understand can be far simpler than the linux kernel, POSIX and everything else. You only have two smaller layers to understand, versus the huge unix ecosystem stack to understand.
Also changing something inside your unikernel stack would be a git commit & deploy away. Patching a kernel vulnerability which you probably don't understand is a significantly longer undertaking.
"Then you have a hypervisor, from what I understand can be far simpler than the linux kernel, POSIX and everything else. You only have two smaller layers to understand, versus the huge unix ecosystem stack to understand."
That's exactly it. It's how it was done in old days for highly assured systems and it still applies. Side advantage is that anything you create that you control you can apply the best software and security engineering methods possible. So, the big messy OS might be hard to hit with static or covert channel analysis but your microkernel/hypervisor/VMM might do fine.
unikernels are only single language now for convenience and research purposes. in the future you'll probably compile to something similar to the llvm ir and generate a unikernel from that. there'll probably be a standard /dev/random, a standard tcp stack, a standard http parser, etc
The point of something like Halvm, the Haskell unikernel, is that can bring to bear a lot of formal methods for proving the components are secure. This sort of thing is untenable in a general purpose OS, but for specialized use case it makes a lot of sense.
Here (https://github.com/GaloisInc/haskell-tor) for instance is a Tor implementation written for HalVM. The surface area for proving this implementation is secure is dramatically smaller than the more general purpose one. That it is more efficient about resources means that you can run more nodes, which in this particular case is very important.
The author gave a talk at QCon last week, I can't seem to find video of it just now...
You're right about where it's going on that but wrong about where it is. The reason is the compiler, state of FP security analysis, and overall TCB. So much to prove that's non-trivial and even non-obvious how before we can trust as secure the object code that started as your Haskell program linking to that.
I advise safe constructions of simple imperative or functional languages until INFOSEC research in security verification catches up to things like Haskell. That's basically subsets of C, Java, Ada, ML, and LISP with certifying or hand-compilation. I'd love a certified Haskell runtime, though. Tolmach et al were making progress in that direction and might pick it back up eventually.
Containers don't sacrifice performance at all — that's a big part of why they are so popular. Containers are completely ordinary processes that happen to be namespaced and isolated through kernel mechanisms. They have direct access to hardware and all sorts of kernel services such as file systems and sockets. The only indirection involved is the overlay file system, but anything I/O-critical is usually mounted from the host. There's no runtime checking or proxying or anything, and therefore no runtime cost.
In time there will probably be mature build tools to compile unikernels from application sources. Instead of running maven or gradle to output a jar, you'll output a unikernel VM image. Containers complicate things because they add an extra layer onto a host OS. With unikernels there will be a layer removed, the host OS itself. You'll still need tools to manage and orchestrate the VMs running the unikernels, but I think things will get simpler, not more complicated.
My simplified version: A container is a way of sharing the resources of a kernel, with some resource isolation. A unikernel is a way of sharing the resources of a hypervisor.
There are pros and cons to each: a container lets you run other things on the same kernel, and introspect the running process (as well as do interception of things like sysdig does.) However, root-on-container generally means root-on-all-containers. So we use VMs to host containers in.
x86 is much smaller in terms of instruction set to a Linux kernel, there are historically fewer vulnerabilities, many fewer LoC, etc. The unikernel compiles your OS and libs together, meaning you can share a hypervisor - which is good for security. You lose some of the convenience of management of containers in order to get this.
You lose the ability to monitor by introspection: all you know is "this VM is using X CPU and X RAM", you can't look into what it's doing inside the VM. It's the same as a VM without an agent, today.
This can be considered a security win and a usability loss.
Does the xen hypervisor offer same-page merging?
I believe kvm as well as lxc/cgroups do, at least to some extent and in their own idiosyncratic ways.
From the point of view of a former sysadmin and current "private cloud" enthusiast that's a crucial question that likely leads to intricate discussions on security as well.
I pretty much stay away from all this stuff, but I keep a keen eye on it. The whole virtualization space is pretty interesting (potentially), but it seems like people aren't too clear about the different use cases. Rather, they promote whatever bit of tech they learned as The Solution, even though VMs, docker, unikernels, et al obviously have different characteristics.
Nobody is stopping you from using VMs instead of containers. I bet most deployments aren't using containers. And unikernels are very much experimental and research right now, especially since they aren't made to easily package an existing application.
Containers, VMs and Unikernels all have different characteristics and philosophies, if one fits all your needs there is no need to make it more complicated by introducing another type. But if you need something specific, they are there.
There's a huge spike of value created when you cross the threshold of "you really don't need to think about sysadmin stuff, you can just focus on the application layer."
Heroku was the first to cross it, and they do a bang-up job with Rails apps, but ever since it came out there has been this sort of question hanging in the air of "how does this generalize?"
A VM on something like Amazon EC2 was the best we could do at that time. It certainly generalizes, but you have to do quite a lot of sysadmin work to keep an app running healthily. Docker was an improvement on that, taking away some underlying system work. But then losing the security benefits of VMs and still, in the end, requiring a bunch of systemsy stuff.
Of course Heroku kept firing with its buildpacks supporting lots of platforms and a host of infrastructure provided as a service. But that doesn't really provide all of the control application developers need. You're sort of at the mercy of what infrastructure there's a market for, at market prices.
So what's happening is both sides are sort of pressing in to the middle... On the docker side trying to build general-purpose containers that can run on a shared foundation of rock solid system images, and on the IaaS side trying to do as much as possible over a network connection.
We will eventually meet in the middle, with a set of secure, well-defined interfaces between an application and the infrastructure it relies on, which is robust enough to handle the vast majority of software.
Then again, if you squint your eyes that's kind of what we already have. We're just slowly decreasing the surface area of those interfaces and increasing their security. What you're feeling is the pain of that surface area being still pretty big and the job of surveying daunting. Luckily you don't need to map the whole thing, just know enough to make good decisions this week.
And honestly, I think for most people, focusing on how to build more human-relevant software is going to be way more helpful to your career than learning about the successor to Docker. But that's just some unsolicited career advice. :)
> There's a huge spike of value created when you cross the threshold of "you really don't need to think about sysadmin stuff, you can just focus on the application layer."
Right up until you discover that availability and data integrity and security are still problems, as you discover when it turns out your understanding of Amazon's AZs is faulty and your business goes offline.
I tend to think of unikernels as having to 'front-load' more of the work. That's why you can treat the end artefact as almost disposable.
With a traditional stack, it may be easier to set up, but you're then committing to acting as caretaker for the code that runs on that machine, inc. patches, upgrades, version constraints, etc. There's probably more work there over the long term.
This is kind of where the distinction of 'cattle vs pets' comes from.
You're very likely not alone, but this is just a symptom of an underlying truth that in engineering, there are always trade-offs. Even with fundamentals like sort algorithms or hash tables, the best implementations for general cases may not be the best choice for some individual cases (mostly-sorted inputs, for example).
Quicksort and universal hash tables use randomization to avoid problems that arise with pathological sets of data, and so their performance can only be given on average.
If even simple CS primitives have such trade offs, then those are certainly going to propagate up the complexity tree as we engineer solutions on top of them.
In general, it's probably best to just know that these things exist and what their strengths and weaknesses are (they will almost certainly always have both) so that you can pick the right tools for the job at hand.
Unikernels are about security by auditability, mostly. Where containers have all of the security problems of two layers of operating systems and a container daemon and a complete OS kernel (before you even get to a hypervisor), unikernels are just your app and the drivers it needs sitting on the hypervisor. Or even on bare metal.
It's a fairly different philosophy—I don't think anyone really cares about the performance aspect of it (those people care more about getting zero-copy low-level IO in userland while still reaping the "control-plane" benefits of having an OS, using e.g. Snabb Switch.)
It's more like the philosophy behind Erlang releases or black-box "appliance" VMs. Which is, basically, that you don't maintain "the OS" as a thing separate from your app; you just maintain "your app", and any bits of OS are basically part of the vendored deps of each successive app release.
If one looks at how many OSes evolved in the early days of computing history, there was always an overlap between rich language runtimes and the OS.
The language runtime and libraries built on top of it were the OS.
Seen by this lenses, POSIX could be seen as the C runtime, if one would stretch this example.
So for me, it is quite natural that when one has languages with rich runtimes, the OS is just another layer that can be taken out and we can go back to the days when runtimes were mostly targeting bare-metal.
I can certainly see a lot of benefits with this. We host for some of our customers, and this would allow us to isolate each customer completely from each other. That is worth a lot when they store sensetive data in the application.
Seperate VMs would as well, but it's not really feasible to run a full Windows Server for each customer.
All my code is position independent and my compiler only makes position idependent code.
My OS is ring-zero-only. Just in time compilers do not need a linker in a single address space -- they just insert the address of the function call.
> Even core modules in the Linux kernel don't apply to every installation. Things like USB drivers are useless in a virtualised "cloud" environment, but are still included in the kernel.
You don't have to compile them in, or load them as modules, if you don't want. Or am I confused about what they're trying to say?
I've been using Linux for long enough to recall when you had to recompile your kernel in order to add device drivers...
It is said in a very confusing manner. The core idea is dependency inversion: instead of the admin dictating the capabilities of the kernel, dictating the capabilities of the applications, we turn the whole thing around.
The client application might use a library dependent on, say, a high-frequency clock. Therefor, during the build of the application, the specific capability is build into the kernel. Synchronisation and sanity checks of this high-frequency clock runs only when this specific image (or unikernel) is loaded.
It's a pretty neat idea and fits well in the global trend of dependency inversion. It also highlights the gaping architectural hole in the current container (docker) offerings: by using a base-image, you specifically ask for everything that is necessary for: networking, IPC, process management, scheduling, file system management, package management, printing, the whole POSIX interface etc. etc. This in turn creates a huge attack surface from a security perspective and makes maintenance and quality control much harder.
My sole question with unikernels is: doesn't this just move resource management (scheduling) upwards and security (access to resources) downwards? Or in other words, are we not creating two extra rings?
1) How does a unikernel differ from a library OS? Is a unikernel a kind of library OS?
2) Is the specialization the important aspect? I see OSv referred to as a unikernel but it does no specialization, it supports almost the entire Linux ABI. I'd argue it is a library OS, though.
Yes I think 1. is true, it is a library OS and your application, but very much in the library OS model.
OSv is a bit unusual architecturally, it is almost a very small OS that runs applications in a single address space. It is perhaps a hybrid architecture.
I've described it by saying unikernels are constructed by library operating systems. ie, the unikernel is the artefact but the libOS is what you (as a dev) actually work with.
1) According to the source article, a unikernel is a kind of library OS and I think you hit the right spot with
2) specialization. AFAIK the the concept of a library OS is from a time when single purpose computing was not on the agenda. We could argue a lot about the different ways of multiplexing and isolating different workloads and how maybe privilege levels were meant to solve the same problem as hardware virtualization but the spirit of unikernels (and library OSs) is to use language and program against an API that specializes to whatever ABI your target happens to be.
The name unikernel is used more in the context of specialized applications in the sense of what a UNIX process is supposed to be (do one thing) but I'd say the important aspect is still the use of language technology which then enables specialized code.
I'm not too familiar with OSv but I think the goal is to provide a runtime for off the shelf java applications. It should be possible to fuse an application with just the needed parts of libc and drivers, though.
> Honestly, if you're thinking about all the levels an application is built upon, it's a miracle things even work.
One mitigating factor is that all those russian-doll levels don't switch back and forth all that much (thinking in computer, not human, time). Moreover, some of those levels (like the BIOS) do their thing and then they cede control until the end of time (i.e. reset).
So yes, the multilayered cake is pretty impressive, but the situation is not that dire.
Compile and then link the application code with the particular pieces of the toolkits that you needed and with the provided kernel into a single download image, and either embed or boot or network boot the target box using it. Remote debugging was available, as well as a relational database, etc.
Not just another that knows of VMS and the prior innovations but one whose site taught me many things about those days. Glad to get a chance to say (tips hat) thanks for the info. :)
Wish modern systems were designed and implemented with even half the skill that VMS, etc were. I've never gotten to experience an OS reliable enough that I temporarily forgot reboot command. Desktop Linux will have to do. ;)
I knew HP would kill it after acquisition due to rule of no competing product lines (i.e. VMS vs NonStop). I was relieved that they spun it off to another company that began a Xeon port. Two quick questions that you might know already if you've contacted them:
1. Has the company been more clear privately than the website on how long it will be before a Xeon port materializes?
2. Have they significantly reduced the OpenVMS licensing costs that kept many off it?
OpenVMS and NonStop (NSK) do not compete. Entirely different products and markets.
VMS Software Inc (VSI) have licensed OpenVMS from HPE. They've not acquired the product.
The company was newly formed, not a spin-off of HP.
VSI have been circumspect on their x86-64 release schedule.
The team includes many of the same development folks that ported OpenVMS from Alpha to Itanium.
OpenVMS I64 Itanium software list prices are unchanged from those of HPE. Whether and how much the VSI sales reps might be discounting, you'd have to ask them.
"OpenVMS and NonStop (NSK) do not compete. Entirely different products and markets."
Outside legacy market, I thought they both advertised as being for business-critical systems that can't afford downtime. I remembered plenty of HP advertisements on that for OpenVMS & its clustering. NonStop obviously does that stuff with even better availability due to FT HW/SW. What makes you say they don't compete?
Note: I'm not talking about the five 9's, HW-supported stuff that OpenVMS probably can't touch.
This is also how most embedded applications utilizing an OS also works these days. The OS is a library, you link in whatever you need from the OS + compile time configuration of OS features etc. + link in whatever you need for your application.
I remember you always had to bundle everything together with the exception of what was already present in the firmware.
Also some books I had access in the mid-90's were all about creating your own libOS with tasks and lightweight communication primitives and then bundling everything together.
Coding since the 80's, but the most embedded stuff I did was PC (bypassing MS-DOS), Amiga and Symbian and reading also PIC programming on Elektor.
So maybe my perception might not be the correct one.
I don't understand the point of running a unikernel inside a Docker container. I mean, Docker runs on top of the Linux kernel, so how is this reducing the stack in any way?
OSv looks really promising. Turns regular jvm bytecode into unikernals. If you are building immutable apps, why not deploy them with an immutable foundation as well? The HN hivemind has declared microservices bad for bootstrapping new apps, but I'm convinced an end to end immutable workflow could actually dramatically boost team wide productivity. Tomorrow is a snow day so I get to play with OSv and see if I'm right.
http://lsub.org/export/clivesys.pdf clive is the golang one. It's not focused on actually running golang programs. It's acutally more of a research project by some academics who are looking into replacing unix style ipc with golang style channels (because they can return an error). I was unable to get any response from them depsite trying several different contacts. I also couldn't get their system to actually compile even though I tried several different targets.
I keep meaning to write a counter to the linked paper by Anti on I/O and whatever else I spot once I get past that part. Paints an inaccurate history of I/O and where problems come in. The I/O actually started more complex and effective with dedicated I/O processors, architecture, & programs. This eventually became Channel I/O for mainframes:
Thanks to that, the compute nodes are always computing and I/O processors are always handling the interrupts, protocol processing, whatever. This kept utilization and throughput high. Things being so expensive are why they stripped components out until the small machines DOS and UNIX were on didn't support crap, esp Channel I/O. Like most legacy effects, such fundamentally architecture was kept in there as hardware improved and got beyond cheap enough for a Channel I/O-like solution. We've seen over past decade various offloading mechanisms but the original architectures were fundamentally better for smart I/O and virtualization. They just weren't adopted, forcing people to work around the crap we have in clever ways.
Note 1: Even true for embedded as I/O coprocessors can greatly aid determinism and throughput. Upon my suggesting it, Ganssle sent me a SOC that had a mid-to-high end part for compute plus a weaker CPU for I/O handling with many supposed benefits. The old mainframes had a high-end processor for compute plus low-end one for I/O handling. Put a smile on my face to see the lesson rediscovered. :)
Note 2: A number of systems, esp kernelized architectures, tried to minimize the amount of OS for user applications or OS's starting with VM/370 I believe and KeyKOS later doing nanokernel. So, that isn't new either. Rump kernels easy integration with kernels without forks or extensive modifications are a significant innovation, though.
I wonder how much of this is premature optimisation, or even optimisation targetting the wrong layer, where people would be better to focus on application changes.
honest question: would elaborate run-time-environments e.g. erlang-vm considered unikernel for an application running in this environment. for example, as we now know, erlang provides some fairly low-level primitives for application execution e.g. it's own os-agnostic preemptive thread scheduling, per thread async io, per-thread-gc etc. etc.
Unikernels : hypervisor :: processes : traditional operating system
Joyent CTO Bryan Cantrill gave a great talk at Velocity NYC 2015 about the history of containers (going back to March 18, 1982 when Bill Joy added the chroot system call) and the problems of running containers on operating systems in VMs on hypervisors:
"Stop killing kittens and melthing the ice caps: Run containers on bare metal already!"
Not an expert here, but it seems to me like unikernel is the latest buzzword. The alleged advantages are:
> Improved security: only the core of the OS is implemented, no video or USB drivers that aren't needed and could be a source of intrusion.
I am not aware of any video or USB driver vulnerability which led to security concerns on a typical server setup in the last decade.
> Very small footprint: imagine being able to remove 95% of the kernel size, simply because your application doesn't need it.
This may free up 1 MB of RAM or maybe two. Unrelevant unless you're doing embedded programming, which is not the environment unikernels are targeting.
> Specialised implementations: you know your application and you can tweak and run your kernel exactly the way you want it.
Too vague of a statement to even criticize, anyway my applications conform to the POSIX specification so that they are portable and run exactly the way I want them.
> Quick enough to be "Just in time" to summon a unikernel live (similar to live spawning Docker instances), with boot times less than 1 second.
This may be a fair point in a system where the deployment time of a new instance is critical.
> I am not aware of any video or USB driver vulnerability
> which led to security concerns on a typical server setup
> in the last decade.
CVE-2015-3456 was raised in April which had a privilege escalation due to a floppy driver controller, and caused widespread reboots and replacements of images in the Amazon cloud:
A very minimal linux kernel, a basic set of CLI primitives, NPM, and a few JS ports of traditional POSIX tools.
Looks like it'll be an easy path to building immutable stateless web servers. It can be run on bare metal (Raspberry Pi), Docker containers, and everything in between.
Start small and extend using the existing ecosystem.
I'm really looking forward to this project reaching full maturity.
70 comments
[ 3.5 ms ] story [ 128 ms ] threadWhen I just had a full VM with a full stack running on some hypervisor I could treat it just like a physical machine so management tools worked pretty much the same (with a few little exceptions). When containers came into the mix it just hit me with so much more work to admin. Unikernels seem like a lot of work too.
Am I alone in thinking this??
I'm happy you get to work in an environment with no legacy applications that require a specific old operating system but that's not the case for a massive portion of sysadmin jobs.
1: https://www.youtube.com/watch?v=coFIEH3vXPw
Whereas for unikernels, it's the opposite
> Unikernels leverage the advantages of virtualisation to create an operating system that's as specialised and optimised as possible.
It makes no sense to craft a hyper efficient kernel and then run Ruby on it. If your requirements are such that Linux is too slow, you are probably an HFT. Or Google.
"One hundred novel implementations of /dev/random" is a security nightmare. What you want is one implementation of /dev/random that everybody uses. Like uh, Linux.
Same goes for every other element of the kernel. A secure kernel does not come from using (I'm sorry to say) a hipster programming language, it comes from decades of abuse that is thrown at mainstream kernels.
Security with a C / C++ based system that has to be everything to everyone is pretty much intractable. If you have a java unikernel server for example, you only have one programming language which you can understand from top to bottom with a lot of well written open source libraries.
Then you have a hypervisor, from what I understand can be far simpler than the linux kernel, POSIX and everything else. You only have two smaller layers to understand, versus the huge unix ecosystem stack to understand.
Also changing something inside your unikernel stack would be a git commit & deploy away. Patching a kernel vulnerability which you probably don't understand is a significantly longer undertaking.
That's exactly it. It's how it was done in old days for highly assured systems and it still applies. Side advantage is that anything you create that you control you can apply the best software and security engineering methods possible. So, the big messy OS might be hard to hit with static or covert channel analysis but your microkernel/hypervisor/VMM might do fine.
Here (https://github.com/GaloisInc/haskell-tor) for instance is a Tor implementation written for HalVM. The surface area for proving this implementation is secure is dramatically smaller than the more general purpose one. That it is more efficient about resources means that you can run more nodes, which in this particular case is very important.
The author gave a talk at QCon last week, I can't seem to find video of it just now...
I advise safe constructions of simple imperative or functional languages until INFOSEC research in security verification catches up to things like Haskell. That's basically subsets of C, Java, Ada, ML, and LISP with certifying or hand-compilation. I'd love a certified Haskell runtime, though. Tolmach et al were making progress in that direction and might pick it back up eventually.
There are pros and cons to each: a container lets you run other things on the same kernel, and introspect the running process (as well as do interception of things like sysdig does.) However, root-on-container generally means root-on-all-containers. So we use VMs to host containers in.
x86 is much smaller in terms of instruction set to a Linux kernel, there are historically fewer vulnerabilities, many fewer LoC, etc. The unikernel compiles your OS and libs together, meaning you can share a hypervisor - which is good for security. You lose some of the convenience of management of containers in order to get this.
Not necessarily... http://unikernel.org/blog/2015/unikernels-meet-docker/
This can be considered a security win and a usability loss.
From the point of view of a former sysadmin and current "private cloud" enthusiast that's a crucial question that likely leads to intricate discussions on security as well.
Containers, VMs and Unikernels all have different characteristics and philosophies, if one fits all your needs there is no need to make it more complicated by introducing another type. But if you need something specific, they are there.
Heroku was the first to cross it, and they do a bang-up job with Rails apps, but ever since it came out there has been this sort of question hanging in the air of "how does this generalize?"
A VM on something like Amazon EC2 was the best we could do at that time. It certainly generalizes, but you have to do quite a lot of sysadmin work to keep an app running healthily. Docker was an improvement on that, taking away some underlying system work. But then losing the security benefits of VMs and still, in the end, requiring a bunch of systemsy stuff.
Of course Heroku kept firing with its buildpacks supporting lots of platforms and a host of infrastructure provided as a service. But that doesn't really provide all of the control application developers need. You're sort of at the mercy of what infrastructure there's a market for, at market prices.
So what's happening is both sides are sort of pressing in to the middle... On the docker side trying to build general-purpose containers that can run on a shared foundation of rock solid system images, and on the IaaS side trying to do as much as possible over a network connection.
We will eventually meet in the middle, with a set of secure, well-defined interfaces between an application and the infrastructure it relies on, which is robust enough to handle the vast majority of software.
Then again, if you squint your eyes that's kind of what we already have. We're just slowly decreasing the surface area of those interfaces and increasing their security. What you're feeling is the pain of that surface area being still pretty big and the job of surveying daunting. Luckily you don't need to map the whole thing, just know enough to make good decisions this week.
And honestly, I think for most people, focusing on how to build more human-relevant software is going to be way more helpful to your career than learning about the successor to Docker. But that's just some unsolicited career advice. :)
Right up until you discover that availability and data integrity and security are still problems, as you discover when it turns out your understanding of Amazon's AZs is faulty and your business goes offline.
I tend to think of unikernels as having to 'front-load' more of the work. That's why you can treat the end artefact as almost disposable.
With a traditional stack, it may be easier to set up, but you're then committing to acting as caretaker for the code that runs on that machine, inc. patches, upgrades, version constraints, etc. There's probably more work there over the long term.
This is kind of where the distinction of 'cattle vs pets' comes from.
Quicksort and universal hash tables use randomization to avoid problems that arise with pathological sets of data, and so their performance can only be given on average.
If even simple CS primitives have such trade offs, then those are certainly going to propagate up the complexity tree as we engineer solutions on top of them.
In general, it's probably best to just know that these things exist and what their strengths and weaknesses are (they will almost certainly always have both) so that you can pick the right tools for the job at hand.
It's a fairly different philosophy—I don't think anyone really cares about the performance aspect of it (those people care more about getting zero-copy low-level IO in userland while still reaping the "control-plane" benefits of having an OS, using e.g. Snabb Switch.)
It's more like the philosophy behind Erlang releases or black-box "appliance" VMs. Which is, basically, that you don't maintain "the OS" as a thing separate from your app; you just maintain "your app", and any bits of OS are basically part of the vendored deps of each successive app release.
The language runtime and libraries built on top of it were the OS.
Seen by this lenses, POSIX could be seen as the C runtime, if one would stretch this example.
So for me, it is quite natural that when one has languages with rich runtimes, the OS is just another layer that can be taken out and we can go back to the days when runtimes were mostly targeting bare-metal.
Seperate VMs would as well, but it's not really feasible to run a full Windows Server for each customer.
> Even core modules in the Linux kernel don't apply to every installation. Things like USB drivers are useless in a virtualised "cloud" environment, but are still included in the kernel.
You don't have to compile them in, or load them as modules, if you don't want. Or am I confused about what they're trying to say?
I've been using Linux for long enough to recall when you had to recompile your kernel in order to add device drivers...
The client application might use a library dependent on, say, a high-frequency clock. Therefor, during the build of the application, the specific capability is build into the kernel. Synchronisation and sanity checks of this high-frequency clock runs only when this specific image (or unikernel) is loaded.
It's a pretty neat idea and fits well in the global trend of dependency inversion. It also highlights the gaping architectural hole in the current container (docker) offerings: by using a base-image, you specifically ask for everything that is necessary for: networking, IPC, process management, scheduling, file system management, package management, printing, the whole POSIX interface etc. etc. This in turn creates a huge attack surface from a security perspective and makes maintenance and quality control much harder.
My sole question with unikernels is: doesn't this just move resource management (scheduling) upwards and security (access to resources) downwards? Or in other words, are we not creating two extra rings?
2) Is the specialization the important aspect? I see OSv referred to as a unikernel but it does no specialization, it supports almost the entire Linux ABI. I'd argue it is a library OS, though.
OSv is a bit unusual architecturally, it is almost a very small OS that runs applications in a single address space. It is perhaps a hybrid architecture.
The first line of the Wikipedia page is written this way - https://en.wikipedia.org/wiki/Unikernel
2) specialization. AFAIK the the concept of a library OS is from a time when single purpose computing was not on the agenda. We could argue a lot about the different ways of multiplexing and isolating different workloads and how maybe privilege levels were meant to solve the same problem as hardware virtualization but the spirit of unikernels (and library OSs) is to use language and program against an API that specializes to whatever ABI your target happens to be. The name unikernel is used more in the context of specialized applications in the sense of what a UNIX process is supposed to be (do one thing) but I'd say the important aspect is still the use of language technology which then enables specialized code.
I'm not too familiar with OSv but I think the goal is to provide a runtime for off the shelf java applications. It should be possible to fuse an application with just the needed parts of libc and drivers, though.
One mitigating factor is that all those russian-doll levels don't switch back and forth all that much (thinking in computer, not human, time). Moreover, some of those levels (like the BIOS) do their thing and then they cede control until the end of time (i.e. reset).
So yes, the multilayered cake is pretty impressive, but the situation is not that dire.
[1] https://news.ycombinator.com/item?id=10585404
One example of this general design is VAX ELN.
The following is a brochure from the V2.2 release, from 1986:
http://bitsavers.trailing-edge.com/pdf/dec/vax/vaxeln/2.0/VA...
Compile and then link the application code with the particular pieces of the toolkits that you needed and with the provided kernel into a single download image, and either embed or boot or network boot the target box using it. Remote debugging was available, as well as a relational database, etc.
Wish modern systems were designed and implemented with even half the skill that VMS, etc were. I've never gotten to experience an OS reliable enough that I temporarily forgot reboot command. Desktop Linux will have to do. ;)
re: "those days" — FWIW, those days are still ongoing. The latest OpenVMS release shipped out in June 2015, and the native port to x86-64 is underway.
1. Has the company been more clear privately than the website on how long it will be before a Xeon port materializes?
2. Have they significantly reduced the OpenVMS licensing costs that kept many off it?
OpenVMS and NonStop (NSK) do not compete. Entirely different products and markets.
VMS Software Inc (VSI) have licensed OpenVMS from HPE. They've not acquired the product.
The company was newly formed, not a spin-off of HP.
VSI have been circumspect on their x86-64 release schedule.
The team includes many of the same development folks that ported OpenVMS from Alpha to Itanium.
OpenVMS I64 Itanium software list prices are unchanged from those of HPE. Whether and how much the VSI sales reps might be discounting, you'd have to ask them.
Related: http://labs.hoffmanlabs.com/node/1917 http://www.vmssoftware.com/pdfs/VSI_DrawerSt_v2.pdf
"OpenVMS and NonStop (NSK) do not compete. Entirely different products and markets."
Outside legacy market, I thought they both advertised as being for business-critical systems that can't afford downtime. I remembered plenty of HP advertisements on that for OpenVMS & its clustering. NonStop obviously does that stuff with even better availability due to FT HW/SW. What makes you say they don't compete?
Note: I'm not talking about the five 9's, HW-supported stuff that OpenVMS probably can't touch.
I remember you always had to bundle everything together with the exception of what was already present in the firmware.
Also some books I had access in the mid-90's were all about creating your own libOS with tasks and lightweight communication primitives and then bundling everything together.
Coding since the 80's, but the most embedded stuff I did was PC (bypassing MS-DOS), Amiga and Symbian and reading also PIC programming on Elektor.
So maybe my perception might not be the correct one.
As I mentioned, I have a very small view of how it all works.
It has nothing to do with bytecode per se.
It allows a JVM to run bare-metal, there are plenty of them already in the embedded world.
For example J9 from IBM.
Thanks
https://en.wikipedia.org/wiki/I/O_channel
Thanks to that, the compute nodes are always computing and I/O processors are always handling the interrupts, protocol processing, whatever. This kept utilization and throughput high. Things being so expensive are why they stripped components out until the small machines DOS and UNIX were on didn't support crap, esp Channel I/O. Like most legacy effects, such fundamentally architecture was kept in there as hardware improved and got beyond cheap enough for a Channel I/O-like solution. We've seen over past decade various offloading mechanisms but the original architectures were fundamentally better for smart I/O and virtualization. They just weren't adopted, forcing people to work around the crap we have in clever ways.
Note 1: Even true for embedded as I/O coprocessors can greatly aid determinism and throughput. Upon my suggesting it, Ganssle sent me a SOC that had a mid-to-high end part for compute plus a weaker CPU for I/O handling with many supposed benefits. The old mainframes had a high-end processor for compute plus low-end one for I/O handling. Put a smile on my face to see the lesson rediscovered. :)
Note 2: A number of systems, esp kernelized architectures, tried to minimize the amount of OS for user applications or OS's starting with VM/370 I believe and KeyKOS later doing nanokernel. So, that isn't new either. Rump kernels easy integration with kernels without forks or extensive modifications are a significant innovation, though.
Joyent CTO Bryan Cantrill gave a great talk at Velocity NYC 2015 about the history of containers (going back to March 18, 1982 when Bill Joy added the chroot system call) and the problems of running containers on operating systems in VMs on hypervisors:
"Stop killing kittens and melthing the ice caps: Run containers on bare metal already!"
https://www.youtube.com/watch?v=coFIEH3vXPw
> Improved security: only the core of the OS is implemented, no video or USB drivers that aren't needed and could be a source of intrusion.
I am not aware of any video or USB driver vulnerability which led to security concerns on a typical server setup in the last decade.
> Very small footprint: imagine being able to remove 95% of the kernel size, simply because your application doesn't need it.
This may free up 1 MB of RAM or maybe two. Unrelevant unless you're doing embedded programming, which is not the environment unikernels are targeting.
> Specialised implementations: you know your application and you can tweak and run your kernel exactly the way you want it.
Too vague of a statement to even criticize, anyway my applications conform to the POSIX specification so that they are portable and run exactly the way I want them.
> Quick enough to be "Just in time" to summon a unikernel live (similar to live spawning Docker instances), with boot times less than 1 second.
This may be a fair point in a system where the deployment time of a new instance is critical.
CVE-2015-3456 was raised in April which had a privilege escalation due to a floppy driver controller, and caused widespread reboots and replacements of images in the Amazon cloud:
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3456
A very minimal linux kernel, a basic set of CLI primitives, NPM, and a few JS ports of traditional POSIX tools.
Looks like it'll be an easy path to building immutable stateless web servers. It can be run on bare metal (Raspberry Pi), Docker containers, and everything in between.
Start small and extend using the existing ecosystem.
I'm really looking forward to this project reaching full maturity.