AFAIK it’s not a port of bhyve but a Port of the bhyve userland tools that tap into the MacOS hypervisor framework.
I wish there was a simpler approach to the hypervisor framework. I normally avoid using any third party software that wants to taint the kernel and third party hypervisors fall into that category. There is parallels desktop lite which can be installed from the App Store but it’s extremely limited (and paywalled) for something that I suspect it’s just a pretty interface to the hypervisor framework.
The hard drive images are literally a qemu image for linux kvm that's been qemu-img converted to a raw format, then the kernel and initrd were extracted, and stored in the mac file system. This image was originally on my linux box for a while, before I needed to go mobile on the laptop with it.
I'm seeing about 15% slower raw speed than the bare drive, dd if=/dev/zero of=tmp bs=1M count=512 is giving me between 68 and 250 mb/sec on a 2013 MBA with a nearly full disk, where in the terminal I'm getting between 78mb/s and 290mb/s.
There was Veertu Desktop, which used Hypervisor.framework, but they have pivoted into CI for iOS. At some point it was open sourced, but it seems to have disappeared from their GitHub account. At least someone seems to have made a copy before:
Was not aware of this! Thanks for sharing, this could be a great breakthrough for interfacing with hvf machines. xhyve is sorely missing an accessible management cli.
Mmmm eGPU + Passthrough would be sweet. Imagine gaming or using GPU heavy applications on a Windows VM with dedicated GPU and without the need for a separate set of monitor and input devices like Looking Glass does. https://github.com/gnif/LookingGlass
> QEMU now has ‘hvf’ (hypervisor.framework) available in master as an accelerator, I think it was a summer of code project.
It was not a Summer of Code project, there was (is) a company called Veertu that did the work who sold a virtualization product for running Linux and Windows on macOS on the App Store. Because it used Hypervisor.framework, it could be sandboxed and could be on the Mac App Store.
Google forked Veertu's GitHub repo, adding a few bells and whistles for use in their Android emulator.
Thanks for sharing the scripts. The instruction is intuitive -- well organized and contains a very detailed description of the virtual disk part, very neat comments. As a long-time user of OS X and homebrew, I must say I become very interested in your solution -- running PostgreSQL natively on OS X, or macOS.
However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System. As far as I know, macOS has a newly-added but slowly-performed file system, named APFS. It might be good enough for single disk request but not good for the concurrent requests (like 4; ; ) -- I am not questioning/blaming about the old port design of either xhyve, OS X, or even hardware architecture SCSI, but that's where we can start now.
Anyway, there is a newly-adopted storage technology named NVME emulation which might help a lot in your case -- not only it handles with lots of disk requests at the same time (65535 queues and 65535 commands per queue), but also the architecture makes it possible to program the controller code in software, especially in user world. I don't know all details of NVMe but it works well in my experience -- both of consumer level and enterprise level -- it brings smooth disk latency in the virtual machine even under a very high bandwidth (50Gib transfer in nearly half a minute, can you believe that?). Do you know if it is possible to use NVMe emulation for faster disk access in xhyve and accelerate the PostgreSQL execution natively?
>However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System
I assume this is to build a development environment. Especially if you're working on a laptop, on a client-server app that needs to talk to a Postgress (or other) database, running the database in a VM so it goes where you go is a pretty common setup.
It is true and that's probably the most unclear part of the original blog. I think the author has given some kind of advice -- use a vnet based on host-bridge -- not sure why he just commented out from the script. In my understanding, if I am working on a laptop, I can just set SNAT rules which allow traffic from a private network to go out to the internet. Given that the number of connection won't be large (no more than 400 connections, or 500, a usual number of open file for one process), it is acceptable that the new connections can wait until a period of time.
It would be great if it was possible to buy MacBooks with FreeBSD instead of OSX, which is lately bigger and slower compared to FreeBSD.l, especially due to the useless graphic effects and the bloatware installed (iTunes as example)
How dare you suggest the use of a PeasantOS to a clientele so refined as Apple's?!
Back to your basements, you lutes, and keep hoping you produce something good enough to be included in glorious OSX, that they might take it and in return give you nothing except for the knowledge that you, a mere programmer, have managed to contribute to mighty Apple's bottom-line.
But yeah, that it would be pretty sweet, even though it will never happen. For Apple software is about taking control, not giving it.
For an "average user" Linux, and FreeBSD are absolutely identical. If it's supplied with pre-installed GUI (say Ubuntu as Linux, and TrueOS on BSD side) it's more or less the same point-and-click experience. If it's bare console - well, it would be the same OMG-what's-that experience. Differences will start to appear for a user who wants to dive deeper, but that would make him/her non-average apparently.
If you're using a GUI, they're largely the same, with a slight edge to one or the other depending on your use case. For example, X on TrueOS plays more nicely out of the box with VirtualBox than almost any Linux distro I've tried.
If you're using the command line, FreeBSD has far better and far more consistent documentation.
I don't think iTunes comes pre-installed these days. It gets updated so frequently they decided to just prompt the user to do an install on first boot. Certainly if you do a fresh install of Mojave over the wire, as I did, it's pretty clean. No iTunes, Photos, iMovie, Pages or any of their pro-sumer stuff by default.
I realized I was being a bit cavalier and did a bit of checking. Been a while since I bought a fresh Mac, but last time I must have done a fresh install off the wire straight away and forgotten about it. My bad.
I'm pretty sure iTunes still comes preinstalled in Mojave. Not to mention Chess, DVD Player, Home, News, Stocks, Books, and a fair amount of other crap.
I have a post-install script I always run that does sudo rm -rf on all the apps I don't like. Has no ill effects in all the years I have been using it, and makes my system feel clean.
I've played with xhyve and dockers version of it -hyperkit-, too. Even wrote a little bash script to help with creation, start and edit of VM config files for it.
If anyone's interested.
For example: Pre-APFS, imagine running any software with a custom data store using file system holes. Definitely something you'd rather do in a FreeBSD VM than on OS X directly.
Not at all. I used xhyve to run FreeBSD on my mac laptop, because I want to have a dev environment that matches production (except for performance wise). Mac OS's BSD userland is sort of similar to FreeBSD, but there's plenty of differences, and although it took a while to get xhyve working (and before it was available, I used virtualbox, which also took time to setup), the gains in being able to try something and know it will work on production, instead of trying something and having to try it again were worth it.
I've been moving off Ubuntu in production since it adopted systemd, and a local FreeBSD VM is essential for developing & testing the provisioning and deploy scripts.
I hope xhyve will become a first class supported feature that gets shipped with macOS. I see recent commits from Apple engineers https://github.com/machyve/xhyve
It's a virtual machine, not a container based system. So it's going to perform somewhat like newer versions of VirtualBox I'd imagine. I'm going to take this for a test spin and if I get time I'll post back my benchmarks here.
I've been using this setup for years and glad to see some info here on HN about it. It's very fast but more importantly for development it really does a great job for porting to the real thing. I've never built anything in Xhyve that fails on a real install. Most of the differences I've seen are in vmnet versus like an e1000 device. I don't believe it does full duplex as per the xhyve docs (its a TODO under virtio_net) and on the MacOS side there is no Tap device like with Hyperkit. Despite that web pages can be served quite fast, a lot of ops are a stub to mmap on the host. NFS runs really well as a native format to share between guest and host. Heh, I've left it on for a couple of months accidentally and was still running like a top. My only real gripe is NTP drifts when the host goes to sleep and there is not an out-of-the-box solution for that like say VMWare. Someone has gone part of the way with getting e1000 for FreeBSD with Hyperkit it just needs to be looked into more, but this is a good explanation- and explains the performance differences: http://thebsdbox.co.uk/a-peak-inside-docker-for-mac-hyperkit...
44 comments
[ 2.8 ms ] story [ 83.3 ms ] threadI wish there was a simpler approach to the hypervisor framework. I normally avoid using any third party software that wants to taint the kernel and third party hypervisors fall into that category. There is parallels desktop lite which can be installed from the App Store but it’s extremely limited (and paywalled) for something that I suspect it’s just a pretty interface to the hypervisor framework.
I've had good experiences with it running linux guests, it took a bit of tweaking to get networking setup, but other than that it's been solid.
Can you point me to the resources you used? Thanks.
time (git clone github.com/mist64/xhyve.git; cd xhyve; make; ./xhyverun.sh) w/ "sudo halt" pasted into the resulting shell: 14.85s
https://gist.github.com/wiredfool/0e1aa099a29f562cdc3ba30c36...
The hard drive images are literally a qemu image for linux kvm that's been qemu-img converted to a raw format, then the kernel and initrd were extracted, and stored in the mac file system. This image was originally on my linux box for a while, before I needed to go mobile on the laptop with it.
ymmv, of course.
https://github.com/tithomas1/vdhh
Wish I could use it, but the current implementation calls the xgetbv instruction which is not available on my Westemere EP processors.
All MacOS/QEMU needs is PCIE passthrough and you could have an amazing setup with thunderbolt accessories.
It was not a Summer of Code project, there was (is) a company called Veertu that did the work who sold a virtualization product for running Linux and Windows on macOS on the App Store. Because it used Hypervisor.framework, it could be sandboxed and could be on the Mac App Store.
Google forked Veertu's GitHub repo, adding a few bells and whistles for use in their Android emulator.
Finally, the work was merged upstream to QEMU.
However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System. As far as I know, macOS has a newly-added but slowly-performed file system, named APFS. It might be good enough for single disk request but not good for the concurrent requests (like 4; ; ) -- I am not questioning/blaming about the old port design of either xhyve, OS X, or even hardware architecture SCSI, but that's where we can start now.
Anyway, there is a newly-adopted storage technology named NVME emulation which might help a lot in your case -- not only it handles with lots of disk requests at the same time (65535 queues and 65535 commands per queue), but also the architecture makes it possible to program the controller code in software, especially in user world. I don't know all details of NVMe but it works well in my experience -- both of consumer level and enterprise level -- it brings smooth disk latency in the virtual machine even under a very high bandwidth (50Gib transfer in nearly half a minute, can you believe that?). Do you know if it is possible to use NVMe emulation for faster disk access in xhyve and accelerate the PostgreSQL execution natively?
- SCSI: https://en.wikipedia.org/wiki/SCSI - NVME: https://en.wikipedia.org/wiki/NVM_Express
I assume this is to build a development environment. Especially if you're working on a laptop, on a client-server app that needs to talk to a Postgress (or other) database, running the database in a VM so it goes where you go is a pretty common setup.
Back to your basements, you lutes, and keep hoping you produce something good enough to be included in glorious OSX, that they might take it and in return give you nothing except for the knowledge that you, a mere programmer, have managed to contribute to mighty Apple's bottom-line.
But yeah, that it would be pretty sweet, even though it will never happen. For Apple software is about taking control, not giving it.
If you're using the command line, FreeBSD has far better and far more consistent documentation.
I have a post-install script I always run that does sudo rm -rf on all the apps I don't like. Has no ill effects in all the years I have been using it, and makes my system feel clean.
https://github.com/Ozymandias42/xhyve-manager