Thanks for creating this, it looks fantastic, I just bought it! I love that it's a nice layer over QEMU / Apple's Hypervisor, so i know the performance will be reasonable. One bit of feedback you might consider: Most of the VMs in your gallery are for ARM64, and the x86 one is quite an old version of Ubuntu. Would you consider making a couple more modern x86 versions? My requirement is to have an x86 VM for deployments to Lambda, cross compiling Rust using Docker on M1 simply doesn't work yet and I'm hoping your app will do the trick :) Thanks!
I have the same requirement, but ended up building the final bundles to lambda in the cloud instead. However I am quite excited about the new container image support for lambda meaning I guess you do not have to compile for musl anymore.
This is what I use to run QEMU on my M1 Air. I had a few bumps setting it up but now it runs flawlessly. I SSH to it via VS Code and basically I have a fully functional Linux dev environment with MacOS as the UI, just the way I like it.
Honestly, I haven't been this excited about a new setup since Moore's Law was in effect :-)
QEMU is amazing. I run Linux, ans being able to dockerize Windows with it, and set up a selenium testing environment I can run locally or on CI servers is genuinely mind-blowing. Headless chrome, Firefox plus a windows in docker setup means pretty great automated test coverage without relying on third party infra.
It (QEMU) actually uses the older https://developer.apple.com/documentation/hypervisor apis which has lower level control. QEMU provides device emulation and Hypervisor.framework provides access to CPU’s virtualization hardware. Virtualization.framework is the new macOS 11 api that emulates the entire system but has much worse device emulation support and also lacks framebuffer emulation (I think)
I do this except the Linux is on a separate box and the dev envs are docker containers.
Decided to spend my money on decent, upgradeable hardware instead of shelling out 5k for another MB Pro. My next MacBook will probably be the most low-end I can get.
VSCode's remote/docker support is a godsend. Seems like we're heading back to the dumb terminal paradigm.
I have a Dell R720 that I bought fairly cheaply ~$600 came with 192GB of RAM. I shoved some SSDs in it with a GPU. Sits in my garage. If I'm not home I can still remote into my terminal via wireguard. Good deals can be found using lab gopher:
I saw some VPS providers offer hourly billing, which can get me a 64cores + 64gb ram at like $30 a month. I'm going to go with that. I actually don't have landline/fiber internet at home so your solution wouldn't work for me, but I'll keep it in mind I didn't know about such deals!
Do you have your projects cloned in the remote host?
I ask because I was excited about this prospect only to learn that I could only use Remote: Containers if codebase and container are on the same machine as the project workspace is just volume-mounted in the container.
Yeah I dusted off an old licence I had for resilio sync and use that to sync with my local machine.
You can get around this by running git clone inside your container as part of the startup process. But 1. It can add quite a delay to starting a container depending on the size of your codebase, and 2. Not all of my code is necessarily committed and pushed.
For me, syncing works well but I can see how it would not suit every use case
Fair enough. I don't want to clone from within the container because I like the idea of being able to nuke it at any time and just start over from local code.
Probably similar reasons to what you mean by not having all code committed and pushed.
Oh very interesting. I only need Vagrant support and then I can fully switch to M1 in the future. Looks like this could be the successor for Virtualbox on Mac M1.
Wonder if this also helps developing for x86 Docker too (which is a major pain point on M1 Mac).
Do you mean the emulation performance on M1? In my experience Windows XP is “usable” and Windows 7 feels like a 10 year old desktop running with a spinning drive.
Thats interesting. But it seems like the AltStore requires running a AltServer which is only available for Mac or Windows. It would be great if they provided a Linux build too.
It is kind of ironic though that I need to bring up a QEMU VM to run the AltStore binary which lets me run QEMU on my phone :D
This is a very nice frontend. Someone should definitely port it or rewrite it for Linux. All the existing UIs are either ugly or use libvirt, which is a nasty pile of hacks that has no place on a desktop machine.
Thanks but I think a port is virtually impossible as this was designed in SwiftUI and depends extensively on Apple APIs. GNOME Boxes is a good UI on Linux.
libvirt has put a lot of time into figuring out how to run QEMU in the most secure and performant way (specific low-privilege user, seccomp, all the right options for virtio, etc, etc). If you aren't using it then that's a lot of effort to reinvent, and you probably missed something out.
libvirt wasted a lot of effort into abstracting everything behind a ton of pointless hypervisor-agnostic cruft, only to then deprecate basically every single backend except QEMU.
I have used libvirt for almost a decade before going back to plain QEMU; I must say that using it from the CLI is an herculean task, you often have to edit plain XML in order to do whatever you want, and the cherry on top is that `virsh`is a huge pile of cow dung that often drags you into a Sisyphean fight against the aforementioned pointless abstractions - you find yourself configuring "domains" and "virtual networks" when you really only wanted to launch Windows to run Word.
When I had to run stuff on a big server which only does VM, maybe with OpenStack and other orchestration systems on top, libvirt was often _fine_. I still use it in those use cases (I usually prefer Proxmox this day, though), it works fine, I can connect to remote instances using SSH and Virt-Manager and I like how it often just works.
My use case, 90% of the time, is that I just want to run a damn Windows or FreeBSD VM on my workstation. I want to put disk images wherever I want because my root FS is small (often on ZVOLs, which are a big massive PITA to use with libvirt) without having to read a million different websites and man pages. I want it to work with my firewall configuration, which often changes due to the fact I have to do stuff with this machine, and I don't really care about 90% of the features of libvirt - I just want to run Windows every once in a while and I want my VMs to keep working without too much effort.
Thus, I run plain QEMU with the following configuration, which took me a solid 10 minutes to perfect:
- `nsbr0`, a bridge with a static IP configured using systemd-network;
- `qemu-bridge-helper` set-up in order to attach taps to nsbr0 automatically, using `-netdev bridge` (echo 'allow nsbr0' > /etc/qemu/bridge.conf);
- IP forwarding set to 1 and `masquerade` in nft.conf;
- OVMF downloaded from my distribution's repos;
- `spicy` to attach to a running VM;
- a 30 line script that spawns QEMU in a tmux. If I want to configure a new VM, I just copy paste this shell script and I put it in a directory together with a copy of OVMF_VARS.fd for the efivars.
Et voilà, I get VMs that just work, without having to wage any pointless battle against libvirt in order to get what I want. This is basically just a way to run KVM VMs that does not create a million firewall rules together with useless network interfaces, and doesn't force me to configure "storage pools" when the only thing I care about is running 2, 3 VMs max on my machine.
There's a reason why VMWare sells both ESXi and VMWare Workstation, and why people still love VirtualBox despite how shitty it is.
It's because more often than not something designed to run on servers is hardly be the best solution for a single user. The same features that make libvirt interesting on servers make it clumsy and overcomplicated on workstations, and the more I use it the more I realize it's true.
It's a type-2 hypervisor, so its performance is often quite worse than KVM. Also, its storage drivers are quite wonky, so I/O is often choppy and underwhelming. It has often a better desktop experience, though, except when you run Linux with VirGL. Also, virtualbox cannot emulate other platforms; for instance, I often run AArch64 operating systems in full emulation using QEMU on my x86_64 workstation.
No, KVM (and FreeBSD's bhyve) are kernel modules that for all intents and purposes transform the host kernel into a type 1 hypervisor itself. The hypervisor is not, like in the case of VrtualBox or VMware workstation, running on top of the kernel, it is the kernel itself. The VMs spawned on KVM or bhyve have potentially full access to any hardware capabilities exposed to them. You can passthrough PCI devices to KVM VMs, for instance, using VFIO; you can also fully expose the host CPU allowing nested virtualization
A "true" type-2 hypervisor does not require any modification to the host kernel; at max it ships with a kernel module or two to help implementing things such as custom networking or to get a speed boost kernel side. For instance, it's trivial to share USBs with a guest on the fly with VirtualBox, because the VM is basically a process running on the host, so it can just tap into the USB stack and expose the device to the VM. This is not so simple with KVM.
Yeah I didn’t say ideal. But if you are on Win/Mac (there are a lot of those people) and need a Linux box without much fuss it’s a good go-to solution.
LXD recently added support for QEMU virtual machines in addition to its existing support for LXC containers, and it's almost completely replaced libvirt for me. It's great, pretty much just a matter of adding a `--vm` flag to normal `lxc launch` usage, e.g. `lxc launch images:debian/buster --vm`.
Yes, but the main reason I use VMs is to run different OSes. When I need to run other linux distros I normally use systemd-nspawn (or rarely LXC) and it's enough.
AFAIK you can't, and there's no indication that will change. You may be able to virtualize a whole macOS install, though that's pretty difficult even on Apple hardware (source: I tried several times, and didn't get it to work well)
I think many open-source efforts are heading in the wrong direction. Let Apple make sure our software runs on their hardware, and let FOSS make sure that Apple software runs on FOSS platforms. Not the other way around.
Be careful, I think you need something like 50-60GB of free space to run it. And the script does not check the free space beforehand.
After the script is run, I think the OS X VM will occupy at least 25GB.
PS: the graphics performance is ok, but not great. I could not manage to install the extension package of VirtualBox, so may be if you can install it, then the graphics performance of the virtualized OSX will improve.
afaik there is nothing preventing this from being on the mac app store.
The iPhone app store is so strict because they really want to keep the platform locked down, with no possibility for 3rd party app stores. Although even then there's weird exceptions - Roblox seems to be allowed.
That cat is out of the bag on MacOS, though, and at least for now it seems Apple is keeping it that way - iPadOS is going to be their locked down work OS.
When the Debian install asks you what device to use, give it /dev/sda or /dev/vda. For some reason /dev/cdrom wasn't working for me either. I ended up switching the ISO's device to virtio and tried /dev/vda and it worked.
The ISO was on /dev/vda and the target disk image was on /dev/vdb. Just make sure you install the bootloader on the disk image. Then when I removed the ISO, everything was just fine.
Technically the AArch64 instruction set and the memory system are the only parts that are 'hardware accelerated'. QEMU handles everything else in software.
$ nm /Applications/UTM.app/Contents/Frameworks/libqemu-aarch64-softmmu.utm.dylib | grep -i hv_
U _hv_vcpu_create
U _hv_vcpu_destroy
U _hv_vcpu_get_reg
U _hv_vcpu_get_sys_reg
U _hv_vcpu_run
U _hv_vcpu_set_pending_interrupt
U _hv_vcpu_set_reg
U _hv_vcpu_set_sys_reg
U _hv_vcpu_set_vtimer_mask
U _hv_vcpus_exit
U _hv_vm_create
U _hv_vm_map
U _hv_vm_protect
U _hv_vm_unmap
Thank you for making this available for download from the website too, and not making it app store exclusive. Do you have any plans to support macOS Mojave? What is needed to add GPU support in the future?
Wondering if there could be a Metal -> Vulkan -> Zink -> OpenGL way of doing HW acceleration.
Or Metal -> Vulkan -> DXVK -> D3D for Windows.
AFAIK VirtualBox used some of Wine's D3D code to do D3D on Virtualbox.
124 comments
[ 1.9 ms ] story [ 108 ms ] threadAnd it shows, the UI is beautiful. Will try this out.
And, lovely work on the UI!
This is what I use to run QEMU on my M1 Air. I had a few bumps setting it up but now it runs flawlessly. I SSH to it via VS Code and basically I have a fully functional Linux dev environment with MacOS as the UI, just the way I like it.
Honestly, I haven't been this excited about a new setup since Moore's Law was in effect :-)
Decided to spend my money on decent, upgradeable hardware instead of shelling out 5k for another MB Pro. My next MacBook will probably be the most low-end I can get.
VSCode's remote/docker support is a godsend. Seems like we're heading back to the dumb terminal paradigm.
Like a normal VPS but you can get dat 64 cores + 64gb ram only the day you need them and otherwise pay a low monthly fee.
https://labgopher.com/servers/Dell_R720/
Granted it's not 64cores, but still fast.
In this case it really is more than thousands of times more powerful.
I ask because I was excited about this prospect only to learn that I could only use Remote: Containers if codebase and container are on the same machine as the project workspace is just volume-mounted in the container.
You can get around this by running git clone inside your container as part of the startup process. But 1. It can add quite a delay to starting a container depending on the size of your codebase, and 2. Not all of my code is necessarily committed and pushed.
For me, syncing works well but I can see how it would not suit every use case
Probably similar reasons to what you mean by not having all code committed and pushed.
I wasn't aware of that setup you linked. I'll give that a go, thanks!
Hehem. The paradigm you're referencing is letting you do what you want to do (because it's a powerful paradigm). We need more of that if anything.
Wonder if this also helps developing for x86 Docker too (which is a major pain point on M1 Mac).
There is no hardware assisted virtualization for x86 on ARM.
Oh well.
It is kind of ironic though that I need to bring up a QEMU VM to run the AltStore binary which lets me run QEMU on my phone :D
I have used libvirt for almost a decade before going back to plain QEMU; I must say that using it from the CLI is an herculean task, you often have to edit plain XML in order to do whatever you want, and the cherry on top is that `virsh`is a huge pile of cow dung that often drags you into a Sisyphean fight against the aforementioned pointless abstractions - you find yourself configuring "domains" and "virtual networks" when you really only wanted to launch Windows to run Word.
When I had to run stuff on a big server which only does VM, maybe with OpenStack and other orchestration systems on top, libvirt was often _fine_. I still use it in those use cases (I usually prefer Proxmox this day, though), it works fine, I can connect to remote instances using SSH and Virt-Manager and I like how it often just works.
My use case, 90% of the time, is that I just want to run a damn Windows or FreeBSD VM on my workstation. I want to put disk images wherever I want because my root FS is small (often on ZVOLs, which are a big massive PITA to use with libvirt) without having to read a million different websites and man pages. I want it to work with my firewall configuration, which often changes due to the fact I have to do stuff with this machine, and I don't really care about 90% of the features of libvirt - I just want to run Windows every once in a while and I want my VMs to keep working without too much effort.
Thus, I run plain QEMU with the following configuration, which took me a solid 10 minutes to perfect:
- `nsbr0`, a bridge with a static IP configured using systemd-network;
- `qemu-bridge-helper` set-up in order to attach taps to nsbr0 automatically, using `-netdev bridge` (echo 'allow nsbr0' > /etc/qemu/bridge.conf);
- IP forwarding set to 1 and `masquerade` in nft.conf;
- OVMF downloaded from my distribution's repos;
- `spicy` to attach to a running VM;
- a 30 line script that spawns QEMU in a tmux. If I want to configure a new VM, I just copy paste this shell script and I put it in a directory together with a copy of OVMF_VARS.fd for the efivars.
Et voilà, I get VMs that just work, without having to wage any pointless battle against libvirt in order to get what I want. This is basically just a way to run KVM VMs that does not create a million firewall rules together with useless network interfaces, and doesn't force me to configure "storage pools" when the only thing I care about is running 2, 3 VMs max on my machine.
There's a reason why VMWare sells both ESXi and VMWare Workstation, and why people still love VirtualBox despite how shitty it is.
It's because more often than not something designed to run on servers is hardly be the best solution for a single user. The same features that make libvirt interesting on servers make it clumsy and overcomplicated on workstations, and the more I use it the more I realize it's true.
A "true" type-2 hypervisor does not require any modification to the host kernel; at max it ships with a kernel module or two to help implementing things such as custom networking or to get a speed boost kernel side. For instance, it's trivial to share USBs with a guest on the fly with VirtualBox, because the VM is basically a process running on the host, so it can just tap into the USB stack and expose the device to the VM. This is not so simple with KVM.
https://www.darlinghq.org/
PS: the graphics performance is ok, but not great. I could not manage to install the extension package of VirtualBox, so may be if you can install it, then the graphics performance of the virtualized OSX will improve.
The iPhone app store is so strict because they really want to keep the platform locked down, with no possibility for 3rd party app stores. Although even then there's weird exceptions - Roblox seems to be allowed.
That cat is out of the bag on MacOS, though, and at least for now it seems Apple is keeping it that way - iPadOS is going to be their locked down work OS.
https://apple.stackexchange.com/questions/6109/is-it-possibl...
It's possible that those replies are out of date and the Mac App Store TOS have changed.
I will totally pay for this. Tired of paying yearly subscriptions to Parallels Desktop Lite which is another frontend to the OS hypervisor.
The ISO was on /dev/vda and the target disk image was on /dev/vdb. Just make sure you install the bootloader on the disk image. Then when I removed the ISO, everything was just fine.
https://developer.apple.com/documentation/hypervisor/apple_s...
https://stackoverflow.com/questions/59636198/how-to-compile-...
Is it currently possible to have snapshots, like in VirtualBox, for instance?