Pretty cool! You could also boot into an ephemeral minimal initrd that displays a selection menu instead of doing it in iPXE. That would grab the new kernel and initrd from the network and kexecs it without reboot.
Nice. I'm extra fond of ZFS backed network root filesystem, because it lets you put an OS on ZFS without needing to deal with ZFS support in that OS. (One of these days I want to try OpenBSD with its root on NFS on ZFS, either from Linux or FreeBSD.)
You might find it worth upgrading to 10gbps if you continue to go down this road. The Mikrotik CRS-309 has served me well, and a couple Intel X520-DA2s. I believe those NICs can do iSCSI natively, and pass the session to the operating system with iBFT.
SFP28 might be cheap enough now too, I'm not sure...
I used similar ipxe setup for robotic cluster - every robot booted from the same thing, then kubernetes managed the containe orchestration. it was fun.
I've done a lot of headless/diskless stuff. I haven't done much for years, because my NAS only has gigabit Ethernet ports. I can cascade them and get four Gbps downstream, but it's still painful.
I have recently upgraded my house to 10Gbps Ethernet, with only one room still stuck at gigabit, and unfortunately, it's my main office. I'm working on getting the drop there now (literally, just taking a break here).
Even once I'm done, accessing an iSCSI drive over 10GbE will be 4-8 times slower than a local NVMe drive, but it will sure be a lot better than it was!
Ideally, I could run VMs on the NAS and have great performance, but that's another hardware upgrade...
> UEFI fixes that to some extent, but it’s a pain to maintain the UEFI entries manually and change them every time the kernel updates.
… you don't have to update the UEFI entries every time the kernel updates. (I guess you might if you do like a kernel w/ CONFIG_EFI_STUB, and you place the new kernel under a different filename than what the UEFI boot entry point to then you might … but I was under the impression that that'd be kind of an unusual setup, and I thought most of us booting w/ EFI were doing so with Grub.)
This could be an interesting setup for booting off a NAS like Synology or QNAP. I haven't really used iSCSI, it's intimidating how much prep this takes...
I know it was just a convenient pretext for a learning journey, but do not come away from this thinking llama.cpp needs to be compiled on Windows before use. The GitHib project has a cornucopia of pre-built artifacts to use.
rEFInd is _so_ much simpler: one efi entry, one text config file in the efi partition, nothing that needs to change when the kernel updates, and no massive pile of templating and moving parts to mysteriously break dumping you at an impenetrable grub “rescue” shell.
iSCSI is great and underated, I've been running iSCSI volumes for OrangePI boards which had only SD card support for years. Performance was much better on Gigabit Ethernet vs the SD card.
Heartwarming to see iSCSI still in use! I originally made this account while certifying devices against good ol' RFC 3720 at an interop lab quite a long time ago now...
34 comments
[ 4.5 ms ] story [ 51.4 ms ] threadDoes anyone have an opinion on iSCSI vs NBD?
SFP28 might be cheap enough now too, I'm not sure...
to make this actually work well, consider modifying your switches QoS settings to carve out a priority VLAN for iSCSI traffic
I have recently upgraded my house to 10Gbps Ethernet, with only one room still stuck at gigabit, and unfortunately, it's my main office. I'm working on getting the drop there now (literally, just taking a break here).
Even once I'm done, accessing an iSCSI drive over 10GbE will be 4-8 times slower than a local NVMe drive, but it will sure be a lot better than it was!
Ideally, I could run VMs on the NAS and have great performance, but that's another hardware upgrade...
… you don't have to update the UEFI entries every time the kernel updates. (I guess you might if you do like a kernel w/ CONFIG_EFI_STUB, and you place the new kernel under a different filename than what the UEFI boot entry point to then you might … but I was under the impression that that'd be kind of an unusual setup, and I thought most of us booting w/ EFI were doing so with Grub.)
Hmmh? I haven't done so in years, but configuring multi-boot used to be considerably easier than disk-less operation.
https://github.com/ggml-org/llama.cpp/releases
I have been waiting for such a feature for like 15 years now. Without it, zfs is just a fad and useless filesystem (all that complexity for NOTHING).
ext2 for the win! still
Looks like ZFS is only used to store the image on the server, though. I was expecting this to be more interesting because of that.
rEFInd is _so_ much simpler: one efi entry, one text config file in the efi partition, nothing that needs to change when the kernel updates, and no massive pile of templating and moving parts to mysteriously break dumping you at an impenetrable grub “rescue” shell.
But this idea or ones much like it have been presented before...
"Hassle-free diskless Virtual Machines with Xen and Alpine Linux"
https://jonnytyers.wordpress.com/2016/08/16/hassle-free-disk...
"How to make a Diskless Virtual Machine (KVM)"
https://github.com/lispydev/diskless-kvm
And if you just want a dead easy PXE server for bare metal...
https://www.iventoy.com/en/index.html
* https://en.wikipedia.org/wiki/Open_Firmware