The short version: it's iSCSI targets on the public internet. Pick
an image, get a block device. The free tier doesn't need a signup
at all - iscsiadm -m discovery -t sendtargets -p scsipub.com and
--login to iqn.2025-01.pub.scsipub:blank lands you a 64 MB
scratch disk. There's a small catalog of OS images you can mount
the same way.
The paid tier is where it gets less hobby-shaped: sessions survive
disconnects, a single target can expose multiple LUNs, and SCSI-3
Persistent Reservations work end-to-end (REGISTER / RESERVE /
RELEASE round-trip clean against sg_persist). That last bit is
the cluster-storage primitive — Pacemaker, ESXi HA, and Windows
MSCS all use PR for fencing — so you can actually back a 2-node
failover cluster off a target on the public internet.
The post linked in the submission is the architectural decision
log: Ranch 2.x listeners, a BEAM process per session, COW overlays
with per-sector bitmaps, Caddy-managed Let's Encrypt for the
iSCSI-TLS port without restarting the listener, and the four
open-iscsi quirks that each cost me few hours. There's a section on
what we're deliberately not solving (multi-region, RDMA, etc.)
so you know the scope.
Two companion projects ship as embedded sub-sites on the front
page — one turns an ESP32-S3 into a wireless iSCSI-to-USB bridge,
one lets a Raspberry Pi 3/4/5 netboot directly from a target. Both
linked from the landing page under "Hardware initiators".
Happy to answer any questions about the protocol, the deployment,
or the BEAM-side design choices.
I dislike neg comments but really curious - I can see the how but absolutely clueless about the why. Running a block device over a high latency WAN link seems like a terrible idea, what's the use case?
I wish there was built in iSCSI initiator support on macOS. All of the halfway decent third-party ones either broke many OS versions ago (GlobalSAN) or cost a small fortune ($250 for Atto Xtend)
I think the biggest problem is the various filesystem layers assume the system will be more reliable than it is and suffer. I have a fun weekend experiment running, encrypted remote block storage where I put softraid crypto device(yes I like obsd) on a iscsi target from across the internet. I was vaguely surprised it works at all. But it tends to lock up often as something in the raid layer fails to understand iscsi hiccups.
A fun thought experiment here. Is an encrypted transport layer needed? (ipsec, wiregaurd, tls, etc...) I mean, sure, it probably should be plumbed in I bet a raw iscsi stream exposes a ton of metadata. But does it need to? Would a system designed to encrypt blocks in storage provide protection for those blocks in transit?
8 comments
[ 2.0 ms ] story [ 29.7 ms ] threadThe short version: it's iSCSI targets on the public internet. Pick an image, get a block device. The free tier doesn't need a signup at all - iscsiadm -m discovery -t sendtargets -p scsipub.com and --login to iqn.2025-01.pub.scsipub:blank lands you a 64 MB scratch disk. There's a small catalog of OS images you can mount the same way.
The paid tier is where it gets less hobby-shaped: sessions survive disconnects, a single target can expose multiple LUNs, and SCSI-3 Persistent Reservations work end-to-end (REGISTER / RESERVE / RELEASE round-trip clean against sg_persist). That last bit is the cluster-storage primitive — Pacemaker, ESXi HA, and Windows MSCS all use PR for fencing — so you can actually back a 2-node failover cluster off a target on the public internet.
The post linked in the submission is the architectural decision log: Ranch 2.x listeners, a BEAM process per session, COW overlays with per-sector bitmaps, Caddy-managed Let's Encrypt for the iSCSI-TLS port without restarting the listener, and the four open-iscsi quirks that each cost me few hours. There's a section on what we're deliberately not solving (multi-region, RDMA, etc.) so you know the scope.
Two companion projects ship as embedded sub-sites on the front page — one turns an ESP32-S3 into a wireless iSCSI-to-USB bridge, one lets a Raspberry Pi 3/4/5 netboot directly from a target. Both linked from the landing page under "Hardware initiators".
Happy to answer any questions about the protocol, the deployment, or the BEAM-side design choices.
A fun thought experiment here. Is an encrypted transport layer needed? (ipsec, wiregaurd, tls, etc...) I mean, sure, it probably should be plumbed in I bet a raw iscsi stream exposes a ton of metadata. But does it need to? Would a system designed to encrypt blocks in storage provide protection for those blocks in transit?