In this space it's not too unusual, really. It's not altogether unlike modern AIX or IBM i, which effectively only run on virtual machines managed by a hypervisor built into the firmware on PowerVM machines. Maybe less satisfying since it's less integrated with the hardware, but in a sense the paravirtual devices are kind of just another type of firmware/driver layer.
I kind of hope more (new) OSes adopt a similar model. Developing cool new experimental OSes would be easier if real hardware could be "ignored" in favor of just targeting virtio interfaces or something, and IMO that's where the most interesting OS dev is yet to happen. If you're mostly interested in hardware support and raw performance, I think Linux has "done it". But there's a lot left to explore, and maybe it's best to find a way to get Linux to do the heavy lifting.
> If you're mostly interested in hardware support and raw performance, I think Linux has "done it".
There's alot of room for adjustments/experimentation within the posix-like OS space; Linux is not always the best vehicle for this experimentation (witness various threading/ipc/scheduling differences with others, security protections, etc. that a) exist and/or b) have been imported/emulated in Linux )
I thought about this a while after this, and yeah, I'd agree. Although as far as scheduling and IPC goes, I'd say this is less about "raw performance" as taking a chunk of CPU cycles and better utilizing it. So a VM running under Linux/KVM could experiment there in useful ways.
There are still new/tweaked ways to manage/divide raw hardware, though. Stuff like trying to reduce UI latency which would be hard to do through a PV interface and needs bare metal.
> but there's something unsatisfying about an O/S that can't boot real hardware
According to their roadmap [0] they are aiming to release support for running on HPE DL380 servers bare metal this year, followed by support for some 2 socket Dell servers next year.
The current x86 releases are in a rather raw state, e.g. you still have to cross compile x86 on an Itanium system.
Based on some claims I have seen elsewhere, they plan to release the V9.1 to non-commercial customers, which is meant to happen some time before the end of H1 of this year.
If x86 refers to 32 bit, what family do the 8086/88/186/286 belong to? 8080_16? :-)
I do suspect as time goes by x86 is just becoming more synonymous with the 64 bit variant, the 32 bit and 16 bit modes are increasingly irrelevant to anyone except boot ROM authors and retro enthusiasts.
As mentioned above by someone else, what they mean by "x86” is something of a hybrid. The kernel/ring 0 code is certainly 64 bit, and I suspect they may be using 64 bit long mode everywhere, but they're making special accomodations to the many VMS programs that assume 32 bit pointers.
This isn't relevant just to VAX code -- because previous versions of VMS made these same accomodations, some programs simply never changed, up to Itanium, despite both alpha and Itanium being 64 bit (and only 64 bit, alpha didn't and I don't think Itanium had "true" 32 bit modes)
So "x86” is really referring more to the platform than the specific CPU or mode here, since it'll be a pretty seamless mixture of both.
I have never seen anyone refer to x86 as a synonym for the 64-bit ISA. In the Linux world, it's amd64, in the Windows world it's x64/64-bit, and sometimes in both of those worlds, x86_86. I don't think this loss of distinction is occurring at all.
What they did is more complex than that – the OS is 64-bit, but a lot of legacy apps use 32-bit pointers. All code runs in long mode, but to support those legacy apps, they put code and static data in first 4GB of the address space. 64-bit-clean applications can allocate data above the 4GB mark. 32-bit applications actually run in 64-bit mode, but they assume the top 32-bits of pointers have a fixed value, which means they can safely stash the pointer into a 4 byte memory area.
The other thing they do, is put code in the full 64-bit address space, but then have the linker generate trampolines in first 32-bits which jump to the actual function location so 32-bit function pointers can be used for code living outside the 32-bit address space.
I believe VMS on Alpha and Itanium did similar things. Applications on VAX assumed 32-bit addresses. Even though Alpha was a 64-bit architecture, to make it easy to port apps from VAX, they had applications mostly use 32-bit pointers (which are really 64-bit pointers with half of their value fixed), if you need more memory, you have to change your application to call some new APIs which return arbitrary 64-bit pointers. And then the same setup was maintained when they ported to Itanium, and now x86-64.
Actually IBM z/OS does something similar. IBM mainframes originally used 24-bit addressing, in the early 1980s they moved to 31-bit addressing (not 32-bit, the MSB was used as a flag bit), then around 2000 they moved to 64-bit. However, even in a 64-bit process, they still have the rule that all code has to be in the first 2GB of memory, to simplify interoperation between 31-bit and 64-bit code. But even though all code pointers must be safely expressible in 31-bits, 64-bit applications can use the full 64-bit address space for data.
Not quite the same thing. In x32, processes use 32-bit pointers, and can't use 64-bit addressing. An x32 application can't allocate a 8GB memory buffer. Unlike x32, 64-bit VMS lets you mix 32-bit-compatible and 64-bit addressing in the same process. VMS has two sets of memory allocation routines – _malloc32 and _malloc64 – you call the former to get 32-bit compatible pointers, the later gives you the full power of 64-bit addressing. Your app could be a mix of legacy code (which only supports 32-bit) and 64-bit code running in the same process. You want an 8GB memory buffer, _malloc64 will give you one (assuming there is available system resources, of course).
So I’m totally unfamiliar with x32 but say that there is all kinds of 32bit apps that are just too big for a single person to port to 64bit (or simply too big too old) is this a silver bullet of being able to rebuild 32bit apps to run as 64bit ones?!
Seems like the kind of thing that OS X desperately needed...
On x86-64, Linux supports legacy 32-bit apps (i386 ABI) and native 64-bit apps (x86-64 ABI). However, the x32 ABI isn't either of those. x32 is actually a hybrid 32-bit/64-bit architecture, that lets you use some of the newer features of x86-64 (such as increased number of registers) while keeping 32-bit memory addresses. The theory behind it is that for apps that don't really need a 64-bit address space, 32-bit pointers consume less RAM than 64-bit ones. Using less RAM can have flow-on performance benefits – you fit more in the CPU cache, etc.
While x32 is a nice idea in theory, it never really took-off. It is adding a lot of complexity (a third architecture for distributions to support) in exchange for merely modest performance improvements. And it has nothing to do with backward compatibility – that is the role of the i386 ABI, not the x32 ABI. i386 ABI gives binary backward compatibility, x32 ABI requires a recompile. It might be a little bit easier to port from i386 to x32 than from i386 to x86-64, given the pointers remain 32-bit; however, in practice, people either just stick with the existing i386 ABI binaries, or skip it and make the code changes for 64-bit.
> Seems like the kind of thing that OS X desperately needed...
Apple didn't have to discontinue 32-bit application support for macOS. They made a commercial decision that doing so would save on engineering resources. It ultimately had a business/financial motivation not a purely technical one.
It possibly made their task of implementing x86 emulation on ARM somewhat easier. However, they always could have either (a) invested more to support emulating 32-bit x86 on ARM as well (b) limited x86 emulation on ARM to 64-bit but continue supporting 32-bit on native x86. The fact that they didn't do either (a) or (b) – again, ultimately a business/commercial decision about where to invest, not a technical one.
(And the truth is, 32-bit application support isn't completely discontinued. x86-64 macOS still supports running 32-bit code, as demonstrated by the Codeweavers branch of Wine. They removed the 32-bit libraries from the OS, and blocked the execution of 32-bit Mach-O executables, but there still exists an API which applications like Wine can use to manually set up a 32-bit address space and run some 32-bit code in it.)
The code weaver approach doesn’t sound like something like hat is accessible to small time people... did they have to build their own thunking layers?
Sounds like it’s easier to rebuild as a Win32 and have OS X users use wine then.
I know a company the size of Apple can’t be expected to be some kind of stable dev platform like how Microsoft treats Windows where I can still run Win32 stuff from 1993 on Windows 10 x64. Maybe one day Apple might take an interest in the desktop again but it seems so unlikely.
> The code weaver approach doesn’t sound like something like hat is accessible to small time people... did they have to build their own thunking layers?
It wasn't easy, no. They had to do two things:
1) Fork clang and extensions to C to support mixed 32-bit and 64-bit address spaces
2) Get a special code-signing entitlement from Apple allowing them to call the API to enable 32-bit code execution (not required if you are okay with disabling SIP)
VMS is / was popular in the transaction processing industry: financials, lottery, casino, etc. However, most of the folks I know working in that industry moved on to other platforms over a decade ago.
I have an AlphaStation as part of my retro collection. It has OpenVMS on it, but hasn't been booted in a while.
A university I worked at migrated from independent VMS systems (student, finance, HR) to an ERP system. To make that implementation easier, they only converted the most recent few years of data.
Last I heard they had some sort of virtual machines emulating Alpha hardware still running those legacy systems.
My current employer (also higher ed.) is hosting somebody's ES40. Some people in higher ed. really, really liked VMS and the Alpha/Itanium platforms.
I haven't worked there for a few years, but I'm 99% sure that German stock exchange in Frankfurt (https://www.deutsche-boerse.com/dbg-en) is still operating their clearing operations (central counterparty) and some others on OpenVMS/Itanium servers from HP
If you're dealing with anything Apple you're probably running macOS VMs on Apple hardware. At least for things like build servers, signing and such (CI/CD) when the projects start growing.
I’ll always have a soft spot for VMS, the first “big computer” OS I ever used. And perhaps did a bit of hacking (allegedly) back in 1990.
Did the hobbyist licensing [1] get resolved? I was inspired by the work of Andrew Back [2] getting VMS clusters working on Raspberry Pis.
The fact that it’s still around today is a testament to our purest technologist passions. I’m old enough to remember how we though the (open) web would be (creating one of the first 100 websites), and now curmudgeony enough to complain about what it’s become. Come on, we all know it’s at least a bit shit. But also awesome.
I guess I was lucky enough to be around VMS, SunOS and NextSTEP in a 12 month period and filled with the optimism that the intersection of them, with some of that all seen mostly via an Amiga interface, could become. Man, they were the days.
I think that even with all the pressures put upon the creative devs, product folks, and hard-working ops and support folks that we’re all driving towards what’s best for our organisations - despite the mediocre early 20 MBAs and managers who’ve have no formal training.
Apologies for the extra episode of HCF but in those days it was more exciting than helping folks with cloud, IAC, etc.
But I’m just a grumpy ops bloke. Off to write something in BASIC on my Spectrum and remember why computers are fun.
Same here.. The first multi-user system I had access to was a VAX running VMS. I think the second was a SunOS 4.x box. And I did my BBS / telecom stuff on an Amiga. 90's nostalgia is the best.
> Did the hobbyist licensing [1] get resolved? I was inspired by the work of Andrew Back [2] getting VMS clusters working on Raspberry Pis.
Not sure which issue you are referring to specifically, but VSI provides hobbyist licenses for Alpha and Itanium, and plans to provide them for x86.
Unfortunately, there will be no hobbyist licenses for VAX in future since VSI can't redistribute DEC/Compaq/HP versions of VMS, and they have no plans to make their own VAX version (HP killed VAX support nearly a decade ago). Now the only legal way is to try and buy a VAX license from someone who is still selling them.
Where can we get the download image for Virtual Box? Would like to try my hand at it. Only tried it once one afternoon in 1996 or so. Made me laugh when CDE came up, because I had previously tried it on a Slackware install I had at the time.
43 comments
[ 3.1 ms ] story [ 95.9 ms ] threadhttps://vmssoftware.com/community/community-license/
I guess I shouldn't be surprised that this is a VM-only product, but there's something unsatisfying about an O/S that can't boot real hardware.
I kind of hope more (new) OSes adopt a similar model. Developing cool new experimental OSes would be easier if real hardware could be "ignored" in favor of just targeting virtio interfaces or something, and IMO that's where the most interesting OS dev is yet to happen. If you're mostly interested in hardware support and raw performance, I think Linux has "done it". But there's a lot left to explore, and maybe it's best to find a way to get Linux to do the heavy lifting.
There's alot of room for adjustments/experimentation within the posix-like OS space; Linux is not always the best vehicle for this experimentation (witness various threading/ipc/scheduling differences with others, security protections, etc. that a) exist and/or b) have been imported/emulated in Linux )
There are still new/tweaked ways to manage/divide raw hardware, though. Stuff like trying to reduce UI latency which would be hard to do through a PV interface and needs bare metal.
According to their roadmap [0] they are aiming to release support for running on HPE DL380 servers bare metal this year, followed by support for some 2 socket Dell servers next year.
[0] https://vmssoftware.com/about/roadmap/
Based on some claims I have seen elsewhere, they plan to release the V9.1 to non-commercial customers, which is meant to happen some time before the end of H1 of this year.
"Since the stack and static data on x86 is still allocated in 32-bit address space, a 32-bit pointer is sufficient to point to them."
It seems they're still expecting a lot of VAX users to migrate.
I do suspect as time goes by x86 is just becoming more synonymous with the 64 bit variant, the 32 bit and 16 bit modes are increasingly irrelevant to anyone except boot ROM authors and retro enthusiasts.
This isn't relevant just to VAX code -- because previous versions of VMS made these same accomodations, some programs simply never changed, up to Itanium, despite both alpha and Itanium being 64 bit (and only 64 bit, alpha didn't and I don't think Itanium had "true" 32 bit modes)
So "x86” is really referring more to the platform than the specific CPU or mode here, since it'll be a pretty seamless mixture of both.
The other thing they do, is put code in the full 64-bit address space, but then have the linker generate trampolines in first 32-bits which jump to the actual function location so 32-bit function pointers can be used for code living outside the 32-bit address space.
I believe VMS on Alpha and Itanium did similar things. Applications on VAX assumed 32-bit addresses. Even though Alpha was a 64-bit architecture, to make it easy to port apps from VAX, they had applications mostly use 32-bit pointers (which are really 64-bit pointers with half of their value fixed), if you need more memory, you have to change your application to call some new APIs which return arbitrary 64-bit pointers. And then the same setup was maintained when they ported to Itanium, and now x86-64.
Actually IBM z/OS does something similar. IBM mainframes originally used 24-bit addressing, in the early 1980s they moved to 31-bit addressing (not 32-bit, the MSB was used as a flag bit), then around 2000 they moved to 64-bit. However, even in a 64-bit process, they still have the rule that all code has to be in the first 2GB of memory, to simplify interoperation between 31-bit and 64-bit code. But even though all code pointers must be safely expressible in 31-bits, 64-bit applications can use the full 64-bit address space for data.
Seems like the kind of thing that OS X desperately needed...
On x86-64, Linux supports legacy 32-bit apps (i386 ABI) and native 64-bit apps (x86-64 ABI). However, the x32 ABI isn't either of those. x32 is actually a hybrid 32-bit/64-bit architecture, that lets you use some of the newer features of x86-64 (such as increased number of registers) while keeping 32-bit memory addresses. The theory behind it is that for apps that don't really need a 64-bit address space, 32-bit pointers consume less RAM than 64-bit ones. Using less RAM can have flow-on performance benefits – you fit more in the CPU cache, etc.
While x32 is a nice idea in theory, it never really took-off. It is adding a lot of complexity (a third architecture for distributions to support) in exchange for merely modest performance improvements. And it has nothing to do with backward compatibility – that is the role of the i386 ABI, not the x32 ABI. i386 ABI gives binary backward compatibility, x32 ABI requires a recompile. It might be a little bit easier to port from i386 to x32 than from i386 to x86-64, given the pointers remain 32-bit; however, in practice, people either just stick with the existing i386 ABI binaries, or skip it and make the code changes for 64-bit.
> Seems like the kind of thing that OS X desperately needed...
Apple didn't have to discontinue 32-bit application support for macOS. They made a commercial decision that doing so would save on engineering resources. It ultimately had a business/financial motivation not a purely technical one.
It possibly made their task of implementing x86 emulation on ARM somewhat easier. However, they always could have either (a) invested more to support emulating 32-bit x86 on ARM as well (b) limited x86 emulation on ARM to 64-bit but continue supporting 32-bit on native x86. The fact that they didn't do either (a) or (b) – again, ultimately a business/commercial decision about where to invest, not a technical one.
(And the truth is, 32-bit application support isn't completely discontinued. x86-64 macOS still supports running 32-bit code, as demonstrated by the Codeweavers branch of Wine. They removed the 32-bit libraries from the OS, and blocked the execution of 32-bit Mach-O executables, but there still exists an API which applications like Wine can use to manually set up a 32-bit address space and run some 32-bit code in it.)
Sounds like it’s easier to rebuild as a Win32 and have OS X users use wine then.
I know a company the size of Apple can’t be expected to be some kind of stable dev platform like how Microsoft treats Windows where I can still run Win32 stuff from 1993 on Windows 10 x64. Maybe one day Apple might take an interest in the desktop again but it seems so unlikely.
It wasn't easy, no. They had to do two things:
1) Fork clang and extensions to C to support mixed 32-bit and 64-bit address spaces
2) Get a special code-signing entitlement from Apple allowing them to call the API to enable 32-bit code execution (not required if you are okay with disabling SIP)
Lots of technical details here: https://www.winehq.org/pipermail/wine-devel/2019-December/15...
Yeah, this is a monumental effort that few other than Codeweavers could replicate, and I believe they got some help from Apple engineering to do it.
They've open sourced their code, but they aren't upstreaming it into mainstream Wine – having to use a forked C compiler is just a bit too much.
I have an AlphaStation as part of my retro collection. It has OpenVMS on it, but hasn't been booted in a while.
Last I heard they had some sort of virtual machines emulating Alpha hardware still running those legacy systems.
My current employer (also higher ed.) is hosting somebody's ES40. Some people in higher ed. really, really liked VMS and the Alpha/Itanium platforms.
https://en.wikipedia.org/wiki/OpenVMS
One of those services, millions use it everyday without knowing it.
Several pieces described in that patent still run on VMS in 2021
[1] https://docplayer.net/48457150-I303-physical-to-virtual-serv...
https://vmssoftware.com/products/decwindows-motif/
Did the hobbyist licensing [1] get resolved? I was inspired by the work of Andrew Back [2] getting VMS clusters working on Raspberry Pis.
The fact that it’s still around today is a testament to our purest technologist passions. I’m old enough to remember how we though the (open) web would be (creating one of the first 100 websites), and now curmudgeony enough to complain about what it’s become. Come on, we all know it’s at least a bit shit. But also awesome.
I guess I was lucky enough to be around VMS, SunOS and NextSTEP in a 12 month period and filled with the optimism that the intersection of them, with some of that all seen mostly via an Amiga interface, could become. Man, they were the days.
I think that even with all the pressures put upon the creative devs, product folks, and hard-working ops and support folks that we’re all driving towards what’s best for our organisations - despite the mediocre early 20 MBAs and managers who’ve have no formal training.
Apologies for the extra episode of HCF but in those days it was more exciting than helping folks with cloud, IAC, etc.
But I’m just a grumpy ops bloke. Off to write something in BASIC on my Spectrum and remember why computers are fun.
[1] http://www.openvmshobbyist.com/news.php [2] https://web.archive.org/web/20190620014639/https://www.rs-on...
[EDIT[ typo “produce folks” and “product folks” are completely different!
Not sure which issue you are referring to specifically, but VSI provides hobbyist licenses for Alpha and Itanium, and plans to provide them for x86.
Unfortunately, there will be no hobbyist licenses for VAX in future since VSI can't redistribute DEC/Compaq/HP versions of VMS, and they have no plans to make their own VAX version (HP killed VAX support nearly a decade ago). Now the only legal way is to try and buy a VAX license from someone who is still selling them.