I ran sparc32/64 suns and vaxen under BSD derived os just fine. Symbolic expansion of $arch in nfs mounted filestores to maintain a local/bin path.. no biggie
> There's something to be said about why those architectures, plus many others, died in favor of an x86 monoculture.
I think that has a lot more to do with the economics around Linux, Windows, and macOS on commodity amd64 hardware becoming “good enough” to replace servers and workstations running on other, more expensive, architectures.
Not having to deal with architecture porting issues is just gravy at that point.
This doesn't say anything meaningful. It just states some obvious things with some sort of implied value judgement with no information about how it's supposed to be bad, aside from the author's lack of experience. "you'll almost certainly wind up with a mixed setup". So what? It's not bad just because you say it is if you don't say why it's supposedly bad.
It's also factually incorrect:
"If you try to share things across architectures, you'll get to find all of the ways that modern systems aren't really set up for that, partly because it's been decades since most environments really tried to do this."
What? Did the author give even ONE example? No.
It has never been the case that environments haven't tried to do this. Ever hear of the Internet? It has every architecture you can imagine. If you support 32 bit and 64 bit x86, you're already "sharing things across architectures" because of how different they are.
You appear not to know what is meant by sharing across architectures here.
Chris Siebenmann dates from the times when system administrators would do things like have most of the operating system executables and read-only data files remote-mounted from fileservers. We'd worry about such things as instruction set architectures and bitnessess and endinannesses of non-text file formats. We'd have "architecture-dependent" and "architecture-independent" exports. We'd take care when replacing executable scripts with compiled binaries.
And yes, it has been decades since this was a norm. DASDs have long since become big and cheap enough that just having copies of all of the files locally has long overtaken the idea of having the operating system partly on a remote fileserver that had the limited money spent on it for the really big DASDs.
Microsoft originally had DOS+Windows with a shareable "system" directory and a per-machine "windows" directory. Modern Windows doesn't even incorporate such an idea, not least because many things came along that put per-machine stuff in the "system" directory. Modern Unices and Linux-based operating systems similarly don't normalize the idea of remote-mounted /usr/share and whatnot as Unices did decades ago.
> And yes, it has been decades since this was a norm.
...So why is it relevant?
Sharing files between systems with different architectures Just Works these days, unless you start getting into the really esoteric.
"Having non-x86 machines is Bad, now, today, because of problems I had with managing multiple-architecture deployments 30 years ago" is not a useful statement in any way.
You can use QEMU to do automatic emulation, but I suspect that's not common.
Likely, the binaries are not from the OS, but from users/for users which run on different machines (and more likely than not NFS mounted across the systems to ensure consistency).
I doubt this is related to the OS, more likely user home directories (e.g. people installing conda for x86 would run into issues running on arm), or shared software outside the standard OS setup.
As someone who runs many architectures, I do know what it means. I wasn't aware of the author's experience because Chris didn't offer any details, but that link does offer some clues.
The reality is that there have always been ways to have multiple architectures supported simultaneously, and they're neither all that complicated nor kludgy, unless one has to support poorly written software. Even back in the m68k -> SPARC transition, people had clean ways to keep things separate and clean, and to support the same homes and binaries across both architectures simultaneously.
We had the PowerPC to x86 transition show us how it can be done so cleanly that many people had no idea what architecture they were running. The same can be said about the x86 to ARM transition with Apple now. (Open)VMS explicitly supports multiple simultaneous architectures. We have ways of making fat binaries for other OSes, too - we had them back then, and we have them right now.
We also have examples of doing it wrong: Microsoft Windows. The idea that people had to know and choose whether they want a 32 bit or 64 bit OS is still incredibly stupid, as is needing to choose 32 bit or 64 bit software.
If people are writing binary files in endian specific ways, or with data sizes that depend on the specific processor, that's shitty programming. Any software doing that never had any sort of portability in mind, so that's not a good example at all. That kind of software is chided in public in the open source world, or at very least people know to isolate it and run it where it's safe.
Sharing a user's home across multiple systems, perhaps with different architectures, isn't to save disk space, by the way. Expecting users to manage data and configurations across multiple machines isn't reasonable. Shared homes is still both normal and common in larger environments.
If the author disagrees, it'd be nice to hear some actual examples of why it's a problem. After all, ARM isn't going away and RISC-V is only going to become more common.
Compiling from source is easier said than done for many big open source projects. In practice, they compile only with specific versions of GCC, CMake and libc.
You don't always have 6 different projects to choose from because there just aren't that many projects that solve $problem according to your specific $requirements.
And it's also just a hassle even with good build systems, and I say this as someone who has a bunch of custom build scripts to compile stuff from source.
quite often, this is a major indicator of how quality is valued in an organisation.
Whether we are talking about open-source or not, there has to be a commitment to excellence at all levels of the product, as would be expected in any other real-world manufacturing.
chromium is also an abomination, its pretty much the single biggest thing one could possibly build. the sheer level of insanity in that codebase boggles the mind, >1GB highly compressed source is just not legitimate
i have a hard time imaginging it taking longer to compile than chrome, but yeah, I havent.
but like... there are two great pacific garbage patches, i guess its possible there are two software garbage patches bigger than the combined output software output of many countries.
Having never built it myself, I wonder why the hell is it so slow to compile? I've built Firefox many times, including multi-stage builds with PGO, and it takes maybe 30 minutes tops on my 2015 era desktop. Is it because they use a lot of vendored code?
My build directory is nearly 100G, with over 1 million files... I think it's just a huge old codebase with thousands of third party libraries which all need to be built too. And, being mostly C++, there is heavy use of templates which makes compilation very slow and the resulting binaries large.
Google has a distributed build system that can built the whole thing in about 15 seconds on a large cluster of machines.
They aren't so concerned about someone building it on one machine. (to be fair, they do document ways regular folks like me can do distributed builds too)
I've never put much thought into this but given a "linux x86" binary package, on what range of machines is it supposed to work? for instance, what's x86 mean exactly? I suppose all x86 processors have slightly different sets of instruction. Also, different linux kernels may have different sets of syscalls?
Linux Kernel userland ABI/API is very stable, and backward compatible: you can take a binary compile for Linux 1.0 and run it today. The same applies for CPU instruction, backward compatibility means stuff targeting i686 instruction sets will run on today CPU.
However, if you need more modern stuffs, then obviously you need a higher minimum version. For example, some software require SSE; or docker will only run on a modern Linux kernel.
The interface between the Kernel and Userland is stable.
The interface between Userland-Userland is not, very much not, a binary compiled a decade ago, unless its statically compiled or has no dependencies, will very likely not run, and is why things such as Flatpak and Docker partially exist in the first place.
It's also why closed-source software often include every dependencies in their installation package. And even that is sometimes not enough, like when the host's glibc is too old.
Well, it's worth noting that Linux backward compatibility is not absolute guarantee, but rather "if feature is removed in a forest and nobody is around to hear it, then it's not a backward compatibility break".
Eg. for instance the removal of a.out support: https://lwn.net/Articles/888741/ - the a.out support was almost removed, until a single user happened to notice and object. But then someone came up with a workaround and that single (known) remaining a.out user changed his code, and the a.out support was deleted.
At least for 32-bit x86 I've run into such problems. The Intel Quark platform has such an old CPU core in it that normal Ubuntu binaries wouldn't work on it (even at the the platform was relevant), as it lacks conditional move instructions unlike any other non-museum piece x86.
Also Glibc drops support for old kernel versions time to time, so even statically linked stuff isn't safe when compiling modern stuff against really old kernels.
I regularly run into projects that won't compile on the machine I want to run them on because it doesn't have enough RAM. It's not a panacea, by any means.
Lots of people it's using OpenBSD on macppc with light usage. Even playing some games such as OpenTTD.
Also, software like Luakit has been working fine, and on videoconferencing... once you have USB 2.0 UVC webcams, FFMPEG and drivers working, FLOSS conferencing software (lots of them) will work the same.
It's not that it doesn't work at all, but it is more effort. Even though I spent more time tinkering with lots of stuff than most (and probably still do), at least having the "Just Works™" option is nice.
It's a simple statement of fact, but it does need to be pointed out when ARM or RISC is often hyped. Many forget that that most non-x86's are system-on-chips, with very different components on them to make each a unique hardware platform on its own for system developers (and thus a huge pain in the butt) compared to the more "standardised" x86-64 hardware available from Intel and AMD.
I fear that unless some kind of standardisation is introduced for SoCs, we are looking at a very fragmented future for non-x86-64 platforms.
A good example of what I am saying can be seen with Apple Silicon SoCs todays - since details of some of the components in it are unavailable, system developers are being forced to reverse-engineer it to port other OSes on it. Imagine the effort to do that for 100's or 1000's of SoCs, each of which can have their own unique, custom component (with closed documentation) in it?
What I meant was that Intel and AMD take the pain to be more open about their SoCs than other ARM manufacturers, and ensure there aren't 100's of them. Both are more developer friendly than others who often deliberately choose the ARM SoC's mainly to ensure they can exert control over system developers - by ensuring that details of the component hardware are only available on closed licenses or sometimes not even made available to the public at all.
The author is confused. The points they make are not about x86 vs. non-x86, they're instead about mixing different architectures. Yes, if you write a program and deploy it on three machines, and they all have different pointer sizes, endianness, and executable formats (PE, ELF, etc.) then, yes that can suck.
If this article was written 10 years ago, more people would probably agree. But nowadays, its perfectly possible to have a setup where all your machines, from large servers, to laptops, to small embedded devices, are x86, or aarch64, or something like that, and no mixed architecture is present. There are other, possibly more glaring, issues with sharing software across very different setups: Memory requirements, assumptions about filesystem layout, assumptions about syscalls/capabilities, assumptions about security. The architecture is usually at most a cli switch in your build pipeline, all this other stuff isn't.
That said, if you use Rust, Zig, Java, C++ (with Zig C++, for example), C (with Zig CC), building N different binaries for different targets is a bash script with exactly N lines of code. Not sure that's all that hard.
> where [...] people haven't tried their software, software has architecture specific bugs
I have to agree with this, just from the projects I've been part of, reviewed and used in the past. This is not an ARM issue, its an issue of laziness. People are inherently lazy, and they will only build and test on what is convenient. x86 machine at home, x86 ci/cd server, ... guess what isn't being tested on? Non-x86.
That said, there are not that many things that can go wrong on non-x86 in my experience. They're usually little endian, they usually have 32 or 64 bit pointers, sizes, etc., and if they don't, they're likely not magical either.
People who still write C code, but also don't know that they can't just use `char`, `int` and `long` to mean 8, 32, and 64 bit, are the only population that will have issues here. Sadly, that's a lot of people in my experience.
The real issue is everything else, like I said above (fs layout, security, assumptions about OS specific behavior, ...)
Challenge: specify a mainstream architecture (e.g. targeted by Debian stable) which doesn't have char=1, short=2, int=4, long long=8 bytes with GCC or Clang.
Aren't pointers pretty much universally 64 bits, except for some old or esoteric machines?
And why would you not be able to treat one as an integer? Are there any examples of such ISAs?
Finally, floating point formats do differ, but that shouldn't affect anything except precision and binary format. Also I'm pretty sure "float" is standard IEEE 32 bit and "double" is standard 64 bit, with predefined exponent/mantissa sizes. The only thing that could differ between arches is endianness, and by far most ISAs have settled on little endian.
x86_32 counts as an old machine (the most recent CPU being the first generation Intel Atom). It's being dropped left and right in favour of only supporting 64-bit machines (x86_64, aarch64, maybe RV64).
CPUs shipped by intel today still support (non exclusively) x86-32. All major OSs (except for MacOS I guess) support running 32bit code along side 64bit code. New 32 bit binaries are created every day.
It is hardly legacy yet.
Separately, there is also the x32 ABI, although that hasn't really gained any traction.
It’s not gatekeeping in any sense. Technical skills are required to do technical jobs, and this person asked for any criticism or feedback given by posting a public blog that has, for some bizarre reason, gotten to the front page of hacker news. Wtf. Technical incompetence must be addressed head first, or we risk real world problems
> Technical incompetence must be addressed head first, or we risk real world problems
Given there's always been a chronic shortage of qualified and experienced technical staff, isn't the the "real" real-world problem here the raising of the barriers-to-entry for sysadmins jobs? I think most companies would rather employ someone who can at-least try to get something done rather than having no-one at all.
One of the reasons why "the cloud" is so popular among businesses today is because it frees them up from having to run their own systems and infrastructure (who needs a *nix sysadmin when the AWS and Azure portals let you do everything through a GUI?) - whereas I'm sure many (most?) of those same businesses would honestly prefer retaining possession and sovereignty over their data and applications (especiallly latency-sensitive stuff) - and we all lament how the cloud giveth and the cloud taketh-away - but here we are.
Running ‘configure && make && make install’ is hardly a technical skill. Acquiring binaries built elsewhere is fine and saying someone doesn’t qualify if they do that is gatekeeping.
That is correct, it is ‘user6723’ who is talking about that and I was responding to that user. But nevertheless the blog appears to be by someone who doesn’t have much of a clue regarding the subject matter, the arguments are rather confused and nonsensical.
If you quote only half of the sentence, it suddenly means something else! When used to attack it’s a variation of the venerable ‘straw man’ debate technique.
Nice if Mr Siebenmann made insightful comments on a Usenet group about Unix but that doesn’t mean his comments about ARM computers are indisputable.
How about you summarize the points in the blog post that you agree with?
Just because someone posts on a mailing list and calls themselves a sys-admin doesn't mean they are one.
What the author wrote objectively demonstrates a lack of essential skills and also a lack of temperament for what could be called a sys-admin, and therefore they are ultimately disqualified.
Having a standard installation for tools and libraries goes a long way to being independent of operating system updates.
Compiling and making versioned, packages of them (for example, in /opt/corp/prod/lang/gcc/12.3.0/... ) means that there can be a known good set of tools independent of operating system version and when the latter is upgraded, users can still build their products using the same paths and versions.
This is before the widespread use of containers, but doesn't change the advantages of having a reliable set of installed tools where users are in control of what versions they use.
Correct, most sys-admins use binary packages. However if a sys-admin finds maintaining packages for other CPU architectures difficult, like they can't handle something very basic like "uname -m", they aren't qualified for the role.
Whether this is "harsh" or not doesn't change that it is true.
That's ridiculous. Almost every large and small company has some kind of Microsoft server or cloud subscription. The Linux machines that do get deployed are absolutely full of pre-built distros, firmware images, and closed-source executables.
No serious systems administrator is going full Gentoo on their entire setup. It's a waste of time and effort to do so.
Obviously the author knows they could compile Firefox themselves. It's not even that hard, you just need to download a whole bunch of dependencies to your machine and wait for half an hour. That's not how sysadmins people get their software, though.
If you can't do deb-src you aren't qualified to be a sys admin, at least not one that should be paid.
Being an amateur who landed a job and being someone who is qualified for the title are not the same.
>> No serious systems administrator is going full Gentoo on their entire setup. It's a waste of time and effort to do so.
This is a total strawman argument, but I'll knock it down anyway. There are professionals who actually do use Gentoo on their local machine. The local machine (a laptop) doesn't do any compiling, there is a server that builds the packages.
I was actually using Gentoo myself for a couple years and all my packages were automatically built by a cluster of servers using distcc, so any build job was done with something like "make -j128"
Because I knew how to use Unix-like systems properly and having a dedicated build server, Gentoo didn't cost me any time, quite the opposite actually, because Gentoo is flexible with package versions and everything tends to just werk.
>> That's not how sysadmins people get their software, though.
For those of us contending with PowerPC and ARM, it is half of how we get our software, the other half is we have already automated the process and a cicd pipeline has built the software often before we as humans notice a new version is out.
The difference is not mainly about CPU architecture, it’s about the standardized PC architecture vs ARM computers where you need to build a new distribution for every model.
You can easily build a kernel that will run on any PC built in the last decade and any PC that will be built in the next decade. But if you are using an ARM computer you have to run the patched kernel provided by the manufacturer.
That depends on the ARM machine you picked. There's an ARM UEFI standard, for example.
With postmarketOS a ton of Android devices can now also boot the mainline Linux kernel. Samsung certainly isn't helping anyone run modern Linux kernels on their devices, but my old tablet released with Linux 3.1 now runs Linux 6.2.
Getting the necessary kernels and software together isn't even all that difficult most of the time, the problem is that you need someone to set up an appropriate distribution. I could've gone completely compile-free if someone had sacrificed their cloud storage space to offer compiled installer ZIPs for me, because the software on the tablet now just comes from the Alpine repositories.
It’s much better than PC, there’s many more standards to chose from! /s
> Getting the necessary kernels and software together isn't even all that difficult most of the time, the problem is that you need someone to set up an appropriate distribution
Guess what? On PC there’s a ton of people who have set up a distribution and they are all appropriate for all PCs. That makes it a whole lot easier to find such a person for your PC.
Great! Perhaps one day all ARM64 systems will be ‘system ready’ uefi hardware, just like pretty much all x64 systems are PC systems. But today is not that day and that is why the author of the blog post is running into these problems.
The basic boot process is easier, but it has the same "this board doesn't boot sorry try these steps" problem. Try disabling the splash screen and watch all the ACPI bugs and workarounds jump onto the screen during early boot.
It wasn't that long ago that you needed to download five different distros to brute force a working combination if you had a laptop with an Nvidia chip.
Even today you need to pass special boot options on some AMD boards because motherboard vendors can't be bothered to release microcode updates.
Linux 5.8+ refused to boot with 12th gen Intel graphics if you had a DisplayPort to HDMI adapter installed. I believe it took at least half a year before a fix was released (in a later version of the kernel, that wasn't backported). I remember this vividly because I showed off some dumb Gnome extension and he wanted it too, so he upgraded from a lower LTS to the then-current LTS and after a reboot he could no longer boot.
My laptop didn't have sound unless I passed some kind of special parameter in the boot config somewhere. It sometimes freezes on boot while loading GDM. The display flickers and jumps around if you have a maximized application. The Nvidia driver+WM combo doesn't recognize external displays on Wayland.
I've even run into one instance where the amd64 version of the distro shipped tons of applications requiring a certain instruction set, so installing the outdated (and now abandoned) x86 versions of the distro was the only solution.
Linux Mostly Works on Most Systems. The only reason you can get anything done on it is that there are more people using it and fixing the mess that hardware vendors have left behind.
I would say that most people will be able to run Linux smoothly on their hardware if they pick a user friendly distro, but random failure modes are everywhere.
"Picking the right binary for the right architecture", isn't your OS package manager supposed to take care of that? Cross-compiling is often pretty easy if you have custom binaries.
There is a big advantage to using different OSes/architectures with the same programs (compiled to target the right combination of hardware and OS) : it often helps detect subtle bugs, that may not manifest on one platform but be obvious on the next.
For a while I was running a mixed k3s server for fun and curiosity. Most popular images are manifests pointing to -$arch containers that gets selected automatically. It's also fairly trivial to build them yourself either with 2 machines or qemu.
I've stopped using it but multi arch is definitely not the reason. Biggest problem is that 60% of my punny machines resources were being used to keep Longhorn synced - and the constant high usage resource kicking the fans up started to irritate me. But the cluster worked really well regardless of what node the software was running.
There’s an argument to be made that if you want a more robust computing environment, you want more diversity of architecture and OS, not less. Does this take more effort? Possibly. Is that worth it? It depends.
Without architecture diversity, he wants to limit innovation, and promote a monopoly. x86 is anything but an open source standard, due to patents, and thus cannot be easily standardized. So he also voices against open standards.
Having non-x86 machines has actually made my life a lot easier -- most common exploits don't have the proper offsets and simply don't work.
Before you call me lazy for not patching, there are a lot of vendors that have very long patch times. If you have an environment that is mission critical, you would be well advised to not use x86 at all in the server environment. My entire income is based on cryptocurrency, so if I get owned it would result in large uninsurable losses.
I find it hard to disagree with this post, because the points are so broad, and the claim ("makes your life harder") is so low stakes. Most comments I read on here seem to interpret it as a form of "impossible." Sure you might not have some of those problems if you compile from source for all machines, but that is definitely harder than having pre-compiled options.
Especially if you have to maintain many machines (which is mostly implied in the post, but also explicitly mentioned here and there).
Hosting and maintaining a custom package repository for your self-compiled binaries so you don't have to do it on every single machine manually (maybe compiling everything from scratch might take days or the machine couldn't do it on its own anyways) is also obviously making your life harder.
My personal small scale experience, developing a small service on my laptop and running it on my local pi for testing, already ran into many of the problems. Some packages I wanted to use didn't exist, some containers I used locally weren't available for ARM. Are those unsolvable, or even hard, problems? No. But! They are time sinks that make a very simple thing more complex, more brittle and more annoying, than if I just used a homogeneous x86 setup, even if I were a skilled sysadmin (which I am not) or enjoyed fixing those differences and maintaining the fixes (which I do not).
My ARM server experience is similarly disappointing. Sometimes there's software that's just not available and setting up cross compilers sucks.
Running Docker containers on ARM feels like the Gentoo version of Docker. Sometimes you'll find an official image that will work, but most of the time you end up compiling images. In some cases, the base images need compiling too, so you end up with custom Docker bases that you now also need to keep track of in case they need to be updated.
There's no technical reason why it has to be this hard, but if you're mostly running other people's software, everything is just so very suboptimal.
Luckily, Apple's macbooks are now bringing consumer ARM to developers in a way that makes sense. I think there's a direct link between the M1 coming out and the sudden increase in aarch64 Docker images.
There are still major pain points with ARM but the experience is getting better.
I have a mix of x86_64 and aarch64 machines and, damn, NixOS makes it so freaking easy. Most of nixpkgs works on both architectures. You can even use your machines as remote builders and/or use binfmt to transparently run/build stuff through qemu.
When I was helping out a friend with his free oracle aarch64 server, I got baffled on how ARM feels like a second class citizen on most distros.
99 comments
[ 0.23 ms ] story [ 181 ms ] threadSorry I'm not sure I buy this line of reasoning.
I think that has a lot more to do with the economics around Linux, Windows, and macOS on commodity amd64 hardware becoming “good enough” to replace servers and workstations running on other, more expensive, architectures.
Not having to deal with architecture porting issues is just gravy at that point.
It's also factually incorrect:
"If you try to share things across architectures, you'll get to find all of the ways that modern systems aren't really set up for that, partly because it's been decades since most environments really tried to do this."
What? Did the author give even ONE example? No.
It has never been the case that environments haven't tried to do this. Ever hear of the Internet? It has every architecture you can imagine. If you support 32 bit and 64 bit x86, you're already "sharing things across architectures" because of how different they are.
Chris Siebenmann dates from the times when system administrators would do things like have most of the operating system executables and read-only data files remote-mounted from fileservers. We'd worry about such things as instruction set architectures and bitnessess and endinannesses of non-text file formats. We'd have "architecture-dependent" and "architecture-independent" exports. We'd take care when replacing executable scripts with compiled binaries.
And yes, it has been decades since this was a norm. DASDs have long since become big and cheap enough that just having copies of all of the files locally has long overtaken the idea of having the operating system partly on a remote fileserver that had the limited money spent on it for the really big DASDs.
Microsoft originally had DOS+Windows with a shareable "system" directory and a per-machine "windows" directory. Modern Windows doesn't even incorporate such an idea, not least because many things came along that put per-machine stuff in the "system" directory. Modern Unices and Linux-based operating systems similarly don't normalize the idea of remote-mounted /usr/share and whatnot as Unices did decades ago.
* https://utcc.utoronto.ca/~cks/space/blog/unix/FadingMultiArc...
...So why is it relevant?
Sharing files between systems with different architectures Just Works these days, unless you start getting into the really esoteric.
"Having non-x86 machines is Bad, now, today, because of problems I had with managing multiple-architecture deployments 30 years ago" is not a useful statement in any way.
Likely, the binaries are not from the OS, but from users/for users which run on different machines (and more likely than not NFS mounted across the systems to ensure consistency).
The reality is that there have always been ways to have multiple architectures supported simultaneously, and they're neither all that complicated nor kludgy, unless one has to support poorly written software. Even back in the m68k -> SPARC transition, people had clean ways to keep things separate and clean, and to support the same homes and binaries across both architectures simultaneously.
We had the PowerPC to x86 transition show us how it can be done so cleanly that many people had no idea what architecture they were running. The same can be said about the x86 to ARM transition with Apple now. (Open)VMS explicitly supports multiple simultaneous architectures. We have ways of making fat binaries for other OSes, too - we had them back then, and we have them right now.
We also have examples of doing it wrong: Microsoft Windows. The idea that people had to know and choose whether they want a 32 bit or 64 bit OS is still incredibly stupid, as is needing to choose 32 bit or 64 bit software.
If people are writing binary files in endian specific ways, or with data sizes that depend on the specific processor, that's shitty programming. Any software doing that never had any sort of portability in mind, so that's not a good example at all. That kind of software is chided in public in the open source world, or at very least people know to isolate it and run it where it's safe.
Sharing a user's home across multiple systems, perhaps with different architectures, isn't to save disk space, by the way. Expecting users to manage data and configurations across multiple machines isn't reasonable. Shared homes is still both normal and common in larger environments.
If the author disagrees, it'd be nice to hear some actual examples of why it's a problem. After all, ARM isn't going away and RISC-V is only going to become more common.
Less of an issue if you compile from source.
And it's also just a hassle even with good build systems, and I say this as someone who has a bunch of custom build scripts to compile stuff from source.
Whether we are talking about open-source or not, there has to be a commitment to excellence at all levels of the product, as would be expected in any other real-world manufacturing.
That's pretty easy nowadays thanks to container.
You don't have to muck around with environment variables to get the build system to detect your custom toolchain anymore.
but like... there are two great pacific garbage patches, i guess its possible there are two software garbage patches bigger than the combined output software output of many countries.
They aren't so concerned about someone building it on one machine. (to be fair, they do document ways regular folks like me can do distributed builds too)
However, if you need more modern stuffs, then obviously you need a higher minimum version. For example, some software require SSE; or docker will only run on a modern Linux kernel.
The interface between Userland-Userland is not, very much not, a binary compiled a decade ago, unless its statically compiled or has no dependencies, will very likely not run, and is why things such as Flatpak and Docker partially exist in the first place.
It's also why closed-source software often include every dependencies in their installation package. And even that is sometimes not enough, like when the host's glibc is too old.
Eg. for instance the removal of a.out support: https://lwn.net/Articles/888741/ - the a.out support was almost removed, until a single user happened to notice and object. But then someone came up with a workaround and that single (known) remaining a.out user changed his code, and the a.out support was deleted.
Also Glibc drops support for old kernel versions time to time, so even statically linked stuff isn't safe when compiling modern stuff against really old kernels.
Also, software like Luakit has been working fine, and on videoconferencing... once you have USB 2.0 UVC webcams, FFMPEG and drivers working, FLOSS conferencing software (lots of them) will work the same.
I fear that unless some kind of standardisation is introduced for SoCs, we are looking at a very fragmented future for non-x86-64 platforms.
If this article was written 10 years ago, more people would probably agree. But nowadays, its perfectly possible to have a setup where all your machines, from large servers, to laptops, to small embedded devices, are x86, or aarch64, or something like that, and no mixed architecture is present. There are other, possibly more glaring, issues with sharing software across very different setups: Memory requirements, assumptions about filesystem layout, assumptions about syscalls/capabilities, assumptions about security. The architecture is usually at most a cli switch in your build pipeline, all this other stuff isn't.
That said, if you use Rust, Zig, Java, C++ (with Zig C++, for example), C (with Zig CC), building N different binaries for different targets is a bash script with exactly N lines of code. Not sure that's all that hard.
> where [...] people haven't tried their software, software has architecture specific bugs
I have to agree with this, just from the projects I've been part of, reviewed and used in the past. This is not an ARM issue, its an issue of laziness. People are inherently lazy, and they will only build and test on what is convenient. x86 machine at home, x86 ci/cd server, ... guess what isn't being tested on? Non-x86. That said, there are not that many things that can go wrong on non-x86 in my experience. They're usually little endian, they usually have 32 or 64 bit pointers, sizes, etc., and if they don't, they're likely not magical either.
People who still write C code, but also don't know that they can't just use `char`, `int` and `long` to mean 8, 32, and 64 bit, are the only population that will have issues here. Sadly, that's a lot of people in my experience.
The real issue is everything else, like I said above (fs layout, security, assumptions about OS specific behavior, ...)
Also floating point formats can vary a bit even with the size being the same. So a piece of code written for one might be subtly wrong for another.
And why would you not be able to treat one as an integer? Are there any examples of such ISAs?
Finally, floating point formats do differ, but that shouldn't affect anything except precision and binary format. Also I'm pretty sure "float" is standard IEEE 32 bit and "double" is standard 64 bit, with predefined exponent/mantissa sizes. The only thing that could differ between arches is endianness, and by far most ISAs have settled on little endian.
It is hardly legacy yet.
Separately, there is also the x32 ABI, although that hasn't really gained any traction.
https://wiki.debian.org/ArchitectureSpecificsMemo
It shows that there are no such Debian architectures.
Just like in the Unix world of old before the rise of open source, many system administrators never compile any binary. And that is fine.
Given there's always been a chronic shortage of qualified and experienced technical staff, isn't the the "real" real-world problem here the raising of the barriers-to-entry for sysadmins jobs? I think most companies would rather employ someone who can at-least try to get something done rather than having no-one at all.
One of the reasons why "the cloud" is so popular among businesses today is because it frees them up from having to run their own systems and infrastructure (who needs a *nix sysadmin when the AWS and Azure portals let you do everything through a GUI?) - whereas I'm sure many (most?) of those same businesses would honestly prefer retaining possession and sovereignty over their data and applications (especiallly latency-sensitive stuff) - and we all lament how the cloud giveth and the cloud taketh-away - but here we are.
Hint: Chris Siebenmann has administered closed source Unices.
Another hint: Chris Siebenmann's posts to comp.unix.wizards can still be found.
If you quote only half of the sentence, it suddenly means something else! When used to attack it’s a variation of the venerable ‘straw man’ debate technique.
Nice if Mr Siebenmann made insightful comments on a Usenet group about Unix but that doesn’t mean his comments about ARM computers are indisputable.
How about you summarize the points in the blog post that you agree with?
What the author wrote objectively demonstrates a lack of essential skills and also a lack of temperament for what could be called a sys-admin, and therefore they are ultimately disqualified.
Compiling and making versioned, packages of them (for example, in /opt/corp/prod/lang/gcc/12.3.0/... ) means that there can be a known good set of tools independent of operating system version and when the latter is upgraded, users can still build their products using the same paths and versions.
This is before the widespread use of containers, but doesn't change the advantages of having a reliable set of installed tools where users are in control of what versions they use.
Come on, that's too harsh. We're not all Gentoo users, most sysadmins use packages provided by the distros.
There's also the issue of close-source software, that only supports certain distros and architectures.
Whether this is "harsh" or not doesn't change that it is true.
No serious systems administrator is going full Gentoo on their entire setup. It's a waste of time and effort to do so.
Obviously the author knows they could compile Firefox themselves. It's not even that hard, you just need to download a whole bunch of dependencies to your machine and wait for half an hour. That's not how sysadmins people get their software, though.
Being an amateur who landed a job and being someone who is qualified for the title are not the same.
>> No serious systems administrator is going full Gentoo on their entire setup. It's a waste of time and effort to do so.
This is a total strawman argument, but I'll knock it down anyway. There are professionals who actually do use Gentoo on their local machine. The local machine (a laptop) doesn't do any compiling, there is a server that builds the packages.
I was actually using Gentoo myself for a couple years and all my packages were automatically built by a cluster of servers using distcc, so any build job was done with something like "make -j128"
Because I knew how to use Unix-like systems properly and having a dedicated build server, Gentoo didn't cost me any time, quite the opposite actually, because Gentoo is flexible with package versions and everything tends to just werk.
>> That's not how sysadmins people get their software, though.
For those of us contending with PowerPC and ARM, it is half of how we get our software, the other half is we have already automated the process and a cicd pipeline has built the software often before we as humans notice a new version is out.
The base system packages are exempt from the stipulation I posted above.
The range of ARM64 binaries in the various repositories has been increasing in the last years.
You can easily build a kernel that will run on any PC built in the last decade and any PC that will be built in the next decade. But if you are using an ARM computer you have to run the patched kernel provided by the manufacturer.
With postmarketOS a ton of Android devices can now also boot the mainline Linux kernel. Samsung certainly isn't helping anyone run modern Linux kernels on their devices, but my old tablet released with Linux 3.1 now runs Linux 6.2.
Getting the necessary kernels and software together isn't even all that difficult most of the time, the problem is that you need someone to set up an appropriate distribution. I could've gone completely compile-free if someone had sacrificed their cloud storage space to offer compiled installer ZIPs for me, because the software on the tablet now just comes from the Alpine repositories.
> Getting the necessary kernels and software together isn't even all that difficult most of the time, the problem is that you need someone to set up an appropriate distribution
Guess what? On PC there’s a ton of people who have set up a distribution and they are all appropriate for all PCs. That makes it a whole lot easier to find such a person for your PC.
https://cdimage.ubuntu.com/daily-live/current/mantic-desktop...
It wasn't that long ago that you needed to download five different distros to brute force a working combination if you had a laptop with an Nvidia chip.
Even today you need to pass special boot options on some AMD boards because motherboard vendors can't be bothered to release microcode updates.
Linux 5.8+ refused to boot with 12th gen Intel graphics if you had a DisplayPort to HDMI adapter installed. I believe it took at least half a year before a fix was released (in a later version of the kernel, that wasn't backported). I remember this vividly because I showed off some dumb Gnome extension and he wanted it too, so he upgraded from a lower LTS to the then-current LTS and after a reboot he could no longer boot.
My laptop didn't have sound unless I passed some kind of special parameter in the boot config somewhere. It sometimes freezes on boot while loading GDM. The display flickers and jumps around if you have a maximized application. The Nvidia driver+WM combo doesn't recognize external displays on Wayland.
I've even run into one instance where the amd64 version of the distro shipped tons of applications requiring a certain instruction set, so installing the outdated (and now abandoned) x86 versions of the distro was the only solution.
Linux Mostly Works on Most Systems. The only reason you can get anything done on it is that there are more people using it and fixing the mess that hardware vendors have left behind.
I would say that most people will be able to run Linux smoothly on their hardware if they pick a user friendly distro, but random failure modes are everywhere.
There is a big advantage to using different OSes/architectures with the same programs (compiled to target the right combination of hardware and OS) : it often helps detect subtle bugs, that may not manifest on one platform but be obvious on the next.
Sounds a little like the old Linux v Windows or the PC v Mac debate - lots of fuss and really comes down to personal preferences.
I've stopped using it but multi arch is definitely not the reason. Biggest problem is that 60% of my punny machines resources were being used to keep Longhorn synced - and the constant high usage resource kicking the fans up started to irritate me. But the cluster worked really well regardless of what node the software was running.
Without architecture diversity, he wants to limit innovation, and promote a monopoly. x86 is anything but an open source standard, due to patents, and thus cannot be easily standardized. So he also voices against open standards.
Before you call me lazy for not patching, there are a lot of vendors that have very long patch times. If you have an environment that is mission critical, you would be well advised to not use x86 at all in the server environment. My entire income is based on cryptocurrency, so if I get owned it would result in large uninsurable losses.
Especially if you have to maintain many machines (which is mostly implied in the post, but also explicitly mentioned here and there).
Hosting and maintaining a custom package repository for your self-compiled binaries so you don't have to do it on every single machine manually (maybe compiling everything from scratch might take days or the machine couldn't do it on its own anyways) is also obviously making your life harder.
My personal small scale experience, developing a small service on my laptop and running it on my local pi for testing, already ran into many of the problems. Some packages I wanted to use didn't exist, some containers I used locally weren't available for ARM. Are those unsolvable, or even hard, problems? No. But! They are time sinks that make a very simple thing more complex, more brittle and more annoying, than if I just used a homogeneous x86 setup, even if I were a skilled sysadmin (which I am not) or enjoyed fixing those differences and maintaining the fixes (which I do not).
So I waited a couple of years and tried again figuring they’d have got it worked out.
No dice.
Ever since then I’ve stuck with AMD64 for everything.
Running Docker containers on ARM feels like the Gentoo version of Docker. Sometimes you'll find an official image that will work, but most of the time you end up compiling images. In some cases, the base images need compiling too, so you end up with custom Docker bases that you now also need to keep track of in case they need to be updated.
There's no technical reason why it has to be this hard, but if you're mostly running other people's software, everything is just so very suboptimal.
Luckily, Apple's macbooks are now bringing consumer ARM to developers in a way that makes sense. I think there's a direct link between the M1 coming out and the sudden increase in aarch64 Docker images.
There are still major pain points with ARM but the experience is getting better.
My tablet and phone is ARM as well, and I'm not considering switching them to intel either.
When I was helping out a friend with his free oracle aarch64 server, I got baffled on how ARM feels like a second class citizen on most distros.