It took a while for me to fully appreciate the OpenSSH approach to portability:
They primarily develop OpenSSH purely for OpenBSD, using all (including non-portable) facilities of OpenBSD, including crypto and whatnot.
Then, a separate team manages the "portable" version of OpenSSH, which add stubs and does everything else needed to make OpenSSH compile on as many operating systems as possible.
I'm aware that OpenSSH is not the only project using that approach to portability. Nevertheless, I think it is fair to say this is an unusual approach used only on a minority of projects.
I was always puzzled on why they are doing this. This always struck me to be "just" a side effect of project politics and historically grown project structures.
But over the years I started to see some interesting benefits of that approach as well. I'm still not convinced by this model, but I have to admit that, more generally speaking, the OpenBSD project does many things against the mainstream, but quite often they turn to be right.
To me the "natural way" has always been to write portable code in the first place. From time to time, you'll find that parts of it are not portable, so you fix it, and along that way to learned something new about portability and apply it to future improvements on your code as well. Over time, you'll find fewer and fewer portability issue as you get better and better at writing portable code in the first place.
I'm not saying that this is the best way to do this, but to me this was always the obvious thing to do. As a somewhat extreme example, I'd never write a graphical user interface in pure Win32 API and expect it to be even remotely portable by some additional layer. I'd rather use Qt (or GTK, or Dear ImGui, or whatever) for native UIs even for programs that are (for now) meant to be only run on Windows.
To me personally, this has the additional benefit that I can do most of the development and testing in a non-hostile environment (e.g. Debian), then running a cross compiler (e.g. via MXE) and only do the final testing on Windows (well, usually first Wine, then some Windows VM), but at that last stage surprises are extremely seldom.
Be aware - that is an extreme minority of projects in the real world.
Historically, most of the time the development team only knows a specific platform specific language or way of doing things, and doesn’t have the background or experience to even know that what they would be doing in a specific place isn’t portable.
Languages and cross platform toolkits have developed a lot, but I still would question if most development folks would even recognize something like an endianness issue was a potential problem on some random project.
If someone is doing HTML/js/web dev, they’d never need to worry though I guess (barring something really weird).
Can you name one? OpenBSD-affiliated projects like OpenSSH are the only ones I know of that do fully separate releases where one is just for one OS, and the other is the “portable” one
Not really, that is how the games industry has worked since forever and one of the reasons why AAA game studios couldn't care less about 3D APIs portability.
The game idea is developed with one specific platform in mind, and if the game actually gets a publishing deal, the publisher onboards studios whose main skill is to port games into platform XYZ.
With how quickly OS's and browsers update it reminds me of learning to stand on a windsurfer. Then trying to pull up the sail. Then control the sail in the wind. So many unpredictable forces one really has to have a strong and nimble base to work from.
This article describes why portability is hard even across Unix-like operating systems, which are for the most part actually trying to implement standards in good faith.
But for my money, where portability really gets hard is when you try to make something work across platforms where the platform vendors are actively, strategically dedicated to sabotaging portability.
Software engineers who want their skills to be as valuable as possible will forever be at odds with platform vendors who do everything to lock them in.
Portability of software is a language problem. Some languages make it extremely easy. Others, like C and Go, make it difficult to be correct across all platforms.
Hard disagree there. Trying to write cross distro cli tools can get really hairy really fast when you need to deal with distro inconsistencies in file locations, default configs/permissions, systemd vs not (maybe that one is past now), etc...
I wrote an iOS app. I ported it to Android. The only part that didn't need to be rewritten from the ground up (and it worked completely unmodified) was the part written in C.
If you have the same APIs available (OpenGL in this case) C is the most portable language there is. Which proves it's not a language issue, it's a platform issue.
In fact the owners of the platforms care so little about (indeed actively oppose) compatibility between them that increasingly they make their higher-level APIs only available to a single language so you can't even share the parts of your code that aren't dependent on their APIs. Unless of course you drop into the one language that works everywhere and which they have no choice but to support: C (or something else that compiles natively and can present a C API like C++, Rust or Kotlin native).
Java's "write once run everywhere" has become a sad joke.
> the one language that works everywhere and which they have no choice but to support: C
I've been tinkering with Rust bindings for Apple Core Audio. AUGraph, which has a C interface, was deprecated a few years ago. Its replacement, AVAudioEngine, has an Objective-C/Swift API. :(
There are a lot of generalized cross-platform libraries for guis, audio engines, etc, which attempt to abstract over the differences between native libraries. While noble, these are always going to be very limited and difficult because the library interfaces may make radically different assumptions and have very different high-level structures.
To me it seems as though the best strategy to help programmers who want to do cross-platform development similar to you is to focus on providing complete bindings for the platform-specific native libraries (for me the bindings language is Rust but it could theoretically be another one like Go). Once those exist, it expands the amount of code you write that can be shared. It also becomes more straightforward to write cross-platform abstraction libraries.
However, my sense is that the platform vendors are heading towards a future where C ABIs are eschewed and where sharing code will be utterly impossible.
Exactly what Microsoft does with Xamarin, has extensive tooling to generate platform bindings on every update. I can interop with RUST with no-low overhead as needed.
At one point I was porting between different C stacks. I found they would compile just fine. It was when you looked into the details of the different CRT functions you would have some real fun. For example printf is an evil function of randomness, where the parameters are identical but the way it would decode that input string was highly dependent on when it was written and which stack it was in. Now today you have basically 3 different compilers you have to worry about and they mostly act the same now. It is just as you move between libs that it can sometimes get a bit crazy if they use any sort of if/def to change the way some features work (glaring at you pthreads).
if you wrote your iOS app in Xamarin, u could today run it on .NET6 pretty much unchanged. I maintain multiple store apps where i share 90% of the code across all platforms (iOS/Android/Win/macOS) that have extensive integrations with the OS. I can even drop down to native obj-c, java, win32, macos apis as required using the same syntax as any other library thanks to the code-gen MS does. I use powershell for scripting. Dont need a full app? I can have native islands as well or be an island in another native App.
A good pattern is to separate the core logic from the code that interacts with the platform. If these are tangled up porting becomes a nightmare.
Our approach to portability in ZeroTier is to make the core, our "network hypervisor," a completely OS-neutral chunk of code that interacts only via an API. It contains literally no system calls, I/O, etc. Then there's a service harness for desktops, servers, phones, etc.
The problem is, for this to work the API itself must be cross-platform, which means it's either an incomplete abstraction (exposes many options, only some of which are implemented on a given platform), overly restricting (lowest common denominator leading to suboptimal performance and restricted development agility on advanced platforms), or not really cross-platform (supports one platform well but others are inefficient -- might as well use a virtual appliance).
In reality the API is part of the core logic except for purely computational services and therefore all core logic can rarely be fully encapsulated.
You need to design your system with appropriote trade-offs. If you want to be everything to everyone you will end up in mess you describe. Cross-platform game, coreutils or LOB are all different beasts. Unity shows how API surface level will get polluted as you strive for speed in your cross-platform game, but on the flip side you run on all platforms without overhead.
I tend to use DDD and the egg-shell pattern, building a domain / service model in portable code, and leaving the platform specific code to the outer shell behind a refined abstraction. I dont have any APIs with overloads, but sometimes the backing API is a no-op.
24 comments
[ 3.6 ms ] story [ 63.5 ms ] threadThey primarily develop OpenSSH purely for OpenBSD, using all (including non-portable) facilities of OpenBSD, including crypto and whatnot.
Then, a separate team manages the "portable" version of OpenSSH, which add stubs and does everything else needed to make OpenSSH compile on as many operating systems as possible.
I'm aware that OpenSSH is not the only project using that approach to portability. Nevertheless, I think it is fair to say this is an unusual approach used only on a minority of projects.
I was always puzzled on why they are doing this. This always struck me to be "just" a side effect of project politics and historically grown project structures.
But over the years I started to see some interesting benefits of that approach as well. I'm still not convinced by this model, but I have to admit that, more generally speaking, the OpenBSD project does many things against the mainstream, but quite often they turn to be right.
I'm not saying that this is the best way to do this, but to me this was always the obvious thing to do. As a somewhat extreme example, I'd never write a graphical user interface in pure Win32 API and expect it to be even remotely portable by some additional layer. I'd rather use Qt (or GTK, or Dear ImGui, or whatever) for native UIs even for programs that are (for now) meant to be only run on Windows.
To me personally, this has the additional benefit that I can do most of the development and testing in a non-hostile environment (e.g. Debian), then running a cross compiler (e.g. via MXE) and only do the final testing on Windows (well, usually first Wine, then some Windows VM), but at that last stage surprises are extremely seldom.
Historically, most of the time the development team only knows a specific platform specific language or way of doing things, and doesn’t have the background or experience to even know that what they would be doing in a specific place isn’t portable.
Languages and cross platform toolkits have developed a lot, but I still would question if most development folks would even recognize something like an endianness issue was a potential problem on some random project.
If someone is doing HTML/js/web dev, they’d never need to worry though I guess (barring something really weird).
The game idea is developed with one specific platform in mind, and if the game actually gets a publishing deal, the publisher onboards studios whose main skill is to port games into platform XYZ.
But for my money, where portability really gets hard is when you try to make something work across platforms where the platform vendors are actively, strategically dedicated to sabotaging portability.
Software engineers who want their skills to be as valuable as possible will forever be at odds with platform vendors who do everything to lock them in.
I think this is one of those statements that sounds good at face value but doesn't stand up to scrutiny.
If you have the same APIs available (OpenGL in this case) C is the most portable language there is. Which proves it's not a language issue, it's a platform issue.
In fact the owners of the platforms care so little about (indeed actively oppose) compatibility between them that increasingly they make their higher-level APIs only available to a single language so you can't even share the parts of your code that aren't dependent on their APIs. Unless of course you drop into the one language that works everywhere and which they have no choice but to support: C (or something else that compiles natively and can present a C API like C++, Rust or Kotlin native).
Java's "write once run everywhere" has become a sad joke.
I've been tinkering with Rust bindings for Apple Core Audio. AUGraph, which has a C interface, was deprecated a few years ago. Its replacement, AVAudioEngine, has an Objective-C/Swift API. :(
There are a lot of generalized cross-platform libraries for guis, audio engines, etc, which attempt to abstract over the differences between native libraries. While noble, these are always going to be very limited and difficult because the library interfaces may make radically different assumptions and have very different high-level structures.
To me it seems as though the best strategy to help programmers who want to do cross-platform development similar to you is to focus on providing complete bindings for the platform-specific native libraries (for me the bindings language is Rust but it could theoretically be another one like Go). Once those exist, it expands the amount of code you write that can be shared. It also becomes more straightforward to write cross-platform abstraction libraries.
However, my sense is that the platform vendors are heading towards a future where C ABIs are eschewed and where sharing code will be utterly impossible.
Had this not been the case and even your C code would have been touched.
if you wrote your iOS app in Xamarin, u could today run it on .NET6 pretty much unchanged. I maintain multiple store apps where i share 90% of the code across all platforms (iOS/Android/Win/macOS) that have extensive integrations with the OS. I can even drop down to native obj-c, java, win32, macos apis as required using the same syntax as any other library thanks to the code-gen MS does. I use powershell for scripting. Dont need a full app? I can have native islands as well or be an island in another native App.
https://www.youtube.com/watch?v=kesUNeBZ1Os
Our approach to portability in ZeroTier is to make the core, our "network hypervisor," a completely OS-neutral chunk of code that interacts only via an API. It contains literally no system calls, I/O, etc. Then there's a service harness for desktops, servers, phones, etc.
In reality the API is part of the core logic except for purely computational services and therefore all core logic can rarely be fully encapsulated.
I tend to use DDD and the egg-shell pattern, building a domain / service model in portable code, and leaving the platform specific code to the outer shell behind a refined abstraction. I dont have any APIs with overloads, but sometimes the backing API is a no-op.