Author mentions that the APE’s can be booted from the BIOS. Would this imply you could perhaps implement a lambda-like cloud service that could spawn/start the APE on demand? Edit, I’d guess you’d want to/have to, also have a tcp/ip stack in the APE as well (but perhaps just going for containerization of the APE is the better play and not have to worry about any of that..)
Zero/near-zero dependencies, minimal configuration, portable, and trivial to run. Awesome.
Even if Redbean here doesn't end up taking over the world, I hope that more ecosystems and toolchains sit up and take notice. We need more tools that look like this.
I really like your view of the world, that programs should be portable, tiny, and just work. The slamming of so much functionality into a zip file is inspiring.
Would it be possible to do something similar with Free Pascal, allowing the recreation of something like Turbo Pascal, except really, really portable .com output?
Author here. Absolutely. I used to love Turbo Pascal and Delphi when I was younger. If Free Pascal uses GNU LD.BFD or LLVM LLD when it links programs, then all you'd need to do is is configure it to use cosmopolitan.a when linking system call functions like read(), write(), etc. See https://github.com/jart/cosmopolitan Another option is if Free Pascal wants to write all the system call support from scratch, then doing that now is going to be a whole lot easier since the Cosmopolitan codebase does a really good job documenting all the magic numbers you'll need. See files like https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... and https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c... I've been working on a tiny C11 compiler called chibicc which has most GNU extensions and I managed to get it working as an actually portable executable with an integrated assembler: https://github.com/jart/cosmopolitan/blob/master/third_party... I also got Antirez's KILO text editor working as an APE binary. https://github.com/jart/cosmopolitan/blob/master/examples/ki... If we can build a linker too then we can get a freestanding single file toolchain + ide that's able to be a modern version of Turbo C.
Author here. I've been chatting with the Zig BDFL on Twitter recently about contributing APE support and he's been super supportive so far. It's going to be a nontrivial undertaking but would certainly be rewarding for everyone if we can make it happen.
Since I can't upvote your Show HN twice, I'll upvote this comment. I learned something interesting tonight from your websites. Thanks!
chibicc is a really fun idea! If I was in college, I would take a compiler course from you.
Taken from your README:
"chibicc is developed as the reference implementation for a book I'm currently writing about the C compiler and the low-level programming. The book covers the vast topic with an incremental approach; in the first chapter, readers will implement a "compiler" that accepts just a single number as a "language", which will then gain one feature at a time in each section of the book until the language that the compiler accepts matches what the C11 spec specifies. I took this incremental approach from the paper by Abdulaziz Ghuloum."
Speaking of other languages, given that the APE is a zip container, it shouldn't be too hard to have a "starkit" like setup where the binary part is basically node/python/perl/tcl and the main script and all required libraries are in the executable.
I love what you've produced here. I noticed that pthreads seems to be fail-stubbed for the time being, i guess it's a todo with tricky semantics?
As for holding off on UI for the time being is probably the right thing to do, keeping with the minimal style planning on going for raw-x11,etc.
However, as much as I personally love plain framebuffers (late 90s democoder myself), it'd be totally non-accessible for "plain" applications and totally lacking performance compared to anything accelerated when it comes to rendering. Graphics cards are just so many times faster that it's not really even the same kind of usage (only sad part is Apple punting on OpenGL driver updates even if Intel on the low end has become so much better).
That said, having looked for a minimal UI framework and sadly they seem to be far inbetween these days (libUI is fairly compact on win/osX but required GTK on *nix but they've lost a bit of steam since 2018)
Best of luck on it all (hopefully I can snatch out some time to contribute)
So far the only platform where I've managed to get threads to work has been Windows. I came pretty close to getting clone() on Linux working a few days ago. Threads are so hard because no API was ever defined for them in original UNIX and BSD, so therefore each modern system today does them in a completely different way. Another blocker is that not all operating systems let us modify segment registers which is something that the Linux compiler assumes is available for TLS. So it'd be a miracle if it ended up being possible to make threads work portably. fork() on the other hand was easy to get working on all platforms. I love fork() and vfork().
As for GUIs I'm still open to merging raw x11 support. One tricky issue is that we'd need to define a new API that veneers framebuffers, and so far Cosmopolitan has mostly refrained from defining novel interfaces, since the focus has been twiddling with low level magic numbers in order to make the textbook interfaces we already have work everywhere!
Ah segregs is gonna be a pita indeed if it's all hard-wired to the compiler.
Considered making some way to add "portable" modules yet(generic APE code with plat-specific impls) to avoid adding stuff to the core? I know the default calling-conv is slightly different but maybe by smth like defining virtual ports? (ie "send" structs since alignment rules should be mostly the same?)
Threads are a PITA everywhere. On Linux the SYSCALL stuff complicates things because of all the register moves, the FUTEX stuff, and potential races with how threads are exited.
Admittedly they solve different usecases, where Redbean seems to want to serve a whole static site as a single file, which isn't currently possible with Caddy, (but you can run `caddy file-server` to serve the current directory as a static site), but it may become possible in the future with Golang's new embed package https://golang.org/pkg/embed/
That's actually a pretty interesting idea. I wonder if we could make a lite version of Caddy bundled with your website embedded...no need to mess with file systems or permissions or devices and folders to run a static site. Might need some massaging to become a truly pleasant workflow though.
The go std library offers the embed package. You can just embed your files and use the std lib's webserver and generate a single file binary.
No need for Caddy.
Caddy is cool but it has a separate 33mb executable for each operating system. Redbean is a 128kb executable that runs on all platforms so it has fewer moving parts and is 1584x tinier. All the convenience of being able to "just add your assets to the zip executable" wouldn't have been feasible if we needed to repeat that process n times for each operating system.
> Is that smile at the end of your first sentence meant to be condescending?
What? Of course not! You're reading way too much into that. It's simply enthusiasm. I read about Redbean and it being a single file webserver and the first thing to come to mind is Caddy, as "prior art".
Like I said, different usecases; Caddy is a general purpose webserver which has automatic certificate management, H2/H3 support, reverse proxy, file server, a strong plugin system, the list goes on. Of course it'll be bigger in size. Redbean is super cool and does have the advantage in file size and portability, clearly, but it is essentially a single-purpose tool. And there's nothing wrong with that.
The short description is a bit too short; this is mind blowing!
Redbean is a portable, single file executable webserver which also acts as a zip-file for the static content it hosts, and runs on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS without any recompilation...
You can manage the static content using standard pkzip tooling!
Now that's what I call thinking out of the box!
Is it that this binary does some tricks to run on Linux, Mac,
Windows, FreeBSD, OpenBSD, NetBSD, BIOS or does it really work on all operating systems including something like Plan 9 or even TempleOS?
And does it run on different architectures like ARMv6?
Native execution on x86_64 only. "Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS", where "BIOS" means that it will boot a machine and run on bare metal. On non x86_64 systems, the driver script will try to run the binary using QEMU (which has to be installed and on the path):
If i have understood how the portability works, it would be pretty straightforward to add support for another OS which uses the same ABI as the unices and has a compatible system call table - just a matter of adding a column to consts.sh (system call numbers in the nr section, constants from system headers in their appropriate sections) and adding any OS-specific system call rituals to systemfive.S.
Note that all system calls will go through the syscall interface. If your OS has some other way to make system calls, like Linux's vDSO or io_uring, that won't be used.
If your OS had a different system call table, like Windows, then you would need to write more bespoke code for it, like all the Windows wrappers:
.apk files and .ipa files are secretly zip files too. With the addition of a few manifest files and other crufty things embedded in the zip, I bet you could add Android and iOS to the list of supported platforms too.
I've recently succeeded in building working .apk files byte-by-byte completely from scratch so I could maybe help some. Open an issue on https://github.com/akavel/hellomello if interested in some talking/advice and/or watch my talk on NimConf 2020 for an overview (I don't have a link handy as I'm on mobile).
I just spent 30 minutes reading about this. I'm so shocked that I'm logging in to comment after 5 years of lurking.
Justine has built a c library that allows you compile a binary once and have it run it on any os or baremetal. The SAME binary. Quite frankly, that sentence doesn't even make sense to me.
I agree, I got to the page on actually portable executable, and had to read it a few times to make sure I was grokking it properly. This is just sheer cleverness, THIS is the stuff that should be on hacker news!
It's hard to grok cause there are no words to describe it.
When people say that something is cross-platform, they usually mean
a) that the software will build on multiple platforms
b) there is some sort of vm which runs the executable (jvm, beam, wasm)
This is the SAME binary. Running everywhere. Could be super-useful as an archival format for mission critical code. sqlite comes to mind.
Maybe it's late at night, but I'm struggling to find a suitable word for the project that is better than portable or cross-platform.
Literally, I can't find a better word than the project name - "actually portable executable."
@author - if you're reading this please set up something for a few bucks a month on your github sponsors. I don't think I have any use for the library, but this is so outrageous it deserves something more than imaginary internet points.
Thanks for the links. The discussion on the first one is interesting, it wasn't clear to me from the original doc that the UNIX version needs to overwrite itself to get to the ELF header. That's a bit unfortunate, because it means you can't share the program again after you ran it the first time on BSD or Linux. Still very clever, though. I imagine there could be a "repackage for distribution" switch added to put the MZ header back again.
I did the research and what we've accomplished here, while imperfect, is the best of all worlds solution. I like to think of it as an installer that takes a few microseconds, because it only needs to change 64 bytes. I've been considering adding a CLI flag where you can say ./foo.com --restore and it'll put the original 64 byte header back in place. Perhaps one day we can change the Linux, BSD, and XNU kernels so they can recognize the APE executable format. But until that happens we've got a great shell script hack that's now required by POSIX which is exceedingly fast and works on pretty much all systems stretching back decades.
I really don't understand internals ibut I've often downloaded cygwin exe file for commands like grep, tail, etc.. Are you telling me that is no longer required?
On linux, I downloaded the printvideo binary and played a friggin crab video on my terminal after just a chmod. I don't have any other OS to confirm, but it sure looks like it if someone compiles/links coreutils against this.
Everything she does has this level of jaw-dropping amazingness. Between her and Fabrice Bellard i don't know many people who consistently get my chin to hit the table.
I hope that Cosmopolitan becomes mainstream, so much more software could have that It Just Works quality.
Understood (with my apologies, I mistakenly assumed
it was on topic).
FWIW, I like and agree with the things I was referencing, I didn't mean it in any way as a jab or provocation. I shall endeavor to do a better job staying on the topic of the article in question.
Agreed, this is the best programming-related thing I've seen on the internet in a long time. It reminds me of when I was a teenager, excited to become a computer programmer. Then I grew up and joined the real world workforce and it was all far less exciting than I imagined.
The coolest thing about this Actually Portable Executable is that once there's a compiler and linker built with it, I can play around with writing C on Windows without having to faff about with WSL or MinGW or learning what makes MSVC different from the C I learned in university and coded in my first job before I became a Java/JavaScript/Python/etc programmer. When I discovered Go I thought I had discovered the better C that just worked everywhere, but the idea of having plain old C that also just works everywhere is very appealing.
I like Go for the same reason, but I have an irrational fear that someday I might be in a life or death situation and should program my way out using a bare bones computer without Internet; So I keep tabs on C occasionally.
αcτµαlly pδrταblε εxεcµταblε has now piqued my interest, So I might actually indulge in C programming for fun again after all. Thanks Justine.
I think it's really cool project; but could you explain why it's important that a binary works everywhere for you? I don't really get the enthusiasm for this sort of thing.
Maybe I have a blind spot or something or I've just been in Unix-land for too long.
I think the UNIX thing is the blind spot. People who run Linux or BSD don't seem so bothered to recompile for their own platform, safe in the knowledge that it will almost certainly just work.
But on Windows, trying to get open source software written in C to compile has been a pain for decades. There have been some attempts to make it work (notably Cygwin, and now MSYS2), but it can still be challenging, especially with programs that were never designed to be compiled for non-UNIX (or non-Linux!) targets.
Having a C library that is designed from the ground up to target multiple operating systems, and an executable format that means the first compile is the only compile, that takes away a lot of toil that programmers normally have when trying to build their own abstractions around each platform's C library. I guess from UNIX land you could see the benefit as removing the need for autoconf or platform ifdefs.
That malware for OS X that does nothing (so far) from the news earlier this week also has one multi-arch binary for all modern Apple notebook architectures (AMD64 and ARM64)
Agreed. This single sentence at the end of αcτµαlly pδrταblε εxεcµταblε page:
> I believe the best chance we have of doing that [writing software that stands the test of time with minimal toil], is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring the APIs.
...it's a kind of thought I don't think I could ever come up with. Mind blown.
Strong agree; someone needs to give her like a million dollars a year salary and unlimited resources to explore whatever she wants for life, just to see what she comes up with.
or don't, because she produced these gems without any such incentives and whilst I only make the connection in a speculative fashion, the most offended individuals I've known to be upset at the suggestion that money is a necessary component of compulsory exchange or even coercive production, have all been women.
Wow, that cosmopolitan C library is absolutely awesome. The ABI specialization is extremely interesting. Never seen anything like that before. I thought compilers did that. I don't understand why they couldn't properly optimize memcpy.
I wonder how it handles system calls. Let me check the source...
It's funny because the high-level introductions to C I would read as a kid before I really knew much about computing would almost always start out by explaining that one of the biggest strengths of C was its portability, and how it allowed you to write the same code that worked on every platform. Of course I came to learn that's true only in a sense that has no connection to practical reality, this looks like an exception.
Could you please explain to someone from the web domain with very little experience with compiled C programs why this is significantly better than, say, distributing a python program? Python runs on all the platforms mentioned. Or alternatively requiring compilation from source?
The python script/program ultimately depends on a Python interpreter which currently has to be distributed as a different binary for every platform.
So in the end, the difference is purely where the platform-differentiated compilation happens for VM & interpreted languages like Java & Python.
This would instead allow you to ship a single Python executable that simply works everywhere. Whether with a bundled-in script/program, or just the interpreter.
That sort of malware already exists. This does simplify making it by a lot, because now it can much more easily ensure its portability without external communication.
Not sure the is a working example. But they talk of embeding qemu to create cross archetecture executables. But the Window's and actual compiled code would still be x86-64.
She says somewhere in her website that arm/apple silicon support is really easy to add, with no significant size or performance penalty, the same way she added qemu support.
OK I see we are going crazy, me too. Thanks @jart.
I have been thinking about portable web apps with embedded SQLite for some time. I do not have the technical chops at C level to pull this off. I am really inspired by this project. I hope Redbean leads a way to distribute self-hosted apps in today's era of the cloud.
I still use the classic version of Tiddlywiki that is a single self-contained self-editing html file, and I "app-ify" it on Windows by changing the extension to *.hta.
But having the similar solution with the SQLite, would be a perfect combination.
The webserver is novel but I am just wondering, how would it compare to something like Apache/ISS in terms on security/config/etc, or is this mostly just to move small sites around?
AFAIK a fat binary is just a binary with all dependencies included. It would still be compiled for a specific platform. The magic part about this is that it's a single file compiled once that can be run on most platforms. And it also adheres to the format of a zip file, so you can add, modify and remove assets as you please inside the actual file, post compilation.
No, fat binaries are not binaries with all dependencies included. Fat binaries are executable files that contain equivalent compiled code for multiple architectures. Some also facilitate running binaries across different operating systems. More: https://en.wikipedia.org/wiki/Fat_binary
The idea also reminds me of (executable) self-extracting ZIP archives that were common once upon a time.
I am floored. Not only is this just breathtaking sorcery, I also needed this exact thing for a project. I don’t understand probably 70% of the description of this software, but I understand what it does, and I’m more excited to carefully study and use this than I have been about anything in years.
This is something like a "fat binary" with some magic in the header that allows it to run out of the box on Windows and several different UNIXes, a bit like those sneaky shell scripts that run on several platforms[0]. It still needs to include some native code for each of the platforms inside the compiled binary, so it's perhaps better to see it as a slick packaging/distribution solution for C programmers than a total game changer for developers everywhere.
The more interesting thing I think is the mapping of C library functionality between Linux, BSD and Windows. There have been other attempts to do this, but they never seem to get much traction. I think it's because C programmers tend to get wedded to their OS of choice and then invest time in improving that platform's C libraries. The problem with trying to remain portable with everything is that you tend to be stuck working with the lowest common denominator functionality, which can be frustrating when doing OS-specific stuff would open up so many more options.
All that said, the cleverness of this project is inspirational in a way that might encourage other programmers to also focus more on building against simple functionality that is truly portable. It shows that you don't need all the bells and whistles to build cool stuff. That feels aligned with early UNIX philosophy and (more broadly) the hacker ethic, which is why it's neat to see, even if never gets adopted as a mainstream thing.
Appreciate the response. So it's a bigger win for C devs specifically. I am a C# dev so I am trying to understand what opportunities this project unlocks.
Why you have pigeonholed yourself? "X Dev" is so cliche. You can compile C# to x86 machine code - this project has less to do with C and more to do with PE format and syscall tables. It means that it's language agnostic but not standard library agnostic.
Author here. Thanks! I was deeply inspired when I read the source code to early versions of Linux and UNIX. The dream of that simple elegant power is something I'm hoping to restore for a new generation. All I'm doing to achieve that is simply standing on the shoulders of the giants who came before me.
Have you considered trying to make system calls inline to avoid the call overhead? Or already thought about it?
I've been banging my head against the wall of trying to make a micro KVM guest and the glibc startup code uses inline SYSCALL everywhere, I assume to make things faster. It does seem to call CPUID a lot too, but I guess it's not as expensive as having to make full-blown system calls.
Cosmopolitan defines linkable symbols for all the __NR_syscall constants so you can absolutely use inlining if you like to live dangerously. For example: https://github.com/jart/cosmopolitan/blob/91f4167a45f811fde6... There really isn't any performance advantage to doing this, because the SYSCALL instruction itself has something like a 2000+ cycle cost due to all the copying that needs to happen on the kernel side along with things like spectre mitigations which have made is slower. So the 8 cycle cost of having a normal read() function wrapper is the wrong thing to be focusing on. Cosmopolitan aims to address the performance cost of SYSCALL by making it possible to run your binary on bare metal, where your program becomes a kernel, and therefore needn't pay any context switching costs at all.
Can you blog a bit of the career/cognitive/skill evolutionary history behind this? Did the cosmo stuff come out of this or something else?
Learned this in X level school, found out about Y, read about Y, learned this, got this job doing this, learned this from Z, got experience of AA... etc.
Of course, assuming this exists, and you exist. And blogs "exist".
Author here. When I first discovered the idea for Actually Portable Executable I thought I was going mad like Terry Davis. I couldn't even believe that it was possible until I actually did it. It cost me my job but I couldn't be happier with the result. This project is going to make problems disappear for so many developers. It's the sort of tool people can get excited about. I'm so happy that my prior career success and top-notch training have granted me the privilege of bringing something this cool to the world. Enjoy!
In all seriousness: whether it's exciting or not, this is something that should be getting federal funding. It really shouldn't be difficult for large, government-backed organizations to justify their interest. I don't have a lead for you (otherwise, I'd have invoked it by now to work on my own project[1], which has something in common through the use of polyglot files to achieve a similar effect[2], while mostly otherwise taking a different approach), but since you have a high profile, casting a wide net to attract the attention of someone able to secure the funding is something to consider.
Am I the only one having issues trying to get this working?
The webserver starts just fine, but once I add the index.html with zip as in the example, it stops working.
This is on Mac 10.15, CentOS 8 and Ubuntu 18.04 LTS.
Centos:
[centos@test ~]$ ./redbean.com -vv
error: Uncaught SIGSEGV on test.novalocal
./redbean.com
EINVAL/err=22/errno:2/GetLastError:0
Linux test.novalocal 4.18.0-240.10.1.el8_3.x86_64 #1 SMP
Mon Jan 18 17:05:51 UTC 2021
On the mac:
~ my-Maccie$./redbean.com -vv
Killed: 9
On Ubuntu:
ubuntu@localhost:~$ ./redbean.com -vv
error: Uncaught SIGSEGV on localhost
./redbean.com
EINVAL/err=22/errno:2/GetLastError:0
Linux localhost 4.15.0-136-generic #140-Ubuntu SMP Thu Jan
28 05:20:47 UTC 2021
Author here. The zip shouldn't need to be saved in any special format. If WinRAR is posing problems then please file an issue because I'd like to fix that. That error doesn't seem like something that would indicate executable corruption. I want to learn more.
Author here. It works fine on RHEL5 / CentOs5 for me. I can also confirm RHEL7. Please file an issue. What will help in particular is if you can give me the faulting RIP address. That should be in the crash report or in your dmesg log.
$ zip -v
Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.
Currently maintained by E. Gordon.
Addition of index.html shrinks(!) redbean.com from size 204800 to size 204348, with changes starting from char 172938. That clearly corrupts the portable executable format.
IIRC, from vague memories of things flying past, Big Sur won't let a modified binary run once it's been checked by Gatekeeper. Which means the first run gets checked and notarised but when you add the `index.html`, the on-disk binary has changed and Gatekeeper won't allow it to run - I guess it's to prevent malicious code modifications, etc.
> This will also prevent distribution from outside of their appstore giving it will prevent updates.
I think that if an app is correctly codesigned, Gatekeeper has no issue with it. Also if it's not correctly codesigned but you've ticked "allow unsigned binaries", you can still run them.
> the mothership knows everything you run on your machine
I can't remember the details from the last time this came up but IIRC it only sends a hash and possibly even then only the first time you launch it. Either way, they're not going to block out-of-appstore distribution except if you're a known malicious actor.
It looks like you can get around this by adding Terminal.app as a "developer tool" (i.e. it can create processes without hitting Gatekeeper), maybe that works? (once I did that and relaunched terminal, I can run cosmopolitan binaries).
Yeah another killed: 9 on Mac BigSur.
Could you export the tool in a way that we don't have to zip new files into it? Instead we would have to gcc the whole thing once and be done with it.
EDIT: I am an idiot: it's working just fine... the messages looked like an error to me, but visiting https://localhost:8080/ does show a nice HTML page!
No luck for me on win10. I can open the archive but get errors trying to add to it. 7zip and win explorer zip both fail with either unsupported or corrupted archive errors.
Since it's a .com file, after downloading, win 10 also claims 'this dangerous file comes from the internet and is blocked' which you have to check a box to unblock it. I did, but still get the errors trying to add to it.
That's strange. I'd recommend filing an issue. Any details on how I can reproduce the error will be helpful, even if I have to temporarily rent one of those things. I'm on a mission to make binaries portable.
FWIW, it works on QubesOS, which is Xen based. Though it works only in my Fedora and Debian VMs, but not on my Archlinux-based one.
I think that's because I have WINE installed in the Archlinux VMs, since that's what gets started when I run ./redbean.com. But even after I remove WINE, it still doesn't work, I probably have to restart (i.e. remove WINE from the TemplateVM, not the AppVM) for it to work.
It's a neat coincidence that this is on at the same time as an article about CP/M, because I think there could be some overlap in the programs that would be a good fit for cosmopolitan.
Sure, there's server appliances like this, and I hope someone makes a neat services wrapper to abstract at least some platform's intricacies so you can do a "cosmo-service up redbean" on BSD/Linux/Windows.
But coming back to CP/M, there you had a lowest common denominator of terminal applications, too, but spread across different architectures. And you could still produce some quite intersting, if a bit business-like applications. These days you probably can even rely on more ANSI colors and maybe even unicode fonts (hopefull as an option, not mandatory).
The ZIP characteristics of the APE format make it even easier to distribute a whole application in a rather simple way. Yes, sure, you can do regular Unix-style servers and pipe-it-together CLI tools, but I wouldn't mind more self-contained "business" applications with a lo-fi aesthetic regarding interface and API usage. The PICO-8 of TUIs…
Author here. We're living in the most exciting time for developing terminal applications. When Microsoft unexpectedly added support for VT100 and XTERM codes to CMD.EXE it totally changed the equation and ANSI became universal for the first time. Blinkenlights is an example of a TUI application I created using Cosmopolitan and it literally works everywhere. https://justine.lol/blinkenlights/index.html You don't need curses. All that's needed is an ioctl() call which flips a bit in termios. Cosmopolitan polyfills that across operating systems. Another cool example of a demo app is this conway's game of life tui gui: https://justine.lol/apelife/index.html
Cosmopolitan supports Windows 7. The WinMain() polyfill is programmed to gracefully fall back if ANSI isn't available. So the binary still loads fine. You just need to run it in something like MinTTY instead if your TUI program is run on an old version of Windows.
Just make sure you don't have another server running on :8080.
I had syncthing running and kept getting some error whenever I tried to run redbean. The error didn't make much sense, but eventually I realised that this was the casue of the error.
I'm really impressed how portable this is. The only improvement I can see is if it auto-opened your browser to :8080. That way it would be easy to distribute the binary and have people run your application and just interact with it through their browser. No need to ship electron then!
There is the possibility of other programs making HTTP requests so you shouldn't assume trust on any request automatically. What you can do is give the browser a cookie once when your app starts which you then check on every request.
Oh and listen on 127.0.0.1, not 0.0.0.0. That way your app is not needlessly exposed to the network.
I think you could trivially issue a system() call, you'd just have to use the corresponding command to open the default browser for each particular system. (Linux: xdg-open, MacOs: open, Windows: start)
Author here. The name was chosen mostly out of playfulness to describe Windows since I seem to recall XENIX being the earliest example of Microsoft dipping their toes in the water with UNIX support.
265 comments
[ 0.19 ms ] story [ 568 ms ] threadFrom the same person I think
Even if Redbean here doesn't end up taking over the world, I hope that more ecosystems and toolchains sit up and take notice. We need more tools that look like this.
Would it be possible to do something similar with Free Pascal, allowing the recreation of something like Turbo Pascal, except really, really portable .com output?
Also hats off to you @jart!
Is this a one time thing or are you thinking of something bigger going with this project?
I had a quick look into doing this on your original thread[1], but got stuck pretty early on and just didn't have the time to get back into it.
If that's something you'd be willing to pursue I'd love to help any way I can. Feel free to DM me on Twitter[2] any time.
1 - https://news.ycombinator.com/item?id=25558643
2 - https://twitter.com/d0m96
chibicc is a really fun idea! If I was in college, I would take a compiler course from you.
Taken from your README:
"chibicc is developed as the reference implementation for a book I'm currently writing about the C compiler and the low-level programming. The book covers the vast topic with an incremental approach; in the first chapter, readers will implement a "compiler" that accepts just a single number as a "language", which will then gain one feature at a time in each section of the book until the language that the compiler accepts matches what the C11 spec specifies. I took this incremental approach from the paper by Abdulaziz Ghuloum."
Github: https://github.com/rui314/chibicc
Previous HN: https://news.ycombinator.com/item?id=24676851
So I would edit my comment to say: I'd love to take a compiler course from Rui, and a systems course from jart.
That's what I get for browsing HN on my phone late at night.
As for holding off on UI for the time being is probably the right thing to do, keeping with the minimal style planning on going for raw-x11,etc.
However, as much as I personally love plain framebuffers (late 90s democoder myself), it'd be totally non-accessible for "plain" applications and totally lacking performance compared to anything accelerated when it comes to rendering. Graphics cards are just so many times faster that it's not really even the same kind of usage (only sad part is Apple punting on OpenGL driver updates even if Intel on the low end has become so much better).
That said, having looked for a minimal UI framework and sadly they seem to be far inbetween these days (libUI is fairly compact on win/osX but required GTK on *nix but they've lost a bit of steam since 2018)
Best of luck on it all (hopefully I can snatch out some time to contribute)
As for GUIs I'm still open to merging raw x11 support. One tricky issue is that we'd need to define a new API that veneers framebuffers, and so far Cosmopolitan has mostly refrained from defining novel interfaces, since the focus has been twiddling with low level magic numbers in order to make the textbook interfaces we already have work everywhere!
Considered making some way to add "portable" modules yet(generic APE code with plat-specific impls) to avoid adding stuff to the core? I know the default calling-conv is slightly different but maybe by smth like defining virtual ports? (ie "send" structs since alignment rules should be mostly the same?)
Admittedly they solve different usecases, where Redbean seems to want to serve a whole static site as a single file, which isn't currently possible with Caddy, (but you can run `caddy file-server` to serve the current directory as a static site), but it may become possible in the future with Golang's new embed package https://golang.org/pkg/embed/
All this would be roughly 15 lines of code.
Thanks for sharing.
You are affiliated with Caddy and ought to mention that when you promote your own software. https://news.ycombinator.com/item?id=16687686
This is not a competition. Caddy is not even in the same realm as Redbean.
Redbean is a multi-platform binary that doubles as a ZIP. It's roughly 1500 times smaller than Caddy too.
> different usecases
Right. :)
What? Of course not! You're reading way too much into that. It's simply enthusiasm. I read about Redbean and it being a single file webserver and the first thing to come to mind is Caddy, as "prior art".
Like I said, different usecases; Caddy is a general purpose webserver which has automatic certificate management, H2/H3 support, reverse proxy, file server, a strong plugin system, the list goes on. Of course it'll be bigger in size. Redbean is super cool and does have the advantage in file size and portability, clearly, but it is essentially a single-purpose tool. And there's nothing wrong with that.
Redbean is a portable, single file executable webserver which also acts as a zip-file for the static content it hosts, and runs on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS without any recompilation...
You can manage the static content using standard pkzip tooling! Now that's what I call thinking out of the box!
Edited: some rewording and breathing space
And does it run on different architectures like ARMv6?
https://github.com/jart/cosmopolitan/blob/40291c9db386d8a952...
If i have understood how the portability works, it would be pretty straightforward to add support for another OS which uses the same ABI as the unices and has a compatible system call table - just a matter of adding a column to consts.sh (system call numbers in the nr section, constants from system headers in their appropriate sections) and adding any OS-specific system call rituals to systemfive.S.
Note that all system calls will go through the syscall interface. If your OS has some other way to make system calls, like Linux's vDSO or io_uring, that won't be used.
If your OS had a different system call table, like Windows, then you would need to write more bespoke code for it, like all the Windows wrappers:
https://github.com/jart/cosmopolitan/tree/0e85b136ae1beb0d2b...
You would then need to go through the Cosmopolitan libc and add implementations of its functions on top of your wrappers.
Justine has built a c library that allows you compile a binary once and have it run it on any os or baremetal. The SAME binary. Quite frankly, that sentence doesn't even make sense to me.
Check out https://storage.googleapis.com/justine/cosmopolitan/index.ht...
As far as I'm concerned, this is literal magic. Look at the magic numbers: https://github.com/jart/cosmopolitan/blob/37a4c70c3634862d8d...
I could go on, but there's no binary portability comparison with any other language. And she has made some pretty neat optimizations.
Back in the day, I saw some pretty neat stuff with the ELF format, but this takes the cake.
Wow.
Edit: I'm editing because this is just so bloody absurd.
https://storage.googleapis.com/justine/printimage.html
$ ./printimage.com someimage.jpg
Like wow. And also video.
https://storage.googleapis.com/justine/printvideo.html
I'm struggling to put my shock into words. I've been around.
There's engineering. There's academia.
But this falls into straight-up wizardry.
It's hard to grok cause there are no words to describe it.
When people say that something is cross-platform, they usually mean a) that the software will build on multiple platforms b) there is some sort of vm which runs the executable (jvm, beam, wasm)
This is the SAME binary. Running everywhere. Could be super-useful as an archival format for mission critical code. sqlite comes to mind.
Maybe it's late at night, but I'm struggling to find a suitable word for the project that is better than portable or cross-platform.
Literally, I can't find a better word than the project name - "actually portable executable."
@author - if you're reading this please set up something for a few bucks a month on your github sponsors. I don't think I have any use for the library, but this is so outrageous it deserves something more than imaginary internet points.
If interested, see previous discussions of APE:
https://news.ycombinator.com/item?id=24256883
https://news.ycombinator.com/item?id=25556286
Seems more likely to have installed programs on read-only media and a little tmpfs, instead of a writable binary and no tmpfs.
Do you have any information on some of the other strategies you researched?
I hope that Cosmopolitan becomes mainstream, so much more software could have that It Just Works quality.
FWIW, I like and agree with the things I was referencing, I didn't mean it in any way as a jab or provocation. I shall endeavor to do a better job staying on the topic of the article in question.
The coolest thing about this Actually Portable Executable is that once there's a compiler and linker built with it, I can play around with writing C on Windows without having to faff about with WSL or MinGW or learning what makes MSVC different from the C I learned in university and coded in my first job before I became a Java/JavaScript/Python/etc programmer. When I discovered Go I thought I had discovered the better C that just worked everywhere, but the idea of having plain old C that also just works everywhere is very appealing.
αcτµαlly pδrταblε εxεcµταblε has now piqued my interest, So I might actually indulge in C programming for fun again after all. Thanks Justine.
Maybe I have a blind spot or something or I've just been in Unix-land for too long.
But on Windows, trying to get open source software written in C to compile has been a pain for decades. There have been some attempts to make it work (notably Cygwin, and now MSYS2), but it can still be challenging, especially with programs that were never designed to be compiled for non-UNIX (or non-Linux!) targets.
Having a C library that is designed from the ground up to target multiple operating systems, and an executable format that means the first compile is the only compile, that takes away a lot of toil that programmers normally have when trying to build their own abstractions around each platform's C library. I guess from UNIX land you could see the benefit as removing the need for autoconf or platform ifdefs.
> I believe the best chance we have of doing that [writing software that stands the test of time with minimal toil], is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring the APIs.
...it's a kind of thought I don't think I could ever come up with. Mind blown.
but yea I agree
I wonder how it handles system calls. Let me check the source...
https://github.com/jart/cosmopolitan/blob/master/tool/build/...
https://github.com/jart/cosmopolitan/blob/master/tool/build/...
> void OpSyscall(struct Machine *, uint32_t);
Pretty interesting... Need some time to figure out how it works.
The system call entry points:
https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...
https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...
It uses a jump slot technique which is interesting:
https://github.com/jart/cosmopolitan/blob/master/libc/sysv/m...
System call numbers:
https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c...
https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...
Looks like it packs the system call numbers of all operating systems into a single constant.
For some reason there are some dedicated implementations of Linux system calls using inline assembly rather than the entry point:
https://github.com/jart/cosmopolitan/tree/master/libc/linux
More related files:
https://github.com/jart/cosmopolitan/blob/master/libc/calls/...
https://github.com/jart/cosmopolitan/blob/master/libc/calls/...
Also funny how it calls Linux "GNU/Systemd"!
Yeah, it would have taken less bits to use GNU/POS instead, because it ain't Linux anymore.
Poettering Operating System.
So in the end, the difference is purely where the platform-differentiated compilation happens for VM & interpreted languages like Java & Python.
This would instead allow you to ship a single Python executable that simply works everywhere. Whether with a bundled-in script/program, or just the interpreter.
But, the paranoid in me is now curious/afraid if this approach can/will lead to multi-OS malware?
I have been thinking about portable web apps with embedded SQLite for some time. I do not have the technical chops at C level to pull this off. I am really inspired by this project. I hope Redbean leads a way to distribute self-hosted apps in today's era of the cloud.
But having the similar solution with the SQLite, would be a perfect combination.
The webserver is novel but I am just wondering, how would it compare to something like Apache/ISS in terms on security/config/etc, or is this mostly just to move small sites around?
The idea also reminds me of (executable) self-extracting ZIP archives that were common once upon a time.
Will I be able to write apps in a single language and have then run truly native on any OS or what else will we be able to do?
The more interesting thing I think is the mapping of C library functionality between Linux, BSD and Windows. There have been other attempts to do this, but they never seem to get much traction. I think it's because C programmers tend to get wedded to their OS of choice and then invest time in improving that platform's C libraries. The problem with trying to remain portable with everything is that you tend to be stuck working with the lowest common denominator functionality, which can be frustrating when doing OS-specific stuff would open up so many more options.
All that said, the cleverness of this project is inspirational in a way that might encourage other programmers to also focus more on building against simple functionality that is truly portable. It shows that you don't need all the bells and whistles to build cool stuff. That feels aligned with early UNIX philosophy and (more broadly) the hacker ethic, which is why it's neat to see, even if never gets adopted as a mainstream thing.
[0] https://github.com/llamasoft/polyshell
I've been banging my head against the wall of trying to make a micro KVM guest and the glibc startup code uses inline SYSCALL everywhere, I assume to make things faster. It does seem to call CPUID a lot too, but I guess it's not as expensive as having to make full-blown system calls.
Can you blog a bit of the career/cognitive/skill evolutionary history behind this? Did the cosmo stuff come out of this or something else?
Learned this in X level school, found out about Y, read about Y, learned this, got this job doing this, learned this from Z, got experience of AA... etc.
Of course, assuming this exists, and you exist. And blogs "exist".
1. <https://wiki.triplescripts.org/wiki/The_why_and_how_of_tripl...>
2. E.g., <https://www.youtube.com/watch?v=TUpd70Mu0Ek>
Centos: [centos@test ~]$ ./redbean.com -vv
On the mac: ~ my-Maccie$./redbean.com -vv Killed: 9On Ubuntu: ubuntu@localhost:~$ ./redbean.com -vv
EINVAL/err=87/errno:2/GetLastError:203 The parameter is incorrect.
Probably the ZIP needs to be saved in a certain format.
https://pastebin.com/8RyPWQFf
This is on Windows 10 19042 build.
7-Zip doesn't even see the folders for that matter.
The binary didn't work on Ubuntu 18.04 nor CentOS8 either, but I'll report a issue later on today.
This will also prevent distribution from outside of their appstore giving it will prevent updates.
So as anything Apple, the world security here is being used as code for personal machines remotely controlled by the mothership.
You trade privacy and freedoms for a little convenience.
I think that if an app is correctly codesigned, Gatekeeper has no issue with it. Also if it's not correctly codesigned but you've ticked "allow unsigned binaries", you can still run them.
> the mothership knows everything you run on your machine
I can't remember the details from the last time this came up but IIRC it only sends a hash and possibly even then only the first time you launch it. Either way, they're not going to block out-of-appstore distribution except if you're a known malicious actor.
https://docwhat.org/upgrading-to-catalina
`sudo spctl --master-disable`
EDIT: I am an idiot: it's working just fine... the messages looked like an error to me, but visiting https://localhost:8080/ does show a nice HTML page!
Since it's a .com file, after downloading, win 10 also claims 'this dangerous file comes from the internet and is blocked' which you have to check a box to unblock it. I did, but still get the errors trying to add to it.
[1] https://github.com/jart/cosmopolitan/commit/218ef491476f15ab...
run-detectors: unable to find an interpreter for ./printimage.com
I think that's because I have WINE installed in the Archlinux VMs, since that's what gets started when I run ./redbean.com. But even after I remove WINE, it still doesn't work, I probably have to restart (i.e. remove WINE from the TemplateVM, not the AppVM) for it to work.
In any case, incredible work, I am in awe!
Sure, there's server appliances like this, and I hope someone makes a neat services wrapper to abstract at least some platform's intricacies so you can do a "cosmo-service up redbean" on BSD/Linux/Windows.
But coming back to CP/M, there you had a lowest common denominator of terminal applications, too, but spread across different architectures. And you could still produce some quite intersting, if a bit business-like applications. These days you probably can even rely on more ANSI colors and maybe even unicode fonts (hopefull as an option, not mandatory).
The ZIP characteristics of the APE format make it even easier to distribute a whole application in a rather simple way. Yes, sure, you can do regular Unix-style servers and pipe-it-together CLI tools, but I wouldn't mind more self-contained "business" applications with a lo-fi aesthetic regarding interface and API usage. The PICO-8 of TUIs…
I had syncthing running and kept getting some error whenever I tried to run redbean. The error didn't make much sense, but eventually I realised that this was the casue of the error.
I'm really impressed how portable this is. The only improvement I can see is if it auto-opened your browser to :8080. That way it would be easy to distribute the binary and have people run your application and just interact with it through their browser. No need to ship electron then!
Oh and listen on 127.0.0.1, not 0.0.0.0. That way your app is not needlessly exposed to the network.
> Is the Cosmopolitan Runtime lean and mean like Go's 2mb Hello World executables?
:-)
this is delightful