I am really confused by one part of your blog post: it says that you had to embed Lwip as applications don't send/receive IP packets... but you also describe this as x86 virtualization running specifically-Linux, and Linux absolutely has its own TCP stack, so I don't understand what Lwip is actually accomplishing here... I would have thought you'd essentially be implemented as a paravirtualized network adapter, but then the article mentions system calls? I guess I just fundamentally don't understand at what level of the stack you have this implemented... did you replace the Linux kernel and reimplement its entire system call layer?
CheerpX currently focuses on user mode (ring 3), and does not fully emulate the kernel (ring 0). We do, however, implement a subset of the Linux system call interface, which is enough to run most applications.
Can we use custom derp/headscale setups? Would be interesting even for use internally then, hell rolling custom versions with preloaded software could also be pretty cool all connected to a corporate headscale/derp.
Could be cool to see an implementation using Chrome's experimental Direct Sockets API [0]. The API requires using a form of new web packaging known as Isolated Web Apps which have a strict CSP [1] though, so it might not be possible to use it for this since I'd assume it's dynamically executing a lot of code.
It’s a cool hack - what can I say. Surprised at how it instantly loads to a usable shell, too, if there’s a lesson to be learned there. Is there a freeze-dried memory image being used?
Absolutely not. The app starts at the entry point and execute instructions as normal. Both the main executable and libraries are all loaded on demand from a disk image delivered chunk-by-chunk over HTTP. The disk backend is based Clouflare workers with multiple levels of caching to maximize perf.
Given the absolute dysfunction on how browser standards are adopted this is expected behavior. Huge security risk, huge implementation cost, very low competition, and people hate electron apps.
I tried compiling mandelbrot.c (single threaded one) from the benchmarks game and execution on jslinux took 12 seconds, while webvm completed it in 1.2 sec. The host takes 0.03 sec with the same gcc flags.
Hi, Lead Dev of WebVM and CTO of Leaning Technologies here. Happy to answer any questions you might have... potentially in a few hours, it's quite late here :-)
Hello there, I havent tried webvm yet but looks similar to v86. There were lot of network related challenges working with v86, for eg. You cant do curl, DNS resolution does not work out of the box. How did you address these challenges in webvm. I think it is very interesting and exciting what you guys are building.
You want it to be 3D!! Loading this is a multiplayer social VR sandboxed WebXR environment would be so amazing one day! I wonder if it could all run off-thread in a Worker and one day maybe use webGPU to update textures async! I hope computers can be fun and care free like a game one day and AI companions will let people learn and do anything! Thanks for working to make this!
Right, I can go to insert mode, but to leave is, tricky. I found going to replace mode and escape worked in FF. I'm sure its a browser issue, I also have vimium so probably a nightmare of hooks!
Have you tried with <Control-Left Square Bracket>? aka Ctrl-[, or ^[, as seen sometimes in the terminal. This works for me in most terminals as an alternative to Escape when using vi/vim.
In mobile (at least in mine, firefox) the input become buferred autocomplete and when accepted via return, the typed word gets duplicated before entered. Workaround is to accept via space and delete as necessary
I remember back in the day I wouldn't even check the listings, just joining whatever #topic or #hobby or #software and finding it filled with people almost always.
Also see https://oftc.net/, where a bunch of older-school development projects hang out. (That's where the Xen Project moved to after freenode imploded, before eventually switching to Matrix.)
Another option is a community hosted Mattermost (it's like Discord/Slack/IRC). It's pretty easy to run, all the chat are in PostgreSQL so it's "easy" to produce archive logs (like IRC has). Or do any other magic.
IRC <3 Still daily driving it with some friends. I wouldn’t be surprised if my Discord chat history was unavailable in a decade, so IRC is a nice option to run on the side. There’s value in simplicity, and I admit the risk of sounding like a tech hipster.
IRC has no chat history either, right. I get the simplicity of IRC but searchable history is a bonus for Discord. As long as the service is available, searching is kinda possible. With irc you have to find out which bot provides history, which is then usually split over multiple files
My client dies the logging, and I can e.g. grep decade old logs. I’m not sure if you can get same level of access to Discord logs (=export them). I guess Discord bot that logs everything as a historian is a partial solution (I guess log bot cannot catch DMs).
> Last but not least, an IRC server/channel in lieu of discord? I'm stuck in my ways!
IRC is nice for people who use it, but it has a huuuuge barrier to entry for new users. The very first thing you have to do is choose a client out of a list of 100 clients, half of which haven't been maintained since 2003. Then you need to set up a bouncer somewhere so that you can see messages that were posted when you were away. Those two things will filter out 99.99% of new people.
Or, you could use Discord (or Matrix); people sign up for an account and that's it.
Our open-source project switched from IRC to Matrix (after having been on IRC for over a decade before that), and got a noticeable uptick in engagement.
Unfortunately, Matrix still has some long-standing UI quirks, which would make me reluctant to use it in a context where I want people to pay me money; particularly if some of those are non-technical users.
3. Discover you can't join a lot of channels because you're not registered; or worse, that your messages sent to the channel are silently discarded
And you still have the problem that if you're in the wrong time zone and happen to close the window, you may miss responses.
There are advantages to having high barriers to entry for a community; Ham radio is essentially a global chat room with an entrance exam, for instance. But there are costs too.
Your point 3. is true generally, but - as I understand - parent wanted to operate a customer support channel and dismissed IRC. In this case: just don't lock your channel for Guests.
Timezones affect other platforms, but an IRC bot might help customers when admins sleep.
But I'm not sure IRC is best for this use case (e.g. forums, wiki are probably better), but "high barrier of entry" is arguable.
Hey Alessandro - I'm a huge fan of WebVM + y'all's other work at Leaning Tech.
Curious - is your implementation of how you wrangled lwIP for the networking piece open source? I had previously read the article you linked elsewhere, re: networking, but would love to see the specifics of your approach there.
All the components of the network stack are open source: namely the Tailscale go client and the lwip networking stack. Some (not fully trivial) glue code is also required, which is not currently published. We do plan to cleanup the stack and release it as one of our upcoming internship projects.
I tested in Firefox on Linux. It works well except:
1. Copy and paste does not work and this could be bad for basically every editing task (programming etc).
2. The screen freezes sometimes and keypresses don't appear. Usually pressing backspace unfreezes the screen. Firefox about:performance doesn't show any CPU usage in the tab. Linux's htop (on my machine) doesn't show anything strange with Firefox.
And a question: I saw that new files persist after closing the tab and opening it again. I didn't investigate my localstorage etc with Developer Tools. Did you use that or some other browser feature? How truly persistent are those files?
1. Support for traditional copy/paste shortcuts is problematic, since Ctrl-C is interpreted as the SIGINT signal. Copying via the mouse menu should work. Pasting is supported via Ctrl+Shift+V.
To answer you question: filesystem chunks are cached/persisted using a local IndexedDB. It provides a fairly strong level of persistence, although the user can wipe it out of course.
Copy&paste via menu works (I didn't think about it) and Ctrl+Shift+V works too. Ctrl+Shift+C opens Firefox's Developer Tools. It's difficult to find a reasonable shortcut.
Could it be possible to run containers within this environment?
I ask because I personally think we are headed towards a future where web browsers are just sandboxed container environments where we download and run containers for each app...
Eventually, even calling them "web" browsers might feel a little odd, almost the way in which we "dial" a phone number today, even though we now mostly key it in on a touch screen.
Well, the environment itself is a container. Supporting Linux "namespaces" used to implement containers is also a possibility.
As described in the project README we provide a GitHub Action workflow to convert Dockerfiles into bootable images for WebVM: https://github.com/leaningtech/webvm
These are full system emulators, this is more a compatibility layer ("Linux syscall emulator") for emulated userland binaries. I.e. a bit higher level and a bit more usable/flexible as a result.
Beyond the userland-only limitation, syscall emulation is probably incomplete at best[0], and likely doesn't pass the Linux test suite for the subset of calls that are supported.
This runs a NodeJS server and frontend in your browser so you can see what Graphweaver can do without having to install or run anything more than a browser on your computer. The example has an SQLite database joined to a REST API. Graphweaver serves a GraphQL API with the result.
Working offline is not truly possible since that would require downloading the full 2GB disk image, which is something that is deliberately avoided by the system architecture.
State is already locally maintained via IndexedDB, no data is ever sent back or saved on our servers.
Totally unusable on mobile unfortunately (at least Firefox and Chrome on Android). Might be a limitation of how the keyboard works, but every command gets duplicated (at least) when I enter it.
Note that "CheerpX enables you to run existing 32-bit x86 native binaries".
For some reason support for wasm64 (in browsers) has been stagnated for years, which is a pity.
136 comments
[ 3.3 ms ] story [ 200 ms ] threadAlso, a little below:
CheerpX currently focuses on user mode (ring 3), and does not fully emulate the kernel (ring 0). We do, however, implement a subset of the Linux system call interface, which is enough to run most applications.
[0]: https://github.com/WICG/direct-sockets/blob/main/docs/explai...
[1]: https://github.com/WICG/isolated-web-apps/blob/main/README.m...
Even better, consider joining our Discord: https://discord.gg/yTNZgySKGa
* Education (Linux, Programming, Security, ...)
* Live docs for arbitrary languages and binary libraries
* Preservation of historical software and games
* Virtualization of legacy Windows enterprise apps.
* Dev environment for Web IDEs
Just a few examples, the list could go on for long
What are the use cases you Invision?
Last but not least, an IRC server/channel in lieu of discord? I'm stuck in my ways!
Nice work too, I hope this doesn't come across as flippant.
EDIT: Side note, I opened vi and escape is.. well maybe a browser issue but 'cant exit vi' .. Never thought id say that again
The trend of using discord over forums does suck though, it’s really hard to search old discussions.
A channel listing will give you _many_ results.
It's not like it used to be in the 90s but that's still something.
Seems reasonable to go with the growing platform with 1000x more users.
I agree! I am a nobody and forever will be a proud nobody!
Not OP, but they said the following in an answer to similar question: https://news.ycombinator.com/item?id=40940707
"The technology is extremely flexibile, off the top of my head:
* Education (Linux, Programming, Security, ...)
* Live docs for arbitrary languages and binary libraries
* Preservation of historical software and games
* Virtualization of legacy Windows enterprise apps.
* Dev environment for Web IDEs
Just a few examples, the list could go on for long"
IRC is nice for people who use it, but it has a huuuuge barrier to entry for new users. The very first thing you have to do is choose a client out of a list of 100 clients, half of which haven't been maintained since 2003. Then you need to set up a bouncer somewhere so that you can see messages that were posted when you were away. Those two things will filter out 99.99% of new people.
Or, you could use Discord (or Matrix); people sign up for an account and that's it.
Our open-source project switched from IRC to Matrix (after having been on IRC for over a decade before that), and got a noticeable uptick in engagement.
Unfortunately, Matrix still has some long-standing UI quirks, which would make me reluctant to use it in a context where I want people to pay me money; particularly if some of those are non-technical users.
1. go to https://web.libera.chat/
2. click Start (as a Guest account)
And you still have the problem that if you're in the wrong time zone and happen to close the window, you may miss responses.
There are advantages to having high barriers to entry for a community; Ham radio is essentially a global chat room with an entrance exam, for instance. But there are costs too.
Curious - is your implementation of how you wrangled lwIP for the networking piece open source? I had previously read the article you linked elsewhere, re: networking, but would love to see the specifics of your approach there.
edit: Saw your other reply!
https://labs.leaningtech.com/blog/webvm-virtual-machine-with...
1. Copy and paste does not work and this could be bad for basically every editing task (programming etc).
2. The screen freezes sometimes and keypresses don't appear. Usually pressing backspace unfreezes the screen. Firefox about:performance doesn't show any CPU usage in the tab. Linux's htop (on my machine) doesn't show anything strange with Firefox.
And a question: I saw that new files persist after closing the tab and opening it again. I didn't investigate my localstorage etc with Developer Tools. Did you use that or some other browser feature? How truly persistent are those files?
2. We had these reported from several users and we will investigate as soon as we can: https://github.com/leaningtech/webvm/issues/102
To answer you question: filesystem chunks are cached/persisted using a local IndexedDB. It provides a fairly strong level of persistence, although the user can wipe it out of course.
Copy&paste via menu works (I didn't think about it) and Ctrl+Shift+V works too. Ctrl+Shift+C opens Firefox's Developer Tools. It's difficult to find a reasonable shortcut.
I don't believe I've ever seen Ctrl+insert and shift+insert makes to anything _other_ than copy and paste.
Why have everybody kept using Ctrl-Shift-C and Ctrl-Shift-V since forever included myself?
I ask because I personally think we are headed towards a future where web browsers are just sandboxed container environments where we download and run containers for each app...
Eventually, even calling them "web" browsers might feel a little odd, almost the way in which we "dial" a phone number today, even though we now mostly key it in on a touch screen.
As described in the project README we provide a GitHub Action workflow to convert Dockerfiles into bootable images for WebVM: https://github.com/leaningtech/webvm
VMs
https://copy.sh/v86/
https://bellard.org/jslinux/
https://jamesfriend.com.au/pce-js/ (https://github.com/jsdf/pce)
https://www.pcjs.org/ (lots of hardware and OSes) (https://github.com/jeffpar/pcjs)
Mac OS
https://infinitemac.org/ (https://blog.persistent.info/2023/03/infinitemac-dot-org.htm...)
https://jamesfriend.com.au/projects/basiliskii/BasiliskII-wo...
https://jamesfriend.com.au/pce-js/pce-js-apps/
To get a flavor of the work required when emulating the kernel, you can read up on the work it took Windows developers to emulate Linux syscalls on top of the NT kernel for WSL 1 here: https://learn.microsoft.com/en-us/archive/blogs/wsl/wsl-syst...
WSL devs eventually abandoned that effort in favor of running the actual kernel in a VM -- that's how WSL 2 is implemented.
[0] I'd be surprised if sys_fork actually forks the process using a web worker, for example.
On webvm.io
> $ time python3 -c "max(range(10**7))"
real 0m1.558s
On https://copy.sh/v86/?profile=archlinux
> # time python3 -c "max(range(10**7))"
real 0m5.283s
[1] https://duckduckgo.com/?q=comptuers+built+in+minecraft
The technology is extremely flexibile, off the top of my head:
* Education (Linux, Programming, Security, ...)
* Live docs for arbitrary languages and binary libraries
* Preservation of historical software and games
* Virtualization of legacy Windows enterprise apps.
* Dev environment for Web IDEs
Just a few examples, the list could go on for long
This runs a NodeJS server and frontend in your browser so you can see what Graphweaver can do without having to install or run anything more than a browser on your computer. The example has an SQLite database joined to a REST API. Graphweaver serves a GraphQL API with the result.
If a browser is not supported, the site should display a specific error message.
State is already locally maintained via IndexedDB, no data is ever sent back or saved on our servers.
Error: /proc must be mounted
I’ve said for a while that a browser is basically its own operating system. Now it can host operating systems!
Also, for the use case of local dev environment for Web IDEs, this doesn't look fast enough.
runs fast after the first time, but executing any JavaScript is very, very slow: