151 comments

[ 4.9 ms ] story [ 214 ms ] thread
cool, looks like the natural extension of client-side programming.  why should we be limited to running things in the browser while our local machines remain underutilized? just spin up a sandboxed VM via Arc - brilliant! and - like so often - a seemingly obvious idea in hindsight... 

should also be great for downloading torrents in a sandboxed environment.

This is pretty cool. It has always seemed sort of silly to me that I have powerful so many powerful computing devices in my life and yet most of the important applications in my life are inherently centralized and running on someone else's servers in a far away location. Obviously the power of a remote datacenter is necessary for many applications, but for others it seems unnecessary or even like a hindrance.
Hasn't this already been done with Java applets? Would the difference here be that instead of running Java, you could run almost any language?
> Would the difference here be that instead of running Java, you could run > almost any language?

More than just any language - any Linux software.

And what are the dependencies? What is the lifetime of the VM? It says that Arc uses VirtualBox, does that mean I would need a full install of VirtualBox to use this?
Yes, I just tried it. The first screen on the downloaded installer says it will download & install Virtualbox, and run Virtualbox + Arc upon system startup.
VirtualBox is included in the Arc installer. It doesn't have to be downloaded or installed separately.
Congratulations! This is a great idea. If I needed port a network and/or performance critical Linux app to the web, Arc offers some unique advantages. However there is some serious competition from Java apps, jslinux, Emscripten w/ asm.js, and (P)NaCl.

I don't have a Mac, so I haven't tried out your demo yet, but ideally you should make this work like genymotion, using the existing VirualBox install in headless mode. This would also make a much quicker install option for existing vbox users. Hopefully you can avoid the mistake YouWave made of interfering with an existing vbox installs.

Where did you find the file to try it? I only saw the blogpost.
There was an example site that was linked in the blog post: http://peggo.co/

It will only work with OS X 10.7+ though.

Oh ok, that's why it worked for me then. I thought he was making the beta of Arc available already. It looks like it might solve a problem with an in-house project I've been working on at work recently.
(comment deleted)
If I have a VM in the browser can I run.. vim in the browser? My customized instance and all that? Or maybe this wouldn't be too feasible for io-heavy uses?
> If I have a VM in the browser can I run.. vim in the browser? My customized instance and all that? Or maybe this wouldn't be too feasible for io-heavy uses?

vim, emacs, anything that runs on Linux.

How does one try out this beta? Is this going to be open source?
> How does one try out this beta?

Documentation for Arc and arc.js will be available shortly.

> Is this going to be open source?

Arc will not be.

Perhaps Peggo once Arc has cooled.

>> Is this going to be open source?

>Arc will not be

The project is certainly interesting and I respect your right to license it as you will, maybe for commercial reasons, but a closed source black box with relatively low-level access (like Java) makes me uncomfortable.

I may be alone in my aversion to browser plugins, but most of what Arc could be good for would be better solved with actual native software. See Peggo: A GUI app for OSX with just an input box for the YouTube URL and a choice of where to download the MP3 file would be a better solution for that problem. On the devices where Peggo would be useful (mobile phones, for instance), Arc can't be used.

Some things just shouldn't be web apps—I say this as someone who's never made a native app.

>> Is this going to be open source?

> Arc will not be.

Are you using a commercial license for VirtualBox then (and not the default GPL)?

This needs to be answered.
I am surprised by this. Will you expand on why you're not making it open source?
Author probably wants to make money out of it, why else do people lock stuff down? Funny how virtualbox licensing issue got overlooked, because we are all used to download stuff from github with an assumption that any public repository is public domain.
From an end-user POV, what will using an Arc app entail? Will it be like Flash Player and Java; ie, you download and install Arc once, and then all Arc apps will just work and be super awesome?

I had an idea like this, but instead of a VM, using a client-hosted server. The big concern I couldn't solve was security. If you have, say, Peggo.. What is to prevent other websites from being malicious and connecting to your locally-installed Peggo VM and trashing it or otherwise exploiting it?

> From an end-user POV, what will using an Arc app entail?

If Arc is installed, you're good to go. Everything just works. If Arc isn't installed

  1) arc.js transparently falls back to the cloud and runs the Arc app on a
     server. The user doesn't know the difference.
and/or

  2) Upsell the user to install Arc.
I haven't built the transparent cloud fallback yet.

> What is to prevent other websites from being malicious and connecting to your > locally-installed Peggo VM and trashing it or otherwise exploiting it?

The web server running in the Arc app can check the Referer header to verify the request came from a permissible domain.

Can the Referrer header not be spoofed?
Spoofing it in a client's browser is not possible but it's trivial to spoof referrer headers (or anything else) from a stand alone program. Beyond checking for referrer headers the server should give the client a signed token (returned back by the client to the server) to verify the request is valid. Otherwise if the client is arbitrarily sending requests to the server to "install X, run Y, ..." it'd be very easy to hijack the server for other processing.

As usual this goes back to one of the standard rules of server security: Don't trust anything that comes from the client.

Spoofing with a client is easily done. In Firefox you can use TamperData.
Sort of maybe at an intersection of what the NaCL hopes to achieve? Mini sort-of-virtualization of x86?
(comment deleted)
From the post:

"Arc is only packaged for OS X right now. It will be packaged and available for Ubuntu and Windows soon."

Something has to go first.
You might want to change the name from Arc to something else. Arc is the name of a lisp dialect written by Paul Graham, and is the language Hacker News is written in. http://en.wikipedia.org/wiki/Arc_%28programming_language%29
Arc the language hasn't been officially updated in 4 years. All names have been used at some point - it's not realistic to expect them to be truly unique.
> All names have been used at some point

I'm pretty sure the heat death of the universe will occur before all names are used.

Perhaps I should phrase it as "all good names"...
I believe this is technically accurate.
But "Arc" as a name is widely known and used, at least among the HN sort of community, and although it's a niche language it's actively used within that niche. I think you're overstating things; the name "Arc" was not ready for garbage collection.
Isn't this a problem that the market will correct? If Arc the language has a significant enough following, then "Arc" in common use will refer to the language. I'm not saying people should go around picking names others have used, but it seems a little heavy-handed to assume that because a few thousand people have an attachment to a name it is now off limits for everyone else. Moreover, at what point does it become ok? There's also Ark the YC company: http://ark.com/ is that fine because it uses a k?

Taking a look at it from a different angle, even in the case of the legal framework for names - trademark - having Arc the language and Arc the VM thing would be fine.

The truth is, if you don't want people to collide with your name, picking a 3 letter common word probably isn't the way to do it.

It would be lucky for me if you were right, because there's a name I've fallen in love with and want to use for an open-source project that is roughly as semantically distant from an existing project as "Arc the language" is from "Arc the VM thing". But the market isn't the only factor here. There are also cultural norms, which deserve respect because they encode how people ought to treat each other (or in this case, each other's work). So I worry about that.
Yeah, gauging your audience is a different aspect of this problem entirely. If Arc's ultimate audience is intended to be HN that's probably not a very good strategy, but if it isn't, then it doesn't really matter. You're always going to piss someone off if you pick something that isn't very obscure (or made up).

Names are hard. :/

Plus Arc is just one syllable and sounds good. That makes it really tempting.
Arc is an indomitable name. It's short, memorable, recognizable, and pronounceable. It's also a strong prefix: Arc app, Arc box, Arc sync, Arc OS, etc.

Arc passes the highway test with flying colors. Imagine driving down the highway when an 18-wheeler thunders past. If you remember the name stamped on the side of the corrugated shipping crate, it's a good name. 'Arc' in a Neo-grotesque typeface next to an iconic logo on such a shipping crate is as good as burning Arc into your retina with a megawatt laser.

You've evaded the GP's comment, which is about the name "Arc"'s rather obvious existing binding. Why? I'm curious to hear by what process you've decided to ignore that.
At this point the name is so overloaded, I suppose we could criticize most post-'80s things named "Arc" on that grounds. I haven't heard pg/rtm defend why the Arc programming language chose to clobber the existing extension .arc, which is widely used by the ARC archive format. Or the binary 'arc', which has been used for ~30 years as the name for the command-line interface to the industry-standard ArcGIS. There's also a programming language in ArcGIS named the Arc Macro Language.
Those are good points. I suppose the counterargument is that it depends on how much the contexts intersect. It seems to me that in this case the contexts intersect rather a lot. But if I'm wrong, and overloaded project names don't matter, so much the better.
Not to forget Noah's floating palace!

Edit: just realized that was Ark with a K. My bad.

let the living eat the dead
Very nice! Games such as Runescape were built into the browser stored data on users computers to track botting. So many of the first gold farming companies developed technologies like this to implement into Botting clients (which were very sophisticated web browsers).

There was a very interesting tech scene that many don't know about around MMO cheating, especially Runescape.

I was hoping it was a finished version of http://bellard.org/jslinux/
Same expectation here. Deploying a customized VM to the browser (with networking included) will be very interesting.

I am not sure about the real use cases. For learning development will be useful since you don't need to do a more complex interaction between the browser and your server.

Unfortunately, full networking capability as it exists in a typical virtual machine (like Virtualbox), will not be possible in pure javascript. Security features of the browser like the same origin policy restrict this from being possible. Even if you were to use some of the exceptions to the same origin policy, you will be limited to sending HTTP requests. There is no way to send a UDP packet from javascript for example. Of course you can create browser extensions which expose these utilities, but then you're taking the easy way out :)
Would be neat for a project like this to go open source and accept community submissions. Not sure why the author of jsLinux didn't go that route.
Fabrice Bellard is the author of qemu and ffmpeg (among other things), so it is likely he is aware of the benefits of open source and it is a conscious decision (maybe there is interest from someone with relatively deep pockets, or maybe he considers it too hacky?) Don't want to speak for him here though.

I do wonder how qemu compiled via emscripten would compare though...

Fabrice Bellard is the Chuck Norris of all hackers..

its a waste of time trying to figure out what his superior mind could be thinking :)

Just give me one more month, I'm almost there ...
anybody just typed:

$ ls

$ gcc hello.c -o hello

Unbelievable. I tried, echo, ls, find, grep, top, ping, whoami, ps, wget, vi, emacs. Everything is there. All in my browser in Javascript? This is fucking cool. I would love a complete vim in javascript that I can use to replace textareas. This would make it possible to simply use the vim.
The real wtf is installing tcc and running hello-world with a shebang...
Interesting, but dependencies are a long-term killer. You should check this project:

  http://bellard.org/jslinux/
It creates a VM in javascript and can boot Linux.
(P)NaCl already solves this problem, without such hacks as transparently spinning up a virtual machine.

I don't trust Virtualbox to be especially resilient to attacks from malicious VMs. Chrome's sandbox is well-audited and (overall) is sound. A virtual machine host has a much larger attack surface, and generally doesn't assume malicious guests.

Native Client

  1) Is Chrome only.
  2) Can't spawn processes or subprocesses.
  3) Can't open raw UDP or TCP sockets.
  4) Requires apps be ported.
While I like the idea of running a VM in a browser (not sure if I'm convinced it makes sense, I still like the idea):

Argument #1 only makes sense if you support a lot of platforms. Right now you only support Mac OS X (according to another comment).

The number of people who use Chrome globally is larger than the number of people who use Mac OS X. Ergo, if you used Native Client and chrome, you'd be more ubiquitous.

Regarding sockets: chrome supports UDP and TCP: http://developer.chrome.com/apps/socket.html

If there's one platform to build on that is going to cover a large number of people and give close to native performance, it's Chrome+PNaCl. I wouldn't tell everybody to drop what they're doing and adopt that target (it's not ready yet). VMs in the browser are pretty nascent, too.

Why do you think that is? Is it because Google have a large team of engineers who have developed a proper security programme for the project and realised that doing 2 and 3 are bad and that as a result of that and other problems with diong this that 4 is necessary? There are reasons for these limitations.
VM hosts have assumed malicious guests ever since people started renting out VMs. It's a VPS host's nightmare to have a VM root exploit be escalatable to expose all the other VMs on that VM's host.
I trust virtualization as a security boundary, particularly on modern CPUs with stuff like VT-d/VT-x, more than process separation on Unix/Windows. I still trust hardware separation a lot more, particularly because it is so much easier to audit and put multilayer controls on, particularly vs. administrative users.

Virtualbox in specific might be a vulnerability compared to other virtualization systems, though.

Um.. transparently spinning up a VM is not a hack. Its the actual design !

>Chrome's sandbox is well-audited and (overall) is sound.

So why does it fail during the annual Pwn2Own competition? You and I have different definitions of 'sound'.

Also coming back to NaCL.. NaCl's code verifier has never undergone large scale deployment/testing/real world use from millions of users/apps.

Title is wrong. Virtual Machines are not in the browser, they are in your computer bridged to the browser.
browsers are essentially virtual machines themselves, so to be highly redundant (i.e. java already does this), why have VM's in the browser, which is already interpreting code on its own. The next guy is going to come along and make a VM inside of a VM inside of a VM and lets see how slow we can make the browser when its 5 levels deep in abstraction.
This is an actual linux virtual machine... Not a language runtime.

It's too bad that "Virtual Machine" means so many things. Maybe someone has more info on this, but I understand that it was Java who first called their language runtime a "Virtual Machine". Partly motivated because they wanted to create an actual physical machine that ran Java (kind of like Lisp machines). Nowadays with type 1, type 2 hypervisors, and jails/containers/zones, and every programming language on the planet calling their runtime a VM, I'm not surprised that people are getting confused.

"$25,000 every month in hosting costs". Wow, is that right?
As I recall, Dirpy had hundreds of thousands of users. That's a lot of video transcoding.
Dirpy had three million monthly users.
Ah I see, my apologies. I thought it was a beta site, that was't clear to me when I read it.
This is really cool.

It's completely backwards from my personal usage of the Internet; I get away with browsing online using a Penryn-era Pentium, a ARM-based Chromebook, and/or my Galaxy Nexus precisely because the majority of processing costs are offloaded on "traditional" websites rather than on the browser doing the rendering.

But if this means that application developers won't have to recompile every application under the sun (like, say, ffmpeg or Audacity) to run under asm.js or PNaCl, then I think it could mean that we could skip a decade or two of having to reinvent the wheel.

On the other hand, this feels suspiciously reminiscent of ActiveX, so I suspect you're going to have a hard time convincing people to adopt it if the security diehards warn you of running arbitrary code on your machine (even if it is in a sandbox).

"the majority of processing costs are offloaded on "traditional" websites rather than on the browser doing the rendering"

I have not found this to be the case. I find that most websites take a LOT of processing power to display - loaded with flash, scripts, video, etc.

A lot of sites are not really guilty as it is the ad network content inline with the site that pulls all of that computing power, but other sites (boingboing, for instance) generate a lot of CPU use just on their own.

And it gets worse all the time. I suspect that whatever gains we make with efficiency of HTML5, etc., will be immediately consumed by things like the OP is building.

I have a 5 year old macbook air that absolutely does not need to be replaced. Except that I can't have more than 10-12 browser windows open before it's pinwheel city...

With the devices I've used, I find that flash content loads and runs just fine provided that there's at most one running on each page and it doesn't crash.

Granted, I find myself enabling Adblock by default on most sites because most ads nowadays are annoying Flash pop-overs. Back in the day, the Linux implementation of Flash didn't support making the Flash embed transparent, so I had to go into Firebug/Web Inspector and delete the embed/object tags entirely just to read the page. While I think that particular bug has been fixed, even today, most of the Chrome tab crashes I run into are still caused by Flash crashing.

It really bothers me how much stuff depends on Flash still, whether it's putting something in the clipboard from the browser, or just Google Hangouts or Facebook deciding to play a "ping!" when a notification goes off.

The only thing "HTML5" means to me is that my ARM devices can offload H.264 video decoding to the GPU, rather than trying to run a cross-compiled Sorenson decoder on the (relatively underpowered) CPU. Everything else under the "HTML5" banner seems to be just increasingly complicated browser-specific extensions to JavaScript and/or CSS.

I've also noticed that the Chromebook and Chrome for Android will deallocate tabs that I haven't used recently and reload them when I switch to them, which lets me have dozens of tabs "open" on devices which are otherwise only capable of handling three or four tabs at once. Of course, this is only reasonable because of "high speed internet"; I remember being on dial-up in the late 90s running Internet Explorer 4 and opening ten windows in the background so that the pages would pre-load in the background while I read the current page.

It's certainly a great project, but i can't help thinking : so now we're not satisfied anymore with running virtual machines to execute some code, we need the whole OS on top, along with the shell, and the preinstalled programs...

That story reminds me of the last time I tried to compile a blackberry app on my mac a few years ago : Java VM running my code within a blackberry simulator running inside of a windows XP VM on top on my mac OS.

Where will it end ?

People prefer this incremental series of hacks to the existential despair of being faced with a cohesive system, say a lisp machine. :-)
Should have named it red pill. :)

Seriously though, I was more excited when I thought it was about running Linux inside the browser. Personally, I have no interest in installing an extra VM on my system just to make your development easier.

This sounds a lot like reinventing the wheel (JVM). JVMs lack the power of a full linux core, but do you really need it? you just wanted hardware access + native threds (JVM has these and more)

Also this sounds good for just 1 App, but what happens when you try to run more VMs than you have physical cores? And/Or memory, this would directly affect the host.

> what happens when you try to run more VMs than you have physical cores?

The VMs are just processes of the host OS. They're multiplexed over available cores, same as ordinary processes.

I might be off base here but because the processing of data happens on client side is it possible that this could be used to increase privacy through zero knowledge apps?
(comment deleted)
The installation failed.

The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.

10.8.4

Please shoot me an email so I can destroy the bug.
Security wise, this seems like an awful idea: Unless the host is firewalled from the guest, if Arc-style VMs become popular, than you'll have malicious websites starting VMs to scan your host network for unpatched vulnerabilities, and abuse them.

I try to keep my network secure, but e.g. Cisco/Linksys E3000 hasn't received a firmware update in a long time, and it has known exploitable bugs - right now, the fact that it is only accessible from inside the NAT, and that webpages can't do arbitrary accesses is what keeps all those E3000s from being exploited.

(My E3000 has been running dd-wrt, so it's not vulnerable to those problems; but I had to manually upgrade the dropbear ssh because of vulnerabilities - latest official dd-wrt for it is still vulnerable)

Security-wise it is no different than installing a multitude of software packages onto your computer, which many developers are already comfortable with (whether they should be or not).
Perhaps - But Dripy and Peggo are not targeted at developers.
The people who use Dripy & Peggo are also the people download spyware infected "Youtube Downloader" apps already.

I don't see a lot of difference - for good or for bad.

Arc apps will require explicit permission to communicate with a local network. This can be enforced at the hardware layer by the virtual NIC.
How do you define local?

Is it 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16? I guess that would cover 99% of local networks.

I wasn't aware virtualbox has firewalling at the virtual NIC level - my 4.1 doesn't; It's either host-only, bridged, or nat - of which bridged is unlimited, host-only is useless, and nat cannot (as far as I can tell) be firewalled at the virtual NIC level. So how do you do it?

Even better would be to require explicit permission to communicate with any site that isn't the app origin.

A VM can be put on its own VLAN with a traffic routed through a secure firewall. I don't think Arc is as doomed to be insecure as many are claiming.

VirtualBox, at least the version I run, cannot do that on its own. You would need to set up the firewall rules on the host. Which is, of course, possible - but not in a cross platform way (linux uses netfilter/iptables, bsd uses pf, windows uses ... I'm not sure what these days, but many users have a 3rd party firewall as well)

It isn't doomed to be insecure, but its security, portability and convenience/usability have a nontrivial tradeoff which is ignored by the original description. If it's portable and convenient, it is likely going to be lacking on the security front.

Which Dropbear vulnerability? The last one I can see is from 2012.
That's the one. The latest stable dd-wrt is from 2009 - if you're running dd-wrt and haven't updated dropbear, you're vulnerable.

The original Cisco firmware for the E3000 also has vulnerabilities, which were never fixed.

Neat concept, but I don't see a VirtualBox-based implementation ever becoming mainstream... not that it needs to be mainstream to be useful.

It seems like full blown x86 PC hardware emulation is overkill for what you're doing. As others mentioned, NaCL isn't really the right abstraction layer either.

Perhaps a stripped-down version of VirtualBox could be turned into a "standard" browser plug-in and paired with something like Docker, so you're just running a minimal container image.

I like the Docker idea, especially if Arc were providing the base linux VM via read-only mount and the website's Arc App was only a difference image on top of that ... nice and small.