Ask HN: How Do Services Like PythonTutor and Repl.it Run Code Safely
I'm super curious to know how these services work under the hood. Are there any good writeups about safe execution of unknown code? Are they spinning up isolated VM's per user / connection?
7 comments
[ 3.1 ms ] story [ 26.6 ms ] threadhttps://nodejs.org/api/repl.html
Generally you have a client run their own Javascript that speaks to your backend but does not run on your hardware nor within your virtual ecosystem.
See https://techcrunch.com/2018/03/15/repl-it-lets-you-program-i...
Basically, most of them build a sandbox to prevent the guest system from interacting in unexpected ways. There’s a bunch of ways you could do this — intercepting syscall, limiting them using seccmp(2), etc.
Anyway, it's currently docker based, basically the only interesting thing is the run endpoint which is a POST request that copies the payload files to a tmpdir on disk (cached files are just represented as a path).
With the files available I spin up the language appropriate container with the tmpdir mounted as the working directory and the default run command (which can be overriden to specify other build options etc but there's a reasonable default built into every docker image).
The output is streamed as plain-text so it works reasonably well even with no JavaScript but SSE is used when JavaScript is enabled, look a bit prettier. WebSockets are just not implemented yet.
Finally, I found it annoying not being able to preview servers etc on other playgrounds so TCP connections are proxied, each time a subdomain is accessed it looks up the port and maps it accordingly to the container.
There's a bit of overhead to all this tho, from hitting the run button until seeing output can take anywhere from 10 to 60 seconds.
As for safe execution, people have broken out of docker containers before and no doubt people will do it in the future. Best you can do is minimize the damage breaking out of the container would be able to do by isolating the machines that run the containers from the rest of the stack.
At Repl.it we have multiple layers of security with the most important being that the entire execution environment has access to zero user data. Even if someone escaped the container jail, and the VM jail, and compromised the server they really couldn't do much beyond that.
We also use and built a bunch of intrusion detection software and manually look through potential threats.
Finally, if you want a container-based sandbox take a look at GVisor by Google. It's still an early project but has a lot of potential.