Ask HN: Is anyone using cloud dev environments (e.g. Codespaces/Replit) at work?

217 points by nbrad ↗ HN
A lot of devs I know are excited about Cloud Development Environments (CDEs) like GitHub Codespaces, Gitpod, Codeanywhere, Coder, Replit, and CodeSandbox. They seem great, and simplify many aspects of the dev workflow: easy to onboard onto new projects, everyone on the project stays in sync, etc.

But I rarely hear of actual teams using them; it's usually individuals using CDEs for side projects.

Are you using a CDE at work? Would love to hear about your experience.

245 comments

[ 3.5 ms ] story [ 274 ms ] thread
None of those, but on my last contract, the client used D.O. + Docker envs for "local" dev. Personally, I loved it. I didn't have about to worry about getting a project to work on my on-desk hardware. I simply opened VSC, VSC+SSH'ed in, and went to work. It simplified collaboration, etc.

By far, what I appreciated the most was I didn't have unnecessary data on my local hardware. No customer data. No order data. Nuttin'.

Most of the CS classes at my university have moved on to an online Jupyter environment with VS Code preinstalled. It lets students spawn an environment with all the required software for their class preinstalled.

E.g. For Computer System it spawns an environment with GDB pre-installed, while for Intro to AI it has PyTorch/Python3 pre-installed

Do you know how that works exactly or do you have a link where I can educate myself on the topic of using Python online in a sort-of-IDE? I‘m teaching Python fundamentals and don’t know how to sail around helping everyone with their individual problems with their machines while installing python or an ide.
I'll bet they're using this, that's exactly what it is designed for: https://tljh.jupyter.org/en/latest/

Alternatively, you could do something similar with Google Colab and a notebook you make to serve as a template.

(comment deleted)
Having more grads coming out of university unprepared and expecting even more handholding (this time in the form of devcontainers in the cloud preconfigured and "ready", etc) really does not bode well for the health of our industry long term.
Computer science is not exactly related to googling cryptic error messages, but IMO every undergrad CS major should have a tools/linux basics class of some kind
The situation is pretty bad. Some students don’t know how to Google properly because they know they can always reach out to TAs/CAs during office hours

But at the same time if everyone moves to devcontainers, will this skill even matter for the most part? Because the people who want to learn how to do something are going to do it regardless, and those who don’t might not need to worry about it

Yes but seems like most companies are moving to this remote dev container environment that you ssh into. Obviously the google skills are still important but there are plenty of other, more interesting, problems you can use to practice that.
I don't think that's even remotely true, I think you're in a bubble.
How is that different from using shared Unix boxes from back in the day? Using Containers doesn't mean not having access to the file system or console. You basically have the exact same experience as using vscode locally.
Sounds like normal progress. The bulk of the next generation will operate on a higher level abstraction that affords them more productivity with some of them specializing in the lower level stuff that used to be part and parcel of normal SWE duties but is now basement-dweller neckbeard voodoo. Grandpa complained that nobody knows how to set points ignition and jet the carburetor anymore, dad complains that you can't fix anything anymore, we will complain that kids these days never leave the pod and live in VR.
I lived on a boat with a 1972 Atomic 4. I had to set the points and jet the carb every so often. I truly enjoyed working on that thing. My car needed a freaking mechanic with a computer. It was nice to hear it start chugging incorrectly, pop the cover off, measure a few things, then start it back up. No technology required, just well timed explosions.
Online REPLs are great, but students especially in CS should know how to set up their environment.

I remember being a Junior Compsci student watching other students struggle to install the Java JDK on their Windows laptops (install JDK not JRE; configure PATH; install Eclipse; point it at the JDK). The professor had to come around and help them. I was the only student with a Mac and I had transferred from a community college where we had already learned Java in the first semester, including how to install and set up a local environment. All of this to say I feel like colleges should better prepare students to set up their local environments, although I see how useful the online REPL options are for getting started.

I went through a similar struggle in ~2005, seeing the word Tomcat still gives me PTSD from being left to ourselves to install Tomcat server on Windows XP.
This is one of these things that I'm shocked there isn't more widepsread uptake on. It seems to win on some of the more important dimensions: ease of use/UX (if executed well), power of machines -> faster iteration time for many workflows, improved priacy. I am not familiar with pricing for many of these offerings and I suspect it probably is pretty expensive.

Even at megacorps where they have really good cloud dev environments, adoption is not universal. Many many people at my current employer have big under-desk workstations to do their day-to-day programming.

I would challenge if the size of the machine or responsiveness of the cloud is superior to local. Most people are not compiling Rust all day, but refreshing JavaScript. Which is to not even bring up the cost difference to rent or own a beefy piece of hardware. 64 gigs of ram is nothing for a local machine, but comes at a cloud premium.

I do think there are some real wins in ensuring development environments are consistent and versioned. Knowing you can pick up the exact version used to develop the project without dedicated effort is attractive.

> 64 gigs of ram is nothing for a local machine

It's not nothing. Most developers are issued laptops, and a 64GB Macbook costs over $3000. Plus a CDE can be shared.

Then don't buy a macbook, adding ram to a laptop isn't that expensive. You can get a 64 GB stick of DDR5 for $150.
> Most developers are issued laptops, and a 64GB Macbook costs over $3000.

This is unfortunatley a nonsensical argument.

As a company, you are employing a developer to write code. Both the company and the developer are happy when they are at their most productive. And to that extent, a $3,000 laptop is a perfectly reasonable "tool of the trade". Also don't forget companies can lease laptops if they don't have the cash floating around.

Its the same thing in other sectors.

For example, in Finance. A Bloomberg terminal comes in at $30,000 a year. But if you are, for example, a bond trader putting in multiple 6,7 or 8 figure orders on the market every day, then its no secret that Bloomberg has the monopoly on bond data and as a bond trader you absolutely need access to one as the tool of your trade.

In addition, following COVID, most sensible companies have embraced WFH. And so what the company saves on office space they can spend on other things such as better laptops for developers.

Developers do not typically get to choose.
you mean I could transmit my commands at the super fast speed of my internet connection compared to nanoseconds? SIGN ME UP!
I know you joke, but I've been in situations where my local environment was so slow that I'd rather program in vim at 200ms latency, because that still would have been better for my sanity.
The power of machines is less than a 2021+ macbook unless you really pay up for your cloud workspace. Then it's worth the cost if you put the money there instead of towards a good laptop, but people still need some kind of computer.

I get that spinning up a new dev env is really handy, but with something like Ansible it's possible to get a good dev environment on someone's local machine within hours, not much worse.

What am I missing?

It's a mix of many different things.

At work we have very beefy workstations that just aren't available in those hardware specs from cloud vendors. They use workstation grade hardware (A6000 GPU, threadrippers) that offer better performance/price than similarly sized cloud offerings which use datacenter type hardware. To actually tick all same boxes you'd need to go for much larger instance types. Plus it would be questionable whether one could realize the scaling benefit of the cloud because those machines are more pets than cattle, killing them over night and bringing up a clean system the next day would likely upset dozens of different workflows and tweaked setups. And then the software expects network filesystems with low latency random access, not blob storage... They did the cost calculations, buying hardware is cheaper in the long run, including ops.

And then there are customers that have extremely stringent security demands. Segregated networks, video surveillance, no data must leave the premises. Convincing them to move the data into the cloud might not be impossible but it would be a big recertification ordeal for one cloud vendor. And then we'd be locked into that vendor until we could get something else. And we have several such customers.

We've started integrating Codespaces into our team's workflow. It's been a game-changer for onboarding new devs. No more "works on my machine" issues. The ability to jump into any project without the setup hassle is pretty sweet. We're still ironing out some kinks, but overall, I'm pretty bullish on it for professional use.
isnt that what containers are for?
The file defining the environment is literally called devcontainer.json, so yeah, but it’s not really about that, it’s the service aspect which is the value proposition
Nice, thanks! what kind of team are you on (company size, how experienced the engineers, tech stack)? when you say "ironing out some kinks" what were the pain points?

if you're up for quick chat/DM, would love to hear more (link in profile)

I am sorta doing this, locally. I have a handful of servers in my basement that I use vscode+remote-ssh to work with. The benefit is that I can jump from my desktop to my laptop without any fuss. I can leave long-running scripts online for like an experimental batch job and attach to that from any machine, even when I am not home thanks to wireguard/tailscale. It is very nice knowing my dev env is always there in the state I last left it. This also gives the nice benefit of being able to choose your developer env (windows or mac) while running your dev/code/system on a linux system for example. It also gets you around architecture issues, like if your laptop is arm-based and your app needs x86.

I was initially kinda opposed to this and preferred "bare metal development" on my local rig. But the performance is actually pretty incredible for remote development with vscode such that I don't really notice things are running on different hardware.

We recently had some new members join the team and I decided to spin up some dedicated EC2 instances for them to use for this exact purpose. They aren't being used yet, but as our stack becomes more sophisticated I think workloads will transition there. It's done with a custom terraform module that also provisions other assets needed for each dev (regardless of local or remote dev) like an S3 bucket, some dynamo tables, IAM roles, etc. Being able to onboard a new dev with a handful of lines added to a mapping is pretty awesome.

tl;dr I would absolutely consider remote dev spaces.

I use private replit(s) sometimes to work on small snippets of code. Then I use my code snippet as a reference when working on our actual codebase.
I rent server rack space at a datacenter where I have a beefy workstation to do all my work. I use VS Code Remote and SSH port forwarding. The workstation has fast 2TB NVMe and 64gb of ram, 16 cores and a RTX GPU. It's actually cheaper and faster than our production cloud server.

I tried Github Codespaces and thought it was cool but wasn't nearly as fast as my remote workstation.

What's the advantage of having the workstation at a datacenter in this context?
$60/yr vs thousands of $ upfront for a comparable desktop or notebook machine.

Not to mention the superior network connectivity (10 gigabit/s). In fact, there are no 10gbps options from the common last mile carriers available at any price (AT&T, Crapcast). And if they did offer it, it'd probably be $500USD/mo+ just for Internet.

AT&T has a 5gbps fiber plan available for $250USD+/mo, but there are very few (>10 fractional portions of cities nationally) areas where it is offered today.

If you have a strong enough desire for cost-effective, really fucking fast Internet for a fair price, move to Korea. In .kr you can get 10gig for something like $30-50USD/mo.

Edit: Oops, my bad. I meant to reply to this comment:

https://news.ycombinator.com/item?id=37934982.

> Step 1. Create an Oracle Cloud account Step 2. Create an Ampere 6 core, 32gb memory instance for like $5/mo Step 3. Use Jetbrains Gateway to run your IDE as a thin client, executing on that host.

> You get a pretty darn beefy ARM64 VM instance from OCI for extremely cheap. You can get these in a region near you, with low latency. And Jetbrains Gateway works pretty great.

We use OCI, aka Oracle Cloud. As a customer, OCI is more financially appealing compared to the competition (AWS, Azure) for bare compute infra. Not as cheap as OVH, if that fits your requirements. For me though, OCI is preferred because it provides all the standard cloud building blocks of multi-region + (cheap) object storage + block storage.

I mean, fuck Oracle and Larry Ellison, right? That said, building and operating a cloud at scale is a lot of work and I'll leverage their work as long as it makes fiscal sense.

The commenter posted about a self-installed machine. Also, colocating GPU servers is expensive, because it's billed at roughly $75 per amp per month, and a typical machine with a single 280W GPU will use about 5 amps at peak.
That sounds really low. Do you mind sharing where you rent your racks from?
I was thinking the same. I'm paying about $1600/mo for a 42U rack, 2 x 30A Feeds, and 60mb/s 95th percentile, ipv4 + ipv6
> The workstation has fast 2TB NVMe and 64gb of ram, 16 cores and a RTX GPU.

> $60/yr vs thousands of $ upfront for a comparable desktop or notebook machine.

I don't think we're talking about the same things here.

I paid $5000 for the workstation (2019) and upgraded to NVMe later, and I pay $0 for the datacenter because I have referred clients to their business and established a relationship over the years. Because when people find out how much cheaper it is to skip the cloud, they switch but don't know where to go.

I could upgrade to dual 2.5 GigE but I have nothing that needs that much bandwidth. I can download 30gb from hugging face in a few minutes which doesn't bother me.

The DC is about to onboard another client of mine, a startup that ran on AWS credits for 9 months. They are upgrading to a 64 core 2TB ram box that was an extra the DC had, for $3500, which will probably go into the rack next to mine.

Where are you seeing these prices? OVH seems way more than what you’re saying, but might be looking at the wrong thing
Intellij Ultimate edition costs more than the server at this point. Wish they'd bring this over to Community and Android Studio.
nice - I've also found VS Code Remote + SSH to be pretty fantastic
Beefy HEDT/workstation with ssh running are amazing dev environments. Vscode remote ssh or even vim make it a dream.
Too bad Intellij requires the Ultimate version of their software to use their equivalent of the VScode server/client feature.
Don’t worry about the price. It’s absolute crap. [At least PyCharm remote is and I believe it has the same underlying Jetbrains architecture]

Windows randomly quit, take forever to open, there are always indexing issues, etc. Locally everything is great, but remote is completely unstable.

I think it has you install something on the server if you want to, and it's more reliable. But yeah, completely in different league from vscode.
How do you find datacenters which will let you colocate hardware? How much hardware do you have? What was the cost?

I'm really interested in this kind of thing.

Can't answer the latter questions as I'm not OP, but you could start finding a colo here if you're looking at the cheaper end of things:

https://www.webhostingtalk.com/forumdisplay.php?f=131

Also just googling "[major city name] colocation" will usually find results, though be warned pricing is often "call us" vs. stated upfront.

Almost the same for me. The data center is a couple miles from my home and my home has fiber. It’s low latency, I almost never notice a difference from my device. Maybe if I used it for really large files but that’s not my thing. It’s nice keeping my local device crud free and low demand.

Edit to add; cost me about $70/month depending on a few variables on collocation for a 4U. My same apps on AWS would cost 10x more. So that’s plenty of budget for my hardware that I usually get a year or two out of; doing minor repairs/upgrades myself.

Where are you colocating? Looking for one myself
I’d recommend you approach it from a hyperlocal perspective, find the local data center you think will be best for you then look up who is offering colo out of that data center. Then do some network testing. Maybe compare a couple data centers. The providers are going to vary in most cases. So I don’t recommend looking for a specific provider just because it works for me or someone else. You’ll get to pick your bandwidth provider too in many cases.

I like to make sure the network hop path to my home is as direct as possible. My ISP has a large presence in one of the local “backbone” data centers so that’s where I find the best results. But still do testing, it’s not all about location when it comes to latency. If you visit the data centers, or online, they often list names of colo providers. They’re often rather small shops and not large brands so going to be very location specific

For Jetbrains users:

Step 1. Create an Oracle Cloud account Step 2. Create an Ampere 6 core, 32gb memory instance for like $5/mo Step 3. Use Jetbrains Gateway to run your IDE as a thin client, executing on that host.

You get a pretty darn beefy ARM64 VM instance from OCI for extremely cheap. You can get these in a region near you, with low latency. And Jetbrains Gateway works pretty great.

On the plus side, this is an entire VM, so if you've got containers, or whatever else you need to run, that all executes there too.

Jetbrains Gateway is very poor compared to VS Code's remote development plugin. I can't use it without wanting to pull my hair out. It's laggy, installing plugins is painful (per project?!), etc..

Has it improved in the last year?

not really

i so desperately want it to work, but it messes up often enough that it's not worth it yet

fwiw, using local intellij with nfs mounted source is a better experience

Yes, it's improved remarkably over the past year. I had the same experience as you, but it's at the point where it's pretty usable.

There are still irritations. But I am comfortable using it day-to-day.

It's absolutely critical that your remote gateway be nearby. I'm about 10ms away from mine, and though there is sometimes perceptible lag, it's not bad at all.

It's still painful. Maybe it's improved, but my day-in/day-out with it is horrible.
Sounds close to my experience with Rider the other week. Had a fantastic experience migrating my dev environment to WSL+VSCode from VirtualBox. Went to try Rider pointed at the WSL instance and.. Basically gave up on it lol.
Do you mean $50/month? Or maybe $5/day? I’m reading:

> Ampere A1 compute… with cores billed at $0.01 per OCPU-hour and memory billed at $0.0015 per GB-hour in all regions.

So for 6 cores and 32gb memory I’m calculating $78.84 per month.

I’d love to get 32gb of ram for $5/month.. but it sounds too good to be true.

Looks like 4 cores and 24 GB fit inside the free tier: https://www.oracle.com/cloud/free/
It is funny they call it "Always Free".
Either they are aping from Google Cloud or Thomas Kurian brought some Oracle phrases with him to Google, since GCP also calls it that (or at least they used to).
Thank you, Wow that’s a generous free tier!
I have not used a full blown online environment. Except maybe VSCode remote using SSH. I repeatedly find anything that requires a network call somewhere in between a serious impediment disrupting the flow of development. Sometimes I find myself in slow laggy situations with ssh to the point I prefer Mobile Shell mosh. VSCode remote (or similar) via ssh obviously becomes painful.

Most cloud environments are also limited in terms of what you can do. e.g: issue sudo while running a process, attach to a process with a debugger.

Usually when these come development environment ready, it also hides away underlying details - i.e, I no longer know the command line etc to should I need to write infrastructure code/automation later on.

I guess there are domains where these are non-issues. But for a wide variety of my use-cases local development is going to be preferable, because by design there are limitations in the alternative.

I considered it but the cost seemed pretty steep - up to ~$30/day/dev for codespaces. I didn't think local development was close to broken enough to justify it.
I got a cheap Hetzner Auction server for all the work needs https://www.hetzner.com/sb . I use VS Code remote to work there. It's easier and more convenient than Codespaces (and I think cheaper). Has a lot of benefits like instantly sharing a web server with the outside world, ability to run my servers and bots continuously, ability to work from different computers seamlessly. It also functions in many random ways (e.g. as my Anki backup server)
Is this how you work across the company?
no, but I had one employer reimbursing it (changed job since then). they were quite interested in this and might have actually implemented this for others, not sure
We use remote dev VMs with VS Code connected via SSH. It works a treat for a small team working on microservices. You can work from any machine while everyone has their own user account. Sharing code is a breeze. It's easy to test APIs in development (no http tunneling). Deploying to a local (on the VM) docker host for longer running services in test works well and it's super cheap to run.
Cool! I've used a similar workflow with teams that SSH into boxes with GPUs, VSCode makes it completely seamless - surprised i don't hear about it more
Would not using one of the selfhosted CDE solution make it even easier?
Kind of, I used to run code server but you miss out on plug-ins and other custom settings. Connecting via SSH is seamless. You can copy from local -> remote, keep your plug-ins and access from any machine with VS Code.
What if it was as seamless but also managed everything for you. Can send you a link to try it out if you like?
It is interesting that in the comments on this thread I’m not seeing any mention of nix, which is arguably overlapping the topic at hand with the Venn diagram of “spinning up dev environments”.
Love NixOS. Flakes are awesome and I've started playing with flake-parts+devenv. It's been far easier and faster to set up isolated environments(nothing installed globally) than docker compose, that and it's substantially more performant (especially on non-Linux OSs). I've also found of the container route that you usually still need to set yourself up locally for most of the tooling or have a fairly complicated set of images for different use cases and/or set it up with the complexity of a full OS anyway.

Started a new job recently and had my laptop up and running, ready to code in about an hour (only second nix box I've brought up), by day 3 I was building and running the main monolith monorepo with my own local flake. I have since replaced redis, postgres, and two ancillary services in containers with devenv services/processes; it's been really great not dealing with docker volumes, networks, images or building containers and managing pruning them.

It would be interesting to play with automating deployment of my nixos machine configuration into a cloud VM or pod as I work 99% CLI anyway, but I just don't really see the need... this is just easier.

I've heard some reports that Nix is very painful to get working with Python/ML stack - do you know if this is this still (ever?) the case?
Replit to build very quick prototypes of some script and see what it would console log.

GitHub Codespaces for quick stuff, I prefer my IDE.

I use stuff like replit, online sandboxes, jsfiddle, just to execute snippets of code.

I'm aware of IDX though

I've been doing this personally and also pushing for adopting this at work. We are currently moving to monorepo though so we will probably look into this better after we're done with that.
There are a few gotchas, but I've introduced them at both Vanta and Q Bio and have been thrilled with the productivity increase at both places. As long as you stay on top of keeping prebuilds green, you know that everything just works for all of your engineers.
No because I want the entire development environment to run locally (and anywhere else) without WAN connectivity.
You could still totally lock it down and deny WAN access but holepunch just for your vpn of choice.
Doesn’t help when you truly don’t have WAN connectivity. Imagine working on an airplane where the wifi is unusable or unavailable, or working during a long NYC subway ride (where even mobile data connectivity between stations is only occasionally and transiently available in the underground segments of the system).
why would you be working while sitting on some type of conveyance?

doing things like that will not get you any rewards which compare to the effort you've spent. look out a window, I say.

Let’s say I take a train 1 hour each way every day - why not do some work?
because you're not a slave. you don't owe your employer your travel time if you aren't explicitly paid and reimbursed for travel time and costs.

that's your time. don't give it away to your employer for free. get paid for it if you are going to do work while you commute.

Over the past decade I'd reckon 1/4 of my productive output has been whilst sitting on some type of conveyance. Some of the intervals in that period have stretched from 6mo to 1.5 year where >50% is while commuting or otherwise traveling.

Honestly, I'm either on the clock or off, and I have better things to do with my time than look out a window when I'm not being paid.

I genuinely envy your ability to do this. I find that it's simply impossible for me to engage in any meaningful work during commutes. I can't focus.
I used to do code reviews and catch up on email on my phone while on the train. It let me leave the office sooner. The metro runs underground, so there's nothing out the window to see.
It's more the principal so that I can spin up tests anywhere and actually understand the full set of deps.
I had this gut reaction initially, but I can't recall it ever being an issue since we moved to gitpod around 2020. Our dev experience is inherently coupled with network access. It's not as though we keep local caches of package registries, etc.
I have a Macbook Air and vscode-remote over ssh to do anything.
Frankly I dont get the appeal. And I've also used these in a really polished env (a company rhyming with schmookle) where decades of r&d had gone into building this. Where these are useful (though a pain to use) is if you have to work on a system whose components that cannot be built individually. Ie you need a production grade spanner, and env with all of the 15 microservice dependencies with their 15 dependencies each with their 15 dependencies each and so on.

At this point id question if I want to be this tiny piece of a tiny cog in a tiny gear among a billion gears in the huge clunky machine (comp and bottom layer of Maslow hierarchy aside).

I don't really see the appeal. We don't even use the same dev environments at work, some use vscode, others use pycharm or codeblocks or whatever. We have code locally, on github, on gitlab and in other places.

Also I can't really say that I've heard anyone outside of this thread who is excited about this at all. I see people here have different ideas but personally I've never heard of it from anyone I work with or know directly.

(comment deleted)
To me it feels like one of things if executed really well would be amazing, but the first thing that comes to mind when I unwrap it is all of the things that could make it a bad experience.

I can also concur that I work in an area a that is doing somewhat bleeding edge infra work (probably second stage adopters) and none of my colleagues or myself is actively seeking out this technology. We don’t really seem to have the problems it purports to solve in a major way at this time.

I'd say it works well enough for maybe 90% of devs at my company. It solves a lot of real issues for them and dramatically simplifies onboarding.

The issue is that most of my team are in that 10% and now that 90% of the company have their needs met, we're getting pressure to conform because maintaining all the physical infrastructure for a small group gets expensive. Unfortunately, we can't use cloud environments for a reasonable chunk of our work, so that really means we're get forgotten about until we complain loudly enough, no matter how friendly I am with the relevant teams.

The issues you listed are non-existent with Gitpod. You can point whatever editor at it, and work on code residing anywhere.
I've used Github codespaces at work when I wanna make small modification to an open source repo and don't wanna go through the entire process of cloning, building things, etc. It's prety useful.

We have our own custom built cloud dev envs where I work and I def see the value. I don't need to worry about a conflicting version of a dependency that I have installed locally for some prototyping affecting my day-day productivity.

I used Cloud9 for about nine months before Amazon acquired the company.

I loved it-- I loved having separate environments per-project. I enjoyed the collaborative features as well (send a link to look at code or preview something, etc). I see a lot of potential with them and I would love for them to be more mainstream.

After Amazon acquired the company I cancelled my subscription (I was paying annually.. I think it was $190??). I knew Amazon was going to murder the service, require an AWS login and who knows what.

I have tried others since then like code spaces and some open source/self hostable solutions (I have even tried the old self-hostable Cloud9 code).

Ultimately, I gave up on it... why? I didn't like the idea of self hosting (more attack surface area, etc). I didn't like any companies offering the service.

What didn't you like about any of the other companies?
First and foremost, the editor/systems were not open source. Second, the companies are known for requiring certain kinds of logins and locking things down (Google/Microsoft/AWS account, etc).