Unpopular opinion with a lot of my peers, but "fragile local dev environment" is usually a sign of an unwieldy, tightly coupled codebase with a lot of parts, scripts, etc. Trying to hide that complexity with docker or codespaces, etc is just a bandaid in many cases vs dealing with the root issues.
This has not been my experience. First, I work on lots of code bases and this is where a lot of the brittleness comes from. They don’t use the same versions of the same things. Second, C dependencies are a giant pain in the ass when you add the first problem above.
You still have to grab the dependencies before running make, some of which you'll grab from apt/yum/chocolatey/macports, some you'll need to compile manually and put in a platform-dependent location, ...
Then you can run `make world`. Of course, BSD make is not completely compatible with GNU make...
In my experience, even with simple flask projects, I've found just the difference between Ubuntu and Arch to be irritating:
1. Ubuntu seems to have trouble with psycopg2 while Arch does not, so I often use psycopg2-bin just so Ubuntu developers can easily install the requirements (learned the hard way).
2. Some older (supported) editions of Ubuntu create extra lines in `pip freeze`, including `pkg-resource=0.0.0` which throws an error when installing on nearly any machine (including cloud versions of Ubuntu in Github Actions.
I have an application that uses docker-compose and has Elasticsearch as an image. In order to run it, I have to expand the vm_max_map (or something, I forget). But there are different ways of doing this in Windows vs Linux.
Differences between environments are legion. Unless a company wants to just send standardized laptops to every developer, it seems way better to just spin up a cloud vm that is already set. Plus, there's easier reproducibility in a cloud vm vs a physical machine (unless you want to reach for something like nixos, but hell, what company is equipped for that yet?
The difference between Debian and Ubuntu is enough to be a nuisance.
The Dell XPS laptop my employer provided cannot boot Debian and I'm stuck with
Ubuntu. Same packages, maybe a version or two are more recent right? Nope.
Aside from the egregious transparent snap installs when using apt (which can't
uninstall snaps, thanks Canonical), there's a lot of minute details that amount
to quite a time loss (that and having a laptop with no USB/RJ45/HDMI ports).
Simple scripts that work on one OS and not the other, font scaling and
rendering making my standard 1080p screen unreadable, absurd default
configuration that makes WiFi unusable, etc.
Glad I don't have to manage the same oddities for the actual work I do.
Everything is simple once it's all Dockerized.
But that's not true either. I had a different problem setting up the repository
with every developer and they were all using the same OS.
No repo in sources.list (how?), no docker group created when re/installing the
docker package (why?), all kind of weird stuff happening.
So you tell me there's a way to have all devs working and testing on the same
environment? That's great!
But it is _not_ worth handing over everything to Microsoft or any other third
party.
I'll never trust GitHub, GitLab, or any other forge with business-critial stuff.
Host your own code, commit your dependencies, have your build script work
locally.
I've deployed through Git{Hub,Lab} outages and the leftpad fiasco, and the cost
was insignificant. Host a mirror there for cheap if you want, that's what I do
for my public projects (but after the copilot debacle how can I trust GitHub
with anything?).
And what if you're a remote team? You can't rely on the internet. My ISP was
kind enough to remind me of this when I got my very first outage with them one
hour before starting at my current company.
As valuable a product GitHub is and Coderspace may be, it is a step closer to
the Minitel 2.0 where everything is centralized and controlled by a single
entity. It's not the internet, it's MSN at best.
I'm totally with you here. Like, ideally you should be able to clone, install packages, and then do ``npm run start'' or ``python manage.py runserver'' or whatever, and the result should be a mostly functioning web app running locally. If it needs certain attached services, they should be 1) as standard/boring as possible (eg SQL, so you can have mocked versions of them for tests), 2) and added sparingly in only well-motivated situations (eg do you really need to have that caching DB, before you even have profiled real traffic?)
IMO investing time to keep applications "normal" like this pays off in the long run. It keeps development closer to "greenfield", which causes a general multiplying effect. Tutorials are slightly more likely to work, 3rd party packages will be quicker to integrate, etc etc.
(Note that this is separate from dev/prod parity, which is also good.)
It depends on how exactly the tools are used. If you're using docker for reproducible builds with locked-in/platform specific versions of dependencies that need to work seamlessly across platforms then that's pretty good. If you NEED docker/codespaces/etc.. because in order to do basic development and run tests you need N servers running with some specific state, duct taped with non-idempotent bash scripts then this type of stuff is just piling on tech debt.
I think a lot of times the reason local dev environments are fragile isn't because they're local, it's because they are an afterthought and not a lot of effort goes into making them good and stable:
(1) On a team of developers, there usually isn't someone who is designated to be responsible for designing and maintaining a good dev environment.
(2) Many devs don't have the expertise to do it. Build tools, operating systems, dependencies, and configuration management are a related but different skillset than coding.
(3) Many devs don't want to work on it because they enjoy coding more. Just because you enjoy one type of work doesn't mean you enjoy the related work. (This also applies to writing documentation.)
(4) Fixing / improving the dev environment is not rewarded. You need to get X feature done by Y date, and fixing the dev environment may slow you down. It provides a long-term benefit, but not one that is easily measurable, plus that benefit is off the radar when evaluating the contributions of someone whose primary role is coding.
With this cloud environment, they've separated those responsibilities out and given them to a specialized team dedicated to it. IMHO, some of the success is probably due to the organizational structure, not just the technical differences.
This is a sentiment that is very typical among Singaporeans. I suspect it has something to do with the political/business climate over there.
While the USA is also litigation happy, it actually takes an incredibly high amount of deliberate destruction for an employer to succeed in suing an employee.
If the company IDE is annoying to use, then rewriting it is probably a net time saver. The time consuming part of coding is rarely typing the code, afterall.
Don't you then have to maintain a copy of all the dependencies and input data on your personal laptop? I can't imagine myself being productive this way, even for small standalone projects where cloning everything to the personal laptop is feasible. This sounds awful and the company IDE must be even worse somehow.
Reminds me of a project many years ago where I was forced to use a companies crappy home-grown CMS that deployed all their microsites, they had a weird code editor that they expected you to develop in inside the CMS. I wrote a selenium script that would login and upload the contents of my local files into the editor.
I also don't use VS Code usually. My personal preference is a JetBrains IDE, but my point is more engineers should be able to choose the environment they're most productive in.
That looks cumbersome enough to sound like you guys are trying to discourage the method entirely. A custom password that has to be copied and pasted every time? A lot fewer people will put up with that for long enough, and I'm sure whoever came up with that and whoever approved it, they all know this.
How is the performance on jetbrain IDE. I love webstorm & intellij but they are Soo slow compared to vscode for me.
At first I liked them because they have lots of great functionalities out of the box, but vscode has caught up and just yesterday I switched back to vscode from jetbrains.
For java spring and react
> How is the performance on jetbrain IDE. I love webstorm & intellij but they are Soo slow compared to vscode for me.
It's pretty configurable. In my experience, it's pretty quick. It's not text editor or VS Code quick but the only thing that ever trips me up is the occasional re-indexing. Well worth it for how well it does everything else.
Yes, with special Electron shenanigans that still work in a normal browser well I suppose. The web UI has a few features that make it more device agnostic too (e.g. a hamburger menu to replace the File menu. Yes, you can run Codespaces on an iPad.)
Code server (https://github.com/cdr/code-server) works in a similar way using your own PC as a host, but for whatever reason Codespaces is more performant.
To be clear, the UI is a web app. But that is serviced by out-of-proc servers written in many different languages (e.g. language servers are usually written in the same language that they service).
I am wary of requiring an internet connection in order to develop, but there seem to be more and more advantages to using something like Codespaces.
The article describes using pretty much any IDE, so they are just offloading the setup and running of the development environment to the cloud while letting you run your preferred IDE locally.
Dependency management is usually not a big part of programmer workflow (unless you are a devops or something). Download dependencies and docs, go offline, write code.
People who try working offline are exactly the people who want to do it without Stackoverflow.
They're also the people least likely to want to use this kind of setup in my experience.
My company has a remote VSCode setup not that dissimilar from this setup. Big C++ codebase. Many engineers love it, even some of the "old timers." Our interns can be productive on day one and it brings a lot of productivity to not have to worry about caring for your dev environment.
But there are plenty of people who really just want to SSH into a beefy desktop (or directly edit on that desktop) and use their vim/emacs setups. These are the people who know how the whole build stack works, can debug the vscode/codespace magic, and can do amazing things not available in the VSCode plugin library.
So it's good to be able to do both. Allow power users to use their own tools, but don't require everyone to be a power user to sit down and write some business logic. Give nice rails to ride but don't require them.
I can see how a web based IDE speeds up on boarding and reduces the friction in support and collaboration. But I wonder if it reduces the skills engineers develop over time?
What skills would developers really lose? Speeding up boarding seems like a win, not many skills are developed imo trying to set up you personal dev environment. It's mostly just frustrating when a company's tools don't work well on your machine.
Being able to manage the frustration could be one such skill. And also being ready to fix things instead of assuming that employer must set up a perfectly working environment for you.
I bet smaller/poorer companies may want programmers who are ready to fix tooling when it (inevitably) breaks
I see your point, but I think increasing productivity may be worth potentially losing some of the skills you mentioned.
Additionally, it's not like developer set up is the only thing that test your ability to manage frustration. I would say developer set-up is just something you need to get through to start your actual job, and if that can be eased up then it's worth it imo.
For smaller/poorer companies, they could always hire an engineer that specializes in fixing broken tooling. Ultimately it's a company's job to hire competent engineers.
Thinking back to all the times I had to set up my workspace, I wonder how much did I really learn vs just Googling and getting quick fixes.
Some people literally disconnect the cable to get rid of all the online distractions. I've seen this mentioned in a few threads on procrastination here. It's sad some tools will not let you do that anymore.
Okay? That's three editors and there are hundreds of others (though probably <25 that are used by more than a few people). There are sizeable followings for other CLI editors like Neovim, Kakoune, Yi, etc.
Google allows for normal desktop IDEs, plus having a web based one. The funny thing is, many people move to the web based one because it is so good.
But Google is also unique in how piper/citc[0] (our source control) works. It's effectively designed for web/cloud based style development, so the workflow for web based dev and desktop dev are effectively the same.
So web dev can work, but I don't think the existing tech that most of the software industry has is built well to support it.
Google has a monorepo in which nearly all of the company's code is stored. The monorepo uses a custom implementation of Perforce. No personal computer could download a whole copy of the repo. So for that reason people need to check out a virtual/cloud copy of the repo in order to do work. This has the added benefit (for google) of not allowing individuals to have code on their device which is bad if the employee loses their device or has it stolen.
There are some exceptions for some teams (like Chrome). And when I was there exceptions for people who did mobile (iOS/Android) development, though Google was moving pretty aggressively towards a world where even iOS code wasn't allowed on the employee's device.
It's been a while since I've used Perforce, but wouldn't each team have their own 'path' in the depot and set their clientspec to only download that? Why would anyone, other than a build machine, need the whole repo?
Not a Googler but we do have a monorepo... you can start with a team specific path, but you feed that to the build system and ask it to identify all the other paths where your (transitive) dependencies are.
We also "allow" both but the size of the repo and build process are quickly getting out of hand for something to run on a laptop. In the past we had tools to manage the size of what an IDE had to index, etc. but these were abandoned in favor of the cloud effort, so to develop locally with any sanity you have to use years-old IDE versions and pretty much debug it yourself.
The cloud environment builds and runs faster but the autocomplete, go-to-definition, etc. are pathetic.
Don't most IDE's have the ability to edit remote files over SSH? I know VSCode can, and I remember reading a comment from an emacs user doing that. It looks like IntelliJ can do that too, though it seems to edit local copies of files and sync over SSH. There's also SSHFS, though I don't know how good of an experience that is.
It can be a pain to set up, but is it more of a pain than setting up a dev environment the old fashioned way?
Good point! I suspect the issue will be running tests. From what I gather they did all this to save developers the pain of compiling ruby gems + specific stuff on your environment (which can be a huge pain...)
If you edit a file using some sync over ssh option, and want to run a test - how do you proceed? And how is this not the same thing as just checking out the git repo locally in the first place?
> If you edit a file using some sync over ssh option, and want to run a test - how do you proceed?
Well you have your files synced over ssh and also have a terminal over ssh which you use to run your tests. Am I missing something?
> And how is this not the same thing as just checking out the git repo locally in the first place?
It's not just avoiding cloning the git repo. It's avoiding building all the dependencies and getting them configured and running. For most of my projects that's not a big deal but I expect GitHub has several databases with specific configuration, memcached or redis, maybe custom patched builds, etc.
Emacs has TRAMP, which does a very credible job of making remote resources appear local, not just to edits but to most Emacs libraries as well. Its most normal mode of operation is over SSH. It's not very hard to set up and it's pretty magical once it is. Even Magit just works. I was skeptical earlier this year, but 6 months later I'm doing 80-90% of my development in TRAMP buffers without thinking about it.
An IDE needs to be able to do more than just edit files, though - e.g. for debugging, it needs to know how to run the code on the remote box, and it'll be different from local.
GitHub isn't forcing anybody to use a web IDE. The most broadly used access interface is VSCode. Then there are heretics like me who choose to ssh in so we can use Vim. This is described in the article.
This (quick to provision vms of any size you want with your code base etc) for jupyter sounds like a colab and mode killer...the way both of those make it hard to actually check in code leads to a ton of copy and paste code etc. You all should really advertise this capability.
Do you have to drag your .vimrc and plugins along every time you set up a new code space? My vim setup is pretty bespoke, it would be annoying to have to set that up every time.
I once had a setup where I could just clone a repository and run a script to get my dotfiles set up. For Vim, I would just need to then run the command to install the plugins and that was that.
Edit: Here is the CLI program that helped me set this up:
I also have a very bespoke dotfiles config that predates Codespaces (https://github.com/wincent/wincent), so I made a thin wrapper around it that makes it work (https://github.com/wincent/dotfiles). For people with less complicated set-ups (ie. basically the entire universe), it is pretty straightforward.
It's totally worth taking the time to sync your dotfiles via git. There's a little bit of maintenance involved whenever you make a config change on one system that you want to propagate elsewhere, but it pays off.
Personally, I just `git init && git remote add ...` in my home directory and have a .gitignore that ignores everything by default, so I have to `git add -f` whenever I want to sync a config. Submodules work well for `.vim/bundle/the-plugin`. It's also nice for syncing important zsh customizations like this one: https://github.com/MatrixManAtYrService/home2/blob/master/.z...
Well, it's a Linux VM/container, so while you could do that, I don't see why you would, I think all a dev would want to do in a dev container would be command-line in nature.
It's not just a dev container, it's meant to be your workstation, with the code files being stored in it and not on your personal machine. So if I want to install IntelliJ, Beyond Compare, etc. Or even if I don't want to bother with setting up tunnels and port forwarding for testing my web app locally, I could use Firefox in the VM, etc.
Seems it would solve all the issues, which are specifically that it mostly works only with VSCode or terminal apps, but with a remoting solution you could run anything you'd want.
How is the latency btw? I SSH to a beefy box that sits under my table and sometimes I hate the latency of running emacsclient over X. But I am not using VIM and I don't use terminal emacs because of clipboard integration betweeen remote emacs and local desktop. I know there are hacks to make clipboard sync between local and remote emacs sessions without running emacs over X11 forwarding, but I always found those to be janky..
It's very good. The rendering and inputs are done locally. Completion, compilation, linting, and other things are done remotely but they are not instant anyway so the network latency is not really noticeable.
I use Github Codespaces with a Vscode and vscode-neovim. Neovim is running locally.
not op, but amazon uses "cloud desktops" with SSH access company wide, and no one really has latency issues there. So its def something supported at some companies...
I don't know how many people would want to use graphic applications over the network however, seems like it would be janky
I’ve always thought that remoting to edit text files using X or RDP is doing at the wrong level of abstraction.
VSCode editing over SSH works really well in contrast. I think Sublime has a plugin that ain’t bad. Emacs (TRAMP) is probably the worse of the three :/.
Terminal Vim or Emacs over Mosh is a pretty good option too.
I would be happy to let go of an employee who bank their employment on the preference of toolings.
Hear me out:
Toolings are, like the name suggests, a means to an end. If a web IDE is fast, works reasonably, and can continue improve itself. And the learning curve is friendly to engineers with different background and experience. Everyone should be comfortable to be nudged to use it. And if the tool additionally is a critical product of the organization, everyone should be comfortable to be mandated to use it, because it's now a very valuable source of dogfooding.
I'd absolutely never work with someone so arrogant (and, for lack of a more polite word, dumb, as you).
When I work for someone, I do the work they ask me and in return they pay me AND I get to learn something. For my learning process, I need the work part to be over fast. And that means I need to use stuff that I works best for me. I also need to be able to be able to fulfill my potential and spend time doing actual work (again, using tools that work for me). I don't want to spend a day setting up vscode and then realizing I can never edit as fast on it as vim what with the 300ms latency it adds to every keystroke
Yeah, whenever I hear this kind of rhetoric - if my company ever required <some minor annoyance> it's time for a new job - I roll my eyes. Sure, keep pushing yourself out of more companies, there are a lot of good engineers* who will be absolutely as productive as you and who won't be a drain on everyone around them.
*it's always engineers making these statements or I only know engineers
It works both ways. There are plenty of companies that will happily take productive engineers, without forcing them to use a slow, poor-quality IDE as their main tool, full-time. It's not a minor annoyance if it's the tool you use throughout the day, to do a job for which you are then evaluated.
"Tools are just means to an end" is just one of the many rhetorical devices that cushy managers use to eschew responsibility. Of course they're means to an end, but some means are better than others, and usually those with better means end up doing a better job.
I hear you, but I'm suggesting that this trait is part of what differentiates a good engineer. If I'm hiring someone and they have a bunch of recent moves that are stemmed from reasons like "they wanted to use some process that I refused to try" or "they made us go through some training that I thought was a waste of time", then I consider that to be a very risky candidate.
There should honestly be some good faith on both sides. It doesn't make sense for a job, in most cases, to be strict about an editor. But it's also not a good move for eng to be walking around talking about all the fragile things they'll quit abruptly for.
> stemmed from reasons like "they wanted to use some process that I refused to try" or "they made us go through some training that I thought was a waste of time", then I consider that to be a very risky candidate.
I feel like you're gradually making this hypothetical engineer seem more ridiculous. If someone quit over training or their unwillingness to try scrum, that would be bananas of course. There's an extremely large gap between that and mandating the main tool to use to do your job everyday.
Totally, I think if something is making you quit it must be pretty annoying - or you wouldn't risk trading the known with the unknown (which can very well end up being worse).
I also wouldn't mention the real reason in a future job interview, if it sounds ridiculous.
That said, small things can look trivial from the outside and make your life a living hell at the same time.
I'm also not sure I agree that there are few good engineers. I think that's some rhetoric that we all say because we all think we're some of the few good ones.
In my experience (FAANG, startups in the valley and startups out of state/country), strong engineering talent is not particularly difficult to find, but finding a good fit is the tricky part. And I would wager that eagerness-to-rage-quit is a pretty good indicator of someone who will have trouble finding teams that they fit on.
Not a rage quit, but I absolutely would look elsewhere. Folks are allowed to have dealbreakers for where they work.
A lot of people "rage-quit" basecamp over their no politics thing. I probably wouldn't have, but to each their own.
FWIW, I have also spent lots of time at FAANG, startups, etc and haven't had any trouble finding a great fit. I have also never looked more than a week or two for a job when I wanted to leave, so that does factor into my willingness to leave if I'm not enjoying my job.
That depends on what kind of companies you want to work for.
If you look outside of FAANG (companies in the rest of the world with unattractive stock), finding good engineers is a problem and most codebases are filled with horror.
The plus side is that, once you're a good engineer, you get little stress/pressure and a lot of leverage on flexibility (eg. remote in cheap places, in pre covid times), even if you won't make as much as FANG engineers.
Also, interviews don't require 2 months of preparation every time you want to jump ship.
I'm sure if you're paying FAANG money you can find plenty of good engineers happy to do backflips.
I wouldn't call it a minor annoyance when it's a way of working you're practicing for 6-8 hours a day.
I am best placed to determine what tools I work with best to get work done. You wouldn't get an electrician in to do some work and then decide you can suddenly prescribe the tools they use.
I just don't have the patience for this sort of nonsense, there are a dozen clients who won't micromanage the operating system, editor and way of working that I'll be using to get the job done. With that said, obviously it's a two-way street and some willingness to be flexible needs to be shown on both sides. If $client wants me to sync my work with their $uniqueVersionControlSystem then fine, I'm willing to put in the extra effort to try and meet their specific needs within reason. But that doesn't extend to working inside a client-provided VM, using a client-prescribed OS image, using a client-specified IDE or making other such major changes to my established ways of doing business.
And in fact any client trying to micromanage this sort of stuff is a massive IR35 working practices red flag over here in the UK. You're better off terminating the agreement and signing something else as a SoftEng contractor.
And I would be happy to end my employment at an employer that mandates what IDE you use. (I would not be surprised that GitHub forces me to use GitHub for development, or Asana uses Asana to track issues, this one is just too far...)
No hard feelings or anything. I spend 5-8 hours a day working in my IDE. That's a huge chunk of my life and it matters a lot to me to work how I want. I've also found getting this aspect of my worklife right greatly impacts my overall productivity.
> I would be happy to let go of an employee who bank their employment on the preference of toolings.
> Toolings are, like the name suggests, a means to an end.
It goes both ways. Employees are paid to accomplish an end, and I understand employees who are happy to walk away from an employer that micromanages the means to that end down to preferences in tooling. The reason you quit isn't because you miss Emacs (even if you do), the reason you quit is because your boss doesn't even trust you to choose a text editor.
That said, if the thing is your own product, yeah I get it: dogfooding is important. That said, I know of teams at Microsoft who do their (OS-agnostic) work on macOS... at some point accomplishing the end is more important than dogfooding, and you just let the workers use the tools that they're comfortable with.
The article mentions supporting vim and emac, which probably means any editor can be supported.
Visual Studio Code is great. It’s the primary tool GitHub.com engineers use to interface with codespaces. But asking our Vim and Emacs users to commit to a graphical editor is less great. If Codespaces was our future, we had to bring everyone along.
Happily, we could support our shell-based colleagues through a simple update to our prebuilt image which initializes sshd with our GitHub public keys, opens port 22, and forwards the port out of the codespace.
I think terminal editors are supported by allowing an SSH tunnel, and both Sublime and the IntelliJ suite have plugins to allow you to use them against remote systems over ssh
> The article mentions supporting vim and emac, which probably means any editor can be supported.
I read that as you can ssh in and run a terminal-based editor, but that doesn't seem to offer much support for people looking to edit in NEdit or Intellij or Atom or Sublime or whatever else. Maybe some of those could be supported by forwarding an XSession out of the image, but that's going to suck hard for people on macOS whose editor is now the Linux/X version, with all of the attendant mismatch.
From the article, it looks like future support for other IDEs is definitely possible:
"Visual Studio Code is great. It’s the primary tool GitHub.com engineers use to interface with codespaces. But asking our Vim and Emacs users to commit to a graphical editor is less great. If Codespaces was our future, we had to bring everyone along.
Happily, we could support our shell-based colleagues through a simple update to our prebuilt image which initializes sshd with our GitHub public keys, opens port 22, and forwards the port out of the codespace.
From there, GitHub engineers can run Vim, Emacs, or even ed if they so desire."
I use this for developing on linux/pi/jetsons, (need to test linking against real libs, and cross compiling for some reason still isnt a thing), but the compile times are terrible...
I wish this was even more modular so we could have a dumb compile farm, +local ide +run&debug on-device
(I've recently got 90% closer by remote sshing with vs code to a pi vm)
interesting read and lots to glean - but seems to draw a false distinction between local and cloud in many areas -
env portability/consistency/using prebuilds as an example could have been done with in house CD/vagrant/packer ~10y ago. This post doesn't focus specifically on that, and there are additional benefits to the approach taken, but the notion that local dev is incompatible with these types of technologies is a consistent background theme in the writing
also, referring to emacs as non graphical and 'shell based' is a bit off to say the least.
Buy a Cachix subscription and now the org can build once on the local dev's machine, push to the service, then other users and machines can download & run anywhere the reproducible 'apps' on local dev machines, CI, and even production. CI can even build and push the architectures that aren't specific to your local machine to cover just about everything.
This is what we do at work, and it's working great for us.
GitHub's approach seems utterly ridiculous, especially given that I've seen colleagues prevented from working because their Internet connection at some point ran through a part of the world that was sanctioned by Western governments.
Developer, it appears you are somewhere near Crimea! NO EDITOR FOR YOU!
Apparently you can set up Codespaces with any Docker image you want, so there's no reason you couldn't use Nix to generate one for use with Codespaces.
There are lots of options for making your dev environments reasonably quick & easy to set up if you're mostly doing web stuff, or developing for a very small count of platforms, and mostly or entirely with open-source tools. Change any of that and things start to get harder in a hurry.
> My friends, I’m here to tell you I was a Codespaces skeptic before this started and now I am not. This is the way. ~@iolsen
I don't actually doubt that this (and the 4 other glowing employee quotes) are real, but even assuming they are, I can understand people remaining skeptical about the sample size of 5 being broadly representative of the 100s (1000s?) of engineers at the company.
Also slightly hilarious that "This is the way" is a Star Wars callback referring to a character blindly following dogma before eventually realising the error of their ways...
On the other hand, there's quite a few "hints" littered throughout the article that their current setup is a bit of a beast, so I can definitely imagine some engineers being relieved to be able to get such a behemoth off their physical laps/desks: I guess if the local development experience is sufficiently brittle & frustrating, any alternative may seem welcome.
I think people like not having their physical machine tied to their dev environment. IMO "the way" to do this is with ssh and tmux, not a web app. As long as people have choices I guess it doesn't matter.
Yes, I use neovim myself! Codespaces looks for a dotfiles repo under your user account, which you can use to install any personal customizations. Mine, which installs a bunch of neovim plugins and configs, is here: https://github.com/dcreager/dotfiles
Ah, fantastic! That sounds like a really good solution. I also have my carefully crafted configuration preferences for Neovim, and wouldn’t be happy with codespaces it didn’t allow me to configure it. But for what you are telling me, they have done it in a smart way. Nice. And it’s nice that they had the people who work mostly in the terminal in mind when creating this. Neovim is a great editor right now, it has been such a fantastic evolution over Vim, adding a language server, for example. Cheers, thanks for replying.
If you read the article, that's an option with Github's code spaces setup.
> Visual Studio Code is great. It’s the primary tool GitHub.com engineers use to interface with codespaces. But asking our Vim and Emacs users to commit to a graphical editor is less great. If Codespaces was our future, we had to bring everyone along.
snip
> From there, GitHub engineers can run Vim, Emacs, or even ed if they so desire.
I think this defeats the purpose of Codespaces, because it would mean that all the code runs locally and only thing that remote server provides is storage?
The repo owner gets to define the base Docker image that’s used in the codespace. And then it also looks for a dotfiles repo under your user account, which you can use to install all of your personal customizations. You can see mine at https://github.com/dcreager/dotfiles
> I think people like not having their physical machine tied to their dev environment.
I'm curious to know if this statement is true. Most experienced developers these days have a regular desk and a chair where they like to code and focus. How prevalent is the move-and-code scenario?
It's not just that. There is also the matter of laptop getting broken (we all have proper backups, right? right?), stolen, ...
So I do see the appeal in thin client. My only issue is that the experience his closely tied to latency, so sometimes working over ssh is... irritating. Especially on mobile connection.
It’s all Docker images under the covers, so our devs also have the option of downloading and running that image locally instead of in a Codespaces VM. That was always possible before, of course, but Codespaces gave us the impetus to invest the effort in Dockerizing the dev environment more rigorously.
Oh, that is cool. Sounds like a nice mix actually. Local vim, browsers, known system. And container with mount-bind to actually compile and run stuff in.
I've been playing around with a homegrown version of this. It's really not that hard: just set up a docker context to your home lab server and use VSCode remote containers.
I've been able to remove WSL from my PC: Docker desktop was gouging itself on resources. I can now also shut down my desktop and continue exactly where I was on my laptop, and visa-versa. I don't have to pull WIP commits back and forth between the two machines.
It's a major leap forward, and there is no need to use GitHub infra.
I also started doing this, but using containers locally (on a Mac). The ability to have a set dev environment without needing to install local programs or compilers is great. And really once you flip to this model, it doesn’t matter if you’re running your dev in a container or remote VM. So long as you can get a command line to your dev environment, it really doesn’t matter where it is.
Though I like this model, the problem with dev containers on macOS is that the disk io is bog slow. This makes some IO heavy tasks (pip install, npm install) painfully slow.
I have started to run some dev tasks on a remote Linux server, with Dockers, and then just configured automatic file sync on save over SSH.
Very true. It really depends on what the workflow is. But once you’ve already moved to a disconnected dev environment, moving from a container to remote server is a small step.
> This makes some IO heavy tasks (pip install, npm install) painfully slow.
Aside/HTH: The new Yarn has "Plug 'n Play"[1]. They claim that committing [their version of] node_modules is a reasonable thing to do, so that's one way to avoid install entirely. I don't trust that claim, yet (it smells like noisy diffs). You can instead re-enable the global cache[2], and then mirror your host cache directory into your container (whatever `yarn cache dir` gives you).
I have tried none of this myself because I'm not working on anything JS right now. pnpm[3] is what I've been using up til now, and is also a direct upgrade in terms of speed (but I suspect it won't work with mounting host directories).
You edit media file in an editor?, wouldn't you just upload it? You can drag& drop into vscode and probably find the path to the tmp folder and work off that too.
This only does code. You'd probably have to do those locally and SCP them over or, yeah, some mount magic.
> What happen when you need to work from a shitty internet network ? In train ?
You'd have to commit your work, but you'd be able to bring the environment up on your laptop. There are still merits to this, such as the repeatable nature of containers. Depending on where you've worked, you may have been through the hell of a huge list of setup steps when on-boarding a project: that's one thing that containers are really good at automating.
VS Code remotes allows you to drag and drop files from your desktop and they're placed where you want them in your remote environment. I don't understand why this would be any different.
I'm not sure why you're being downvoted, that's a very good question. The short answer is yes*, plus some automation around spinning up a new session.
The wrinkle is that the display server is running on your local machine, and connecting to the thread of execution on the remote server. Vim doesn't really have that concept of separating view from control, but tmux does play a somewhat similar roll.
I'm guessing that VSCode has a distinct back-end and front-end. It's able to run the back-end server on the remote machine/container/whatever over a bog-standard SSH tunnel. Any autocomplete functionality, debugging, etc. will use the resources on the remote machine. It also does port forwarding for you. The front-end (GUI) runs on your local machine.
A closer analogy would be using Plan9, and mounting most of your cloud machine locally. Vim would then just be your local front-end into those mount points.
Containers/docker add the ability to share your exact machine configuration with someone else, and have them create their own copy of it with a single command. It would be almost like sharing a VM image with your team. There's been a lot written about the pros/cons of containers, but they achieve similar goals to VMs, and so-happen to bring a lot to the table in this scenario.
I have an HP 4u from 2010 with 80 threads and I think a quarter TB of memory, 6 SAS, and 2 1TB ssd, currently running Ubuntu 16 and Wok, for VMs. It took a lot of effort to get this working and it's still a huge pain - I have no UPS for it since it uses 220VAC, and UPS cost more than the server cost me, for example.
I can't use docker in general because my internet is too crappy. That was annoying in 2012 when I first used it, but it's even worse now. I've always worked with VMs, since 2001 or earlier - virtualbox, wok, AWS, wok, and now proxmox.
It helps to know what your use case is, I don't run a ton of VMs, just one or two at a time that can use almost all of the resources of the server, and I generally do infrastructure and computing work in general.
I presume none of this counts as a homelab, though, since I don't program in rust.
They aren't as expensive as you'd expect[1][2] ($70-$200). Hell, you could use a RPi for many use cases.
In my case, my homelab is my laptop (which I connect to from my desktop). I don't actually have a server blade lying around (although those can also be relatively cheap when big cloud upgrades their servers).
I pay 30$ a month for a server (yes, a virtual one, if you want docker, you have to pay a bit more) with org-domain and tls cert. I would recommend that to every developer. Yes, some administrative work is required but that is also a valuable learning experience. You would own all the tools that you need to develop efficiently.
Personally I use Gitea, which should be usable by someone used to Github. I don't switch workspaces too often, but I am sure some tools are available.
VS Code remotes are magic. I use a light version of it for 100% of my development on windows with WSL, but I also use it to log into a mac server and edit files on that.
I have no idea what actually went into developing the feature, but I would have to suspect that Electron would have contributed to making the separation of an app client vs server easier than something totally "native".
I don't. My initial comment is assuming they use the public version. Whether they use the public or a private version they still have back-channels for support that the rest of us don't have access to.
I’ve very recently become a GitHub engineer, and I got pre-access to the beta too and I must say - my absolute favourite thing about CoseSpaces is being able to dev on a repo you don’t work on regularly and probably wouldn’t contribute if it meant having to set up environment etc. It’s really nice to just dip into a project with a working environment in seconds, make your PR and then move on.
This was my first idea for introducing Codespaces to my team -- set up the infrastructure to be able to spin up a codespace on our less-used component repos where people don't have a local dev environment prepared.
How does it work when you work on several packages at once,e.g. a lib and an app using it? Is it a monorepo, do you configure both dependencies to be locally editable copies, etc.? For example patching an upstream open source project that is a blocker for your app.
The codespace is backed by a regular Linux VM, so we can clone the upstream repo and edit/compile/debug/etc in that just like we would locally on our laptops and workstations.
You can also create a separate codespace for each package if they’re independent of each other and you don’t need to test things in the downstream repo in a way that requires your changes being available “locally”.
This is a problem that can be solved without Codespaces too. For example, if GitHub were to embrace Nix, every project could have a shell.nix file for all of their dependencies, and the new engineer bootstrap script that installs Nix could then just add an internal Nix binary cache (or use Cachix). Any time the shell.nix changes a GitHub Action could push the closure to the binary cache.
With this setup, you can use whatever local tool you want and, as long as environment variables are respected, they’ll just work. The only requirement on your end is to either run nix-shell first, or to use direnv and ask it to load the nix env for you.
The downside is you still are compiling locally, so you still need sufficient CPU/memory resources, but that’s it.
Unfortunately most companies still aren’t paying any attention to Nix.
I'm not sure that comment applies here. The suggest Dropbox replacement was more complicated than Dropbox, and had more steps. On the other hand, Nix is about the same amount of complexity, and same number of steps for the developer.
Yes, the parent comment looks a bit complicated, but that's just the one time setup, not something that needs to be done by every developer. And Codespaces will require some of that setup too; it's not like it'll magically know what packages and dependancies your project requires to build.
Yeah, it’s really fast. In fact before I joined, I also set up a default CodeSpaces environment for my own repo and then I was up and running in only a few minutes, and that was the first time so I was learning and reading the docs. I could then set up all subsequent new CodeSpaces to start in seconds.
But yes, once a project commits to using CodeSpaces dev environment (even and especially ones with large code / heavyweight environments) there is a lot that can be done to optimise the spawn up time, that cannot be done as easily for local dev.
I'm a big fan of capturing the toolchain(s) with the repo. I have had to hack this for embedded toolchains out of necessity for years (decades?) using VirtualBox on a Mac. I still have a Windows 95 VM sitting around containing a copy of Keil or something that is the only known way to rebuild the code for a certain weird-ass micro from some consulting gig in the late 90's. I wonder if it still boots? Hopefully, that customer forgot about me...
Not about Codespaces, but about Web IDEs in general, they're amazing. I've not used Codespaces but I have used what is being talked about here [0]. It's one of the biggest productivity improvements I've seen from Google. I never really have to think about what packages or software is setup on my laptop and I can change laptops or computers while having the entire state of my IDE preserved. It's a pretty amazing experience.
The only people who I don't think would instantly like this sort of thing are people like game developers who basically don't do any unit testing of their code and run the binaries they generate. For those users you could find quite a few ways around that.
> Also slightly hilarious that "This is the way" is a Star Wars callback referring to a character blindly following dogma before eventually realising the error of their ways...
My friends, I’m here to tell you I was a Codespaces skeptic before order 66 and now I am not. Good engineers follow orders.
Brittle local development in from what I've seen is either due to another symptom of tech debt - an aging monolith with too many different build systems, dependencies and what have you that you end up with way too many Docker containers or the need to have a very long doc for newcomers with "run this script here, then that script there, unless it's a Monday, then run this script over there". Or a team that's gone all-in on a distributed microservices architecture with lambdas all over the place and a dozen or more repos and it's literally impossible (yes there's LocalStack but...) to get a faithful end-to-end local environment up and running.
The nightmarish local development experience of either case are symptoms of deeper issues than local development itself being a problem.
The team here is part of the company that markets Codespaces. They're not actually representative of other companies, so I wouldn't take this as a direction all software development would be taking.
No. You can achieve the same benefits of local reproducibility with Nix/Guix. Guix is more composable IMO because you can program multi-service development environments entirely in scheme.
Is it just me or VSCode is dauntingly complex? There are so many configuration options and I often get lost in it.
I've been using Sublime Text for a decade and it has kept the scope creep to a minimum. I've also used JetBrains IDEs and while there is a massive amount of complexity, it is well organized I think. I just get anxiety from launching VSCode, may be it is because of not being familiar with it, I don't know.
I don't use any of the features of VSCode. I just use it as a simple text editor. I'm mainly doing yaml files, bash scripts, and Jenkins pipelines, so I don't need any fancy features. I just "alias vs='code .'" and then "vs" in any directory I have code. It's just a fancy text editor to me that also shows git statuses as well which is nice.
So you're telling me that the next GitHub outage could take out my dev environment and give me an afternoon off? Time to convince management that we need to switch to Codespaces!
You say “though” as if it absolves them. If they break basic requirements of a Git host like fetch and push while working on this extra feature that’s on them.
There are different risks, but there is not necessarily more risk. Does Codespaces have lower or higher than the availability risks of your current environment?
If you've never had these issues, this product is probably not for you.
The question is, how much time do you spend on dev environment issues per year?
When I was at one of the FAANGs, our dev enviroment took about 1hr to install, and you had to redo it everytime you switched platform / version (about every week).
We spent ~50 hours per year managing our local dev enviroment. probably more in reality.
> The question is, how much time do you spend on dev environment issues per year?
Definitely not little. But at the same time, I usually learn a lot about how stuff works during that. Which I like to think makes me better at helping other people debug weird shit.
I’ve had corporate laptops get BSOD’d because a sysadmin pushed a bad GPO rule that worked fine on 99% of their clients but butchered the developer laptops.
Another fun one was when an org was rolling out CarbonBlack on all endpoints and decided to block all Java. Because apparently Java in the web browser is a security threat. They blocked 30 devs Java runtimes as a result. So there was an entire afternoon wasted for those guys.
To a corporation the risks of GitHub being down might be ok. Plus this will drastically increase on boarding devs to an existing project.
In the context of my comment of having a free afternoon off - having my local environment messed up means that I'll need to spend my afternoon fixing it instead, whereas having Codespaces go down is essentially an announcement to the entire engineering team to go take the afternoon off as there's nothing we can do
I was curious about the cost. The pricing and compute resources just about correspond to an Azure A series instances, but the ram corresponds better to the v2 series, which is cheaper than v1. I wonder if they're upcharging for v2 machines.
Another kid, got access to every single private github repo.
It was on hackernews, he declared it, made a write up about it and got given a pretty small bounty from github. Google it, otherwise I guess the net was scrubbed of it. I follow him on twitter but yeah just google more and you should find out all about it.
I was always surprised it wasn't a much bigger deal.
Basically since then I think any company ( most that I work for ) are clinically insane to upload their proprietary code to the site.
On a related note - was it not just a couple of weeks ago people were concerned the codespaces system is using other peoples proprietary code? ( I havent kept up to that on that one so not sure if its still an issue)
This is very true and a risk with any cloud service. Ie if you make a app and want redundancy, the code is exposed not only by your own security defects, former employees etc, and each cloud provider you host on such as AWS, GCP, Azure, DigitalOcean etc. I think in practice the largest mitigation is the legal system, as its safe to assume your code already exists places you don't want it to.
I see the advantages with this, but this would be pretty annoying on a slow or choppy internet connection. I highly value being able to develop software with little or no internet.
I'm really surprised (and discouraged) to see how many people here are so enthusiastic about this. There is so much room for misuse. You're increasing your dependency on an external corporation, giving them more control over your development environment, giving up much of your autonomy, and somehow this is a good thing?
Development environments being overly complex to setup, long compile times, those are symptoms of software bloat and bad design, but instead of addressing these fundamental problems, people want compilation to happen externally on a 32-core machine so they can sweep those problems under the rug. Okay, let's see how that turns out.
While that does sound quite expensive, companies typically have a limited budget for employee laptops, but appear to have almost unlimited budget for cloud hosted testing environments, which usually have many instances (and services like RDS / Redis etc)
This was the thing that jumped out at me! I'm not sure how the VSCode application interacts with these to potentially put them to sleep, but dang that is $$$ for a dev machine at the scale github is operating at.
I guess lucky for them to be owned by Microsoft now and so that's just a cost of doing business. I have a hard time believing they could have even considered an approach like this if they were still paying for hardware out of pocket.
That pricing is a ~70% markup over standard Azure rates.
Companies typically buy laptops with an assumed 3 year lifespan; so, let's assume a high-end $3000 laptop, that would be ~$83/month. Of course, you need a computer to access Github Codespaces, so this isn't saving all of that money. Maybe companies can cut some costs by buying cheaper laptops?
Or if you want a more apples-to-apples; a Lenovo ThinkStation P620 runs ~$2800 for 32 threads and 64gb memory. DIY would also land somewhere around that $3K mark; a Threadripper 2950x (32 thread) is $1100; 64gb of memory is around $250.
This, of course, is the most expensive option they have; the more standard option would be 4/8; for which you would pay ~$85/mo (all the time, normal work hours), for the privilege of a machine that's far less powerful than any laptop any of our developers have. For god's sake, my M1 MacBook Air is an 8/16, with four of those cores far more powerful than any three year old Xeon Azure has running. And it was like $1400.
I understand maintaining dev machines isn't the easiest thing in the world. I have never once worked anywhere where it was such a problem that the company would justify tens of thousands of dollars in spend. Because it kind of is one-or-the-other; time invested in making Github Codespaces work for your company is time not invested in maintaining local dev environments. And Github's ideal end-state is companies totally rely on Codespaces, so the local dev environments languish.
Even with Codespace your company will still have to buy you a laptop, to run corporate VPN, security/virus software, background screen recording, and zoom/video calls. All of this will still need powerful reliable business grade laptop.
The cost difference between laptop meant for remote development vs regular dev laptop is less than $1k. ($300 for CPU upgrade, $200 for RAM, $200 for SSD upgrade)?
:sigh: this is to close to home. Nothing like malfunctioning virus software + background screen recording to eat all your local machine ram and cpu cycles.
I think a lot of these things are less important by the day though. Are VPNs or advanced security software even necessary if everything is in Notion/Google Docs and something like Codespaces is being used?
End-device security is always important; there would be nothing to stop the compromise and lift of, for example, cookies stored in the browser, or a keylogger to grab passwords.
I don’t think they’re trying to save on hardware costs here. This is to prevent any kind of friction and wasted time when switching between branches/environment.
If your dev costs you $100/h, you only have to save them +/- 5 hours of faffing around with branches/dependencies every month to make it worth it (assuming a 8 hour workday).
It's not worth it right now. The economies of scale could add up to make it break even, if not be cheaper.
Big could, of course. But considering it takes new devs at my company ~3 business days to get up and running (and we're a React shop), it could be worthwhile.
>I understand maintaining dev machines isn't the easiest thing in the world. I have never once worked anywhere where it was such a problem that the company would justify tens of thousands of dollars in spend.
Look, this is yet another product targeting naive VC funded startups. Milking the cows until they bleed.
The kinds of bills that some large company development environments end up running are surprisingly high, but those companies also think the opportunity cost of losing productivity on application team developers to also very high. That's how you find companies that decide to hire very expensive teams to, say, add types to a dynamic language, rewrite a standard library, invest in massive parallelism to run thousands of CPU hour integration tests in 15 minutes, or many other projects like that which no normal company would ever consider. If it gets some project team to cut delivery times by 3 weeks, it's all considered worthwhile.
Where we run into trouble is when we try to copy the shape of those big companies, and the kind of initiatives they fund, into startups, or mid-range companies. In quite a few ways, those top companies are better at development, but the base costs are only worthwhile if you have a money fountain of real revenue that grows far faster than your dev expenses. Trying to copy them with very different conditions is going to lead to tough problems, possibly company killing, and it's the kind of imitation we see all over the industry.
So yes, someone like Github is going to have eye popping expenses, because the alternatives to avoid those expenses just don't make sense to them. If you are not working in a multi billion dollar company, their practices can be interesting, but it makes as much sense to emulate them as it'd make for you to hire an entire Formula 1 pit crew to keep your commuter car in good shape.
If spinning up/down is fast+smooth (e.g., no reconfiguring stuff each time), you could drop that down to ~40h/week taking you to under $500/month. Still pricey, but combined with not having to pay the ~70% markup another commenter mentioned, it could be under $2,000/year.
Add on top of that, spinning down should be automatic. I've worked on bespoke platforms like this where it wasn't. Devs would often forget and/or be more willing to run long running tasks after work hours. The net result was pretty much having to assume an instance would be online 24x7 unless there were at least gentle pokes to disable instances.
And that's on top of the price for every engineer's machine that is used to access the instance...
Sure, you might not need to buy machines that are as beefy (although I bet you'll still need a decent machine to run all your chrome/electron windows) but enterprise maintenance on these machines is still a significant cost.
I definitely don't think one can look at this from the 'saving money' angle.
Ugh, now not only you don't own your code (ahem copilot cough) you also don't own the tools to develop code. The direction the web is taking is worrisome.
The issue is not so much senior devs, but new devs. If they start off with things like that, there' so much magic under the hood, they won't understand how anything works. They don't understand they don't own shit until it's too late.
I remember when I first saw the JAM stack, basically they use external services for *EVERYTHING*, claiming it's "The future™". They don't even own the data/database.
ownership issues aside, the "magic" under the hood is not a problem assuming it actually works. Its when the magic breaks, or works in unexpected ways, then it becomes a problem. Magic is also progress, and progress is usually good.
I think you could look at it in terms of sustainability: "magic" progress has the potential of "magically" vanishing if it's inner workings are not sound, reliant on an unreliable other, etc.
I initially learned about it because a junior dev asked for advice on one of his/her apps. It was a college app, simple, it just gathered info from students. But, they offloaded all data to a third-party database. So you are actually giving a company a lot of sensitive, personal information about a fucking college.
He/She was concerned (and with good reason) about the safety of data. Had to learn the bad way :(
I would trust a service by junior devs more if it was hosted by a third-party database because so many companies have insecurely set up their environments over the years. Especially if the data is encrypted in a way the service can't read it, what would be the harm?
There's nothing about jamstack that requires you to use third-parties (except the CDN part, I guess, but that's just a mirror). You could just as easily host your own CMS and database and call out to it the same way.
It's just that like auth, logging, etc, CDNs have been neatly abstracted out and service-ified in such a way that it's often more economical to go third-party. But that choice is orthogonal to jamstack.
I agree on that, and was going to add that, to be fair to the stack. But the reality is, while that's doable, most people just use third-party services, at least that's my experience.
Sure, but I think that's true in non-Jamstack code as well. Datadog, Snowflake, BigQuery, DynamoDB, Google Analytics, etc. are all examples that I've seen used by non-Jamstack software, and most have a fair bit of lock-in.
The same thing was said about every programming abstraction to ever be introduced. How do we expect people to build memory-efficient applications when they become reliant on a garbage collector?
There are a ton of a companies that help you track cloud spend, while some of it is architectural, most is really about waste, old ec2 image backups, s3 with no lifecycle policies ... garbage
1. Development environments are running on Kubernetes, and VS Code is remoting into them. You don't have to pay a monthly subscription for that. VS Code remoting is free-as-in-beer and Kubernetes is free-as-in-speech.
2. The actual Kubernetes hosting is the product "GitHub Codespaces" running on Azure. GitHub certainly isn't paying a monthly fee to themselves for this; they are, technically, self-hosting it. I agree that you probably do want to have the option to run it yourself, but I think you do, given 1.
This is already a thing, if you look at Java (old-school ik), Oracle vs OpenJDK [1].
I don’t see why that would be a problem though? If I can pay a company to provide improved GC for my infra, where the cost savings made it beneficial, I would do it…
I've never seen a company collapse because their garbage collector posted an Our Incredible Journey™ blog post and then shut down a week later.
I have seen multiple companies collapse because the proprietary Backend-as-a-Service they built their sand castle on top of pulled a vanishing act, however.
These tend to be early stage startups who went with BaaS to get around resource constraints and couldn't survive the rapid need to exfiltrate their data and build a backend with a gun to their head. Larger companies generally have the survival sense not to sharecrop business critical functionality to such a degree.
Which is to say, you wouldn't have heard of them, because they never got big enough to be known to anyone beyond a handful of clients and employees. I've personally been hired to try to rescue such a company, and was not successful.
> Larger companies generally have the survival sense not to sharecrop business critical functionality to such a degree.
Larger companies have also been known to buy out critical service vendors and in-house them when there was a risk that they might otherwise shutdown or pivot, too; that's less often an option fot smaller firms.
Do you consider AWS Lambda and such to be BaaS? I'm pretty new to the field and have been focusing on AWS 'serverless' as the backend for most projects. Would you say this is bad practice overall?
I don't think it's a bad system to build on, but I would be nervous to have it as the only option in my tool belt. Proprietary cloud services often create vendor lock-in, and the skills you learn eventually become non-transferrable as you get more invested in their system. If you only want to work at serverless companies, or on serverless projects, that may be fine. But, knowing how to SSH into a machine, maneuver around, and how to set up a server will always be valuable
(I'm not heavily experienced with Lambda but) I don't think you're in too bad a position, at least from what I've seen, as there does seem to be a portability story wrt those serverless functions.
The objection here isn't to hosting per-se – most companies aren't in the business of maintaining PostgreSQL and should consider outsourcing that task early on. If your host of choice shuts down, that's a standard service, you can dump and restore your DB and move on. Annoying but NBD.
The danger with jamstack or BaaS or similar approaches is working yourself into a place where a proprietary service API is baked into your product as a critical, fundamental dependency; if and when the host shuts down, you're in extreme trouble. That's what I'd heavily caution against.
(There's an argument that when the proprietary service is run by an MS or AWS or Google "they won't go bankrupt" so there's no risk, but even then those companies shift priorities and shut down services not infrequently).
Microsoft, too, will pull projects, or just completely rewrite them and make them incompatible.
Applications using .NET Framework with WinForms are not just a dependency update and recompile away from being ported to .NET 6. WinForms got a whole new API and porting complex UI effectively boils down to a rewrite. At least if you can. Microsoft doesn't support third-party controls in the WinForms designer of Visual Studio for .NET 5+, so have fun trying out positions and compiling over and over.
There's plenty of companies that wasted a ton of resources trying to roll their own version of things that should be services. Remember when Uber built their own chat solution? [0]
> With operations in over 620 cities, it was paramount for us to identify a chat solution that would enable Uber employees to reliably communicate on desktop and mobile regardless of where they were in the world.
Did Uber really need their own chat solution? Are they not better served by a SaaS product, one of 5 that 90%+ of companies are using? Are they still using uChat today? (honest question, I don't know)
This is especially dangerous when talking about security sensitive features. It is highly inadvisable to roll your own crypto
Lot of companies shut down or fail for lots of reasons. I think a business should focus on what they do best and what's critical to their mission.
But at Uber's scale I can see the business need for a custom chat app. Most third party chat apps are bloated, both the cheap and expensive ones. Plus there's the need for custom integrations.
I used to work at a large bank and they operated perfectly fine with Window's messenger. As does every other bank in the US, despite having tens if not hundred of thousands of employees. Some things require more specialized features like trade tickets (which Bloomberg does), but for chatting, you're not gonna be able to build a better solution than already exists. And if you can, then you should spin that off into its own business like Goldman did with Symphony [0]
The nice thing about being a client is that they support all this stuff and it gets better over time. Also something like Slack is absurdly cheap compared to the usefulness and productivity it provides.
> but for chatting, you're not gonna be able to build a better solution than already exists. And if you can, then you should spin that off into its own business like Goldman did with Symphony
The implication that Symphony is a better solution for chatting than already exists / existed.
As a (non-trading) user of the platform, I strongly disagree.
I don't know what the traders think, but I'd hope there's a USP in there for them besides "it's cheaper than a Bloomberg seat."
Uber grew extremely fast in its initial years and that resulted in large number of teams starting NIH projects to "make themselves useful". Uber switched to Slack a while back, in addition to laying off a sizable chunk of the engineering workforce, and it didn't really collapse on itself.
Nowadays, there's a lot more support internally at Uber for buying off-the-shelf solutions instead of trying to reinvent wheels.
> There's plenty of companies that wasted a ton of resources trying to roll their own version of things that should be services. Remember when Uber built their own chat solution? [0]
Sure, there's obviously a big difference between controlling the things that are critical to your business and wasting time outside your core competencies. And part of that difference is the risk profile: if your entire backend disappears, and takes the API your frontend is built around and all of the data with it, you're basically up shit creek. If your chat app of choice disappears, you can sign the company up for a different one tomorrow – it doesn't take 8 months of crunch to "port" your employees, and likely all of your critical business docs weren't in the form of chat history.
> Lot of companies shut down or fail for lots of reasons. I think a business should focus on what they do best and what's critical to their mission.
Agreed. But very few internet companies can claim that the data in their datastore and the entire mechanism by which their product accesses said data is not critical to their business, which is the distinguishing factor here.
Crypto is almost always implemented as a library, and probably ultimately as a kernel module in most cases. There is no reason at all you should need to call out to a remote service to encrypt and decrypt data. That wouldn't even make sense, as it would need to be encrypted before traversing the network to be of any use. And there are plenty of commodity chat applications like Mattermost and RocketChat that you can self-host, again not necessarily needing to use someone else's cloud service but also not need to roll your own application.
I'd argue that it always held true to a certain extent. You don't necessarily need the underlying knowledge to be an efficient programmer in Python, but knowing C or Assembly certainly gives you an edge when it comes to, well, edge cases.
And it isn't all about efficiency. Programming and CS is such a major part of my life, that I want to know the deeper parts, and I want others to understand it too. Programmers who do not seek that knowledge I unjustly judge for it.
> If they start off with things like that, there' so much magic under the hood, they won't understand how anything works. They don't understand they don't own shit until it's too late.
I'm sure this has been said dozens of times throughout the years as we've built more and more tools of abstraction. As software gets more complicated, it's OK if not everyone fully understands every part of the stack. People specialize, and then become experts at their small scope.
Or even if they're not experts at their slice, not every company needs a 10x code wizard who understand the system up and down. Some companies just need someone lightly technical to mess with website templates or write simple scripts and services.
Given the whole outrage over Apple taking advantage of customers' willingness to hand over control of their phones to them, I hope folks can see the parallels here and reject this.
At this point Microsoft's strategy around GitHub, VS Code, Copilot, etc. should be pretty obvious, and we should all be running for the hills.
What are they embracing, how are they extending it, and how do they plan to extinguish it?
My understanding of EEE is that it involves taking some competitive product/standard, making a cool MS version of it, and then killing off the standard version - e.g. implementing a barely sufficient POSIX layer on Windows, getting customers who have "POSIX" as a purchasing requirement to switch, and then getting them onto the NT API.
Most of the examples on that wiki page have been unsuccessful (IE trying to EEE HTML, Outlook trying to EEE SMTP, etc.), because they weren't able to get enough people to rely on the extended version and stop using the standard thing.
What is the analogy here? Microsoft wants to embrace and extend... compiling software? and will kill off... existing ways of compiling software? How will this work?
It's a slightly different flavor of the same thing.
VS Code is an amazing tool that's getting huge adoption because of how awesome it is, and how open source and community centric it is, etc. They've gotten a lot of mindshare and dev love, that's the embrace bit.
The next step is the set of closed source addons. Have you noticed that a lot of the new VS Code features are now in addons that are under a different license. This includes the new remote dev and python tooling. Still free to use and awesome tools, of course, but fully under MS exclusive and controlled. That's the extend.
I don't know what the longer-term plan is. I'm hoping that it won't lead to an extinguish. But if that's what they want, then they could e.g. cripple the open source version, moving all their dev effort into a closed-source, Azure/Github only web environment. Who knows?
I don't see what post was moved, this is exactly EEE, adopt an open strategy, extend it with proprietary extensions, don't let competition use those extensions, extinguish competition by not being able to interface with your proprietary extensions while having created an expectation from the user for them to be available.
>> What is the analogy here? Microsoft wants to embrace and extend... compiling software? and will kill off... existing ways of compiling software? How will this work?
Microsoft is embracing free development tools that compete with purchased tools they sold. SourceSafe vs Git, Eclipse vs Visual Studio.. those are terrible examples but you get what I mean. So you make a free tier of VS, you buy git hub etc.
Extend.. you try to make your tools as ubiquitous as possible. You convince hobbyists and students that they should use your platform because it is more powerful /useful.. it will help you get a job later. This part of the strategy seems really hard because there are so many development tools that are open source, and lots of companies like IBM, Oracle, Google want to keep one of their own safely under their control.
Extinguish. You add features that only really work on your platform, or work better. Think about trying to develop for Android not on Android Studio. I haven't used VS for a Windows project in a long time but I assume it has similar advantages. Or Intel's compilers used to be the first to work with new instructions.
Ideally people would "have" to use your platform and you can start taxing them for that, either with fees or requirements that they implement your initiatives. Think about Google pushing Kotlin.. Android Studio kind of defaults to that now. Of how it supports Firebase, or defaults new projects in segments. I am not mad about that it or anything, it may just be a good idea. But it shows how if you control the dominant tech platform you can support your initiatives.
This isn't something that keeps me awake at night because a number of tech companies are working to promote their own semi-proprietary or open development platforms. But the fact that so many go to the trouble to do that shows it is a concern.
I understand how EEE is supposed to work. My claim is that it doesn't actually work, at least not for this use case.
So for my argument, SourceSafe vs. Git is a great example. :-) You simply can't keep up with a proprietary tool that does its own stuff. You have to use Git. Microsoft tried having a custom version of Git for Windows OS development, and decided that they weren't even able to get themselves to fall for the trap - they ended up getting involved in upstream Git development and turning all their extensions into standard features.
There are definitely cases where if you control the platform you can control what people are doing. I'm arguing that Microsoft has no actual effective control of the platform here.
EEE doesn't have to work to be considered anti-competitive. The intent is nefarious, it's trying to attract people by offering a veneer of good grace and good intentions, only to capitalize on it once you've attracted a strong enough user base that you can get away with closing things down, squeezing things in, and calling the shots.
So it be great if their proprietary remote extensions fail and they are forced to open-source them and make them a standard, but clearly the intent right now is that they don't fail and it gives them a leverage over their competitors.
They first embraced web standards by building a web based open source IDE called VSCode to get people to convert to it over using Visual Studio.
Now that they attracted people to use it, they embraced Linux containers and offered managed dev containers as a service over it called Codespaces running all on Azure.
At this point, it's all good, this is how a company profits on open source and standards.
But other competitors will not be allowed to use them to compete, and they can't be bundled in anything else, so Sublime wouldn't be able to bundle them in or make use of the remoting features.
The remote extensions are key to the attractiveness of VSCode for use with cloud managed containers, so it's a pretty big differentiator.
On top of that, they purchased GitHub, where most open source software lives, and are now trying to move development on those open source projects to adopt their own proprietary extensions to offer contributors a way to quickly contribute using VSCode and GitHub Codespaces.
Sounds like EEE to me. Maybe it will fail to extinguish, but my guess is we will see a lot of people slowly relying more and more on proprietary extensions owned by Microsoft to the point where if you work for certain companies you might not have a choice but to use VSCode.
I think the 3rd E is more likely "extort" these days. They won't extinguish it, but you'll be paying per user per month forever and the prices will always go up, up, up.
I'm not against the cloud, I use it in a daily basis. I don't mean you have to phisically own your servers. But it's not the same, to me, having your MySQL or Posgres DB up in the cloud, than using things like Firebase.
I just don't like the direction of moving the development environment to the cloud. It makes it easier to just consume third-party services and a) Don't understand how anything works, and b) Don't own anything.
If you know, fully aware, what the cloud implies, then sure, use it, but it's dangerous when the decision comes out of ignorance.
I guess there's not really much you can do about it though. People will do whatever is easier, and not everyone wants to know how things actually work.
I assume a huge part of why GitHub is okay with this is that VS Code / Codespaces and Azure are both products from the same company. :)
I do think there's a serious autonomy question here, and I think companies do actually care a lot about maintaining their autonomy from other companies, and in many cases the incentives align a lot. The whole stack here seems like something that you can run internally / self-host:
- Codespaces are specially-configured containers. All the standard infrastructure for running containers (Docker, Kubernetes, etc.) is FOSS and is quite reasonable to run internally. It's not even like OpenStack where it's a knockoff product of what the major cloud vendors run: it's literally what the cloud vendors run.
- Shallow clones, pre-built running containers, etc. are all deployment practices, not products.
- You can ssh into a codespace.
The big problem here in my eyes is that it's reliant on VS Code, whose remoting stuff is not FOSS (and, having tried it at my own workplace pointing it at our self-hosted Kubernetes, it's been a pain to get it to work well with our authentication setup, proxies, etc. without access to the code and we have a number of open bugs filed). I think the challenge is for a free-software IDE to adopt the VS Code model of the IDE running locally but executing code (including running the LSP backend) remotely.
And, in a sense, Emacs already supports this just fine with TRAMP. It's just that the experience of using Emacs and the experience of using any modern IDE (VS Code, any of the JetBrains IDEs, whatever) are very different... and none of them are FOSS.
If you have a FOSS IDE, I think you can get this whole setup working well in a way where you have autonomy over the setup and where you can understand the details just fine. And even if you're using VS Code, you can set all of this up in a way where you're not contacting any external services.
Do you have more details on this? I thought VC Code just uses the language server protocol for remote editing, which supposed to be an open standard. What stops say neovim adding full LSP support?
LSP is definitely an open standard and you can make that work just fine. What I mean is that the components of VS Code itself that do remote execution are not open source, so if it's VS Code that you're running, you're a bit at MS's mercy for how you make it work (my current issue is that it downloads a VS Code remote agent into the container, and that download doesn't work right inside our corporate network, and there's no way to modify that).
If you're not running VS Code, then this isn't relevant to you and I'd imagine you can get a good experience via Neovim using LSP + netrw.
The real problem that Codespace for stuff like the GitHub codebase solves, is not having to bring up the dev environment. Non trivial, organically certifiably grown commercial software has bunch of services that all eat different things in winter. And things like the Rails monolith that GitHub's main service is, have a tendency of being rude to a dev's laptop, squatting the place with whatever port and file they want laying around, installing packages and software to variously namespaced-or-not locations. If you deal with any set of service, they'll all have various hygiene standards and run well or not on your machine, depending on wether your machine looks like the core dev's machine. And then everyone famously wants a custom work machine with freedom to manage it however they want.
Codespace stuff is a way out of insanity. It's a mediated common ground. You could standardize on Docker or Vagrant or whatever, but they come with a lot of pitfalls and the local dev story with them all kind of suck. Codespace on the other hand is a solution that mostly doesn't suck. It's actually a pretty decent experience.
> And things like the Rails monolith that GitHub's main service is, have a tendency of being rude to a dev's laptop
FUD. Everything is neatly handled by rvm (or rbenv) and bundler. You can even use gemsets if you want to be really anal about it. It all goes in version-controlled user directories. I keep my main Mac machine, my work Windows machine, and the production Linux boxes all running the same versioned stack just fine.
What do you mean FUD? Maybe you've got your own way of doing things that works well so far for you with your current projects, but a ton of people don't live with the same experience, and when they do reach a comfortable spot, it's only comfortable until they move to a new set of projects or company.
I'm saying that, while a top-tier, software-as-a-service company might have a development environment that is hard to bootstrap -- because of all the ancillary things that run alongside the "monolith" -- your generalization that "Rails" makes a mess of a development machine is FUD. The Ruby/Rails part of this equation is not the issue. As always, YMMV and TACMA.
I didn't say Rails makes a mess of a dev machine. What I'm saying is that every project makes different sort of assumptions about how it's brought up in a dev environment, and these assumptions vary wildly between projects and stacks, and usually the older and more prominent a project is, odds are it will have the most unexpected impact on anyone's machine. This is going to be true for any combination of libraries, frameworks and programming languages.
I'm not mounting an attack on Rails. Rails' fine.
(Edit) I can't answer your reply. It seems there's many ways to read the quote you brought up. I'm clarifying that the meaning I intended wasn't an attack on Rails. Take it as you will. I'm a bit confused why my original comment, which I meant as a positive-to-neutral tone, is being perceived so negatively.
Sure. Until you get your account banned because you went againts of obscure TOS clause, or because a powerful entity put pressure or paid their way into it.
Can't wait for the equivalent of abusive dmca to be able to put down your entire dev team.
Doesn't docker solve all of this? It has for my group, even using C++, which is notoriously finicky. You can version control your docker file and bootstrap a dev environment in minutes.
> If they start off with things like that, there' so much magic under the hood, they won't understand how anything works. They don't understand they don't own shit until it's too late.
Bingo. The inevitable rug pull is going to be very, very expensive/catastrophic for a lot of people (though, profitable if you understand the underlying systems).
This may not be what you are referring too BUT I can say as a .Net developer it's amazing how little .Net devs know about how the low level dev related stuff works because visual studio holds there hand.
I have blown people's minds by showing them how to just call msbuild or git from the command line etc... I think it's changing since dotnet core took over in that space but had a good 10 year run where generally no one at various work places had any understanding of what visual studio was really doing and it did bite us in the ass at times.
Yes, IDEs are similar in a way that they abstract lot of stuff, so people end up depending on them. I don't mind IDEs, but I encourage learning what they actually do behind scenes, so you won't be completely useless when you don't have them.
I think that's one of the biggest issues I have with codespaces.
There’s two sides to that coin. In visual studio, most of the time you don’t need to bother to see what’s under the hood, and you can focus on feature development. In that way, their ignorance can be a good sign. The flip side is when your stuff won’t build, it can be an ordeal to figure out why, but thankfully it does not happen a lot, and they can leave it to the local experts.
Free software won, the commons ossified as no one cared to invest in fundamental ways, new layers are slapped on to hide the cruft, and the pendulum swings back to proprietary competition.
It's funny because we told people not to trust big companies, they tell us we are paranoid, they do it, they pay the price for it, then they say we were right, then they do it again.
MS changed. Google changed. Apple changed. Why do one expect github to never turn evil ? Haven't learned enough from history ?
Don't let them control your entire stack! Don't let anybody control the entire of anything.
There is a difference this time though. This time, most of us will say it only once.
Fight this bullshit. Call your pro-breakup / anti-trust representatives and give them the reasons this is bad for you and your career. Tell them the narrative and give them ammunition to fight this.
Google, Amazon, Apple, and now Microsoft deserve regulations and/or dissolution.
This isn't a black-box system, it's github. Owning your artifacts is as simple as setting up a job to automatically clone the repo a couple times a day (assuming your entire company is working through the cloud interface and not a single person already has a local checkout).
Lots of companies put their entire databases in AWS - owned by a company that's already evil, no speculation required - where the egress charges make offsite backups a challenge. And western civilization hasn't collapsed yet.
If GitHub changes its policies one day or shuts down the service... just push to a new repo and switch over to regular workstations.
> This isn't a black-box system, it's github. Owning your artifacts is as simple as setting up a job to automatically clone the repo a couple times a day (assuming your entire company is working through the cloud interface and not a single person already has a local checkout).
Isn't the use-case they're selling in the article approximately "the development environment setup is so fragile that it doesn't work on lots of machines?" I'd be willing to wager that after a few years of the whole team using this, the fragility will be much worse, so moving away will be that much harder.
Fragile in the sense that it might take you hours or days to get it all set up just right. It would be a disruption, like any IT problem, but it's not going to threaten an entire business.
I might be wrong, but it seems to be a combination of visual studio code + container plugin + docker. So you would need to install visual studio code, few plugins and spin up a docker to replicate that environment on the local machine.
People act in their own interests, usually prioritizing the medium-term (eg. evaluating every few years/every decade) over the long-term. These products made by the big companies provide immense value that promises to make your life better tomorrow, so most people choose them.
It's 100% economic. "Everything has to be free" means there is no money to be made in selling actual applications. That means the industry pivots to SaaS since the cloud is DRM and by running apps remotely you can charge rent and make piracy impossible. As a bonus you get the user's data, which you can do anything you want with in most cases.
As a bonus SaaS can also claim to be open source if only the client code is open, or if all the code is open but the SaaS still holds all the data and has the network effect. So you get those open source virtue points for free while still locking up the data or the network.
Of course the other response to "everything has to be free" is surveillance capitalism.
> If they start off with things like that, there' so much magic under the hood, they won't understand how anything works.
I feel like there's an ever growing tension in software development where magic like this makes us more productive but also more vulnerable.
Maybe we need to develop better benchmarks for "knowing enough" so when the magic fails, we're not starting at zero trying to figure out what went wrong.
Sometimes I amuse myself by thinking that fantasy games are actually a thinly disguised sci-fi commentary on the direction of modern programming. Take for example the Protoss race in Starcraft or the Sheika tribe in Breath of the Wild - the premise is basically that there's a highly advanced society using the remnants of amazing ancient technology that they themselves no longer understand.
People are fungible resources to be used when convenient and agreeable. Say or do the wrong thing, or have your utility dip below a threshold, and you can be removed.
My biggest fear is giving up control to corporations. It starts with the honeymoon phase and once they control the market... they can dictate who has access, pricing, government backdoors etc...
> The issue is not so much senior devs, but new devs. If they start off with things like that, there' so much magic under the hood, they won't understand how anything works. They don't understand they don't own shit until it's too late.
Any good CS/Engineering program should have the students build and use a dev environment they configured to their liking.
If the only dev environment used is wrapped into some IDE magic I would be very skeptical of the program.
I find RubyMine to be sluggish, and I stick with SublimeText and a set of terminals. However, I was trying to debug a plugin of a plugin of a gem, and installed it again to see if it would help. I found the problem in about 5 minutes because of this navigation. Credit where due; it was pretty awesome.
JetBrains IDEs can require some tweaking of the jvm startup params. Mostly initial and maximum heap size. The defaults are rather low and for projects with lots of dependencies it needs a lot of memory to keep indices and cache warm.
I usually run mine with at least 8GB max heap space (on a 32GB machine) and have a good experience with large Gradle projects.
Now it will be pretty easy to bring back those pesky remote workers to the offices. They will just tweak some code on the backend and poof, the latency of your home connection skyrockets to the point of typing becoming impossible. Luckily there won’t be such an issue with the office internet! I guess we’ll have to come back to the office to keep being productive in this web-based IDE :^)
I wonder why they're not telling the security & compliance side of the story here. Getting rid of local development means they have to worry a lot less about what's going on on their engineers' workstations. They've reduced them to dumb clients; any code going in or out of the repository has to be created, or at least pass through, a VM that GitHub controls. That lets them move the security boundary; I wouldn't be surprised if the next step is to cut off the ability to clone or push to certain sensitive repositories from outside of Codespaces.
That being said, while it has a lot of advantages from an enterprise perspective, I wouldn't want to live that way :)
> Getting rid of local development means they have to worry a lot less about what's going on on their engineers' workstations.
Wait, what?!?
No, it doesn't. Their engineers' workstation software can read or edit their code just as easily as their engineers can. Going into the cloud just adds another liability, it doesn't take any one away.
I don't think so. Software running on your computer has wholesale access to your filesystem.
If those files are accessible only via the web (or if you bother setting up ssh + pubkey, via ssh) that significantly reduces the surface area of possible attacks.
Don't hate on "not being impossible" for "being less likely."
"Less likely" applies to random events. Sabotaging or stealing code has nothing of random in it. You can claim it becomes "harder", what it does, a very small bit that doesn't warrant anything like the claims on the GP.
All it changes is that the attacker will have to launch the browser or rewrite some part of it (what he can do, because the browser auto-updates with your access level), instead of simply taking the files.
Oh, and by the way, not all software running on your machine has complete access to the filesystem. Reading your code and changing your browser requires basically the same level of access.
I stand behind the idea that forcing everything through SSH to a VM they control or over HTTPS to a web app they control has meaningful security advantages. Of course you can craft a hostile client for any service, but it adds a barrier (and a place to run countermeasures) that wasn't there before. It definitely lessens the likelihood of getting caught up by some dumb untargeted drive-by malware, at least.
Also, do keep in mind the target audience for Microsoft's enterprise products - if something lets you check off a bunch of boxes on some government or industry compliance checklist, then that thing has value to those customers, regardless of any actual security benefit.
The idea w/ the VM being that you could run all sorts of heavy anti-malware and pattern detection and file access analysis type of stuff ala CarbonBlack w/o having to manage deploying it to all your workstations and making sure it stays updated and functional there.
>"Less likely" applies to random events. Sabotaging or stealing code has nothing of random in it.
You have N machines, M developers, with access to the internet and development environment E. What are the chances of someone stealing your code given N, M, and E.
Changes to N, M, and E change your expectation on the frequency and severity of attacks.
Seems like the next step is to avoid downloading the full repository on every machine. If you had it stored on a few machines in the same DC (keep hot files in memory), you could just fetch things on demand with little latency penalty.
Though a lot of people seem to have issues with FUSE filesystems.
Imagine the amount of telemetry they will collect from their engineers! And of course the Big Data team will start working on identifying the “slackers” ;)
699 comments
[ 4.0 ms ] story [ 504 ms ] threadThen you can run `make world`. Of course, BSD make is not completely compatible with GNU make...
1. Ubuntu seems to have trouble with psycopg2 while Arch does not, so I often use psycopg2-bin just so Ubuntu developers can easily install the requirements (learned the hard way). 2. Some older (supported) editions of Ubuntu create extra lines in `pip freeze`, including `pkg-resource=0.0.0` which throws an error when installing on nearly any machine (including cloud versions of Ubuntu in Github Actions.
I have an application that uses docker-compose and has Elasticsearch as an image. In order to run it, I have to expand the vm_max_map (or something, I forget). But there are different ways of doing this in Windows vs Linux.
Differences between environments are legion. Unless a company wants to just send standardized laptops to every developer, it seems way better to just spin up a cloud vm that is already set. Plus, there's easier reproducibility in a cloud vm vs a physical machine (unless you want to reach for something like nixos, but hell, what company is equipped for that yet?
The difference between Debian and Ubuntu is enough to be a nuisance.
The Dell XPS laptop my employer provided cannot boot Debian and I'm stuck with Ubuntu. Same packages, maybe a version or two are more recent right? Nope. Aside from the egregious transparent snap installs when using apt (which can't uninstall snaps, thanks Canonical), there's a lot of minute details that amount to quite a time loss (that and having a laptop with no USB/RJ45/HDMI ports). Simple scripts that work on one OS and not the other, font scaling and rendering making my standard 1080p screen unreadable, absurd default configuration that makes WiFi unusable, etc.
Glad I don't have to manage the same oddities for the actual work I do. Everything is simple once it's all Dockerized.
But that's not true either. I had a different problem setting up the repository with every developer and they were all using the same OS. No repo in sources.list (how?), no docker group created when re/installing the docker package (why?), all kind of weird stuff happening.
So you tell me there's a way to have all devs working and testing on the same environment? That's great! But it is _not_ worth handing over everything to Microsoft or any other third party.
I'll never trust GitHub, GitLab, or any other forge with business-critial stuff. Host your own code, commit your dependencies, have your build script work locally. I've deployed through Git{Hub,Lab} outages and the leftpad fiasco, and the cost was insignificant. Host a mirror there for cheap if you want, that's what I do for my public projects (but after the copilot debacle how can I trust GitHub with anything?).
And what if you're a remote team? You can't rely on the internet. My ISP was kind enough to remind me of this when I got my very first outage with them one hour before starting at my current company.
As valuable a product GitHub is and Coderspace may be, it is a step closer to the Minitel 2.0 where everything is centralized and controlled by a single entity. It's not the internet, it's MSN at best.
IMO investing time to keep applications "normal" like this pays off in the long run. It keeps development closer to "greenfield", which causes a general multiplying effect. Tutorials are slightly more likely to work, 3rd party packages will be quicker to integrate, etc etc.
(Note that this is separate from dev/prod parity, which is also good.)
(1) On a team of developers, there usually isn't someone who is designated to be responsible for designing and maintaining a good dev environment.
(2) Many devs don't have the expertise to do it. Build tools, operating systems, dependencies, and configuration management are a related but different skillset than coding.
(3) Many devs don't want to work on it because they enjoy coding more. Just because you enjoy one type of work doesn't mean you enjoy the related work. (This also applies to writing documentation.)
(4) Fixing / improving the dev environment is not rewarded. You need to get X feature done by Y date, and fixing the dev environment may slow you down. It provides a long-term benefit, but not one that is easily measurable, plus that benefit is off the radar when evaluating the contributions of someone whose primary role is coding.
With this cloud environment, they've separated those responsibilities out and given them to a specialized team dedicated to it. IMHO, some of the success is probably due to the organizational structure, not just the technical differences.
While the USA is also litigation happy, it actually takes an incredibly high amount of deliberate destruction for an employer to succeed in suing an employee.
I don't think it's very likely you'd get sued, but I wouldn't want to risk it against well funded adversaries with expensive lawyers.
If you’re looking to connect via SSH, we do have a workaround here: https://github.com/microsoft/vscode-dev-containers/blob/main... -- and I use this regularly for Jupyter Notebooks.
At first I liked them because they have lots of great functionalities out of the box, but vscode has caught up and just yesterday I switched back to vscode from jetbrains. For java spring and react
It's pretty configurable. In my experience, it's pretty quick. It's not text editor or VS Code quick but the only thing that ever trips me up is the occasional re-indexing. Well worth it for how well it does everything else.
Code server (https://github.com/cdr/code-server) works in a similar way using your own PC as a host, but for whatever reason Codespaces is more performant.
Parts of gmail run in the browser, the rest runs on servers written in many different languages at Google.
The article describes using pretty much any IDE, so they are just offloading the setup and running of the development environment to the cloud while letting you run your preferred IDE locally.
People who try working offline are exactly the people who want to do it without Stackoverflow.
My company has a remote VSCode setup not that dissimilar from this setup. Big C++ codebase. Many engineers love it, even some of the "old timers." Our interns can be productive on day one and it brings a lot of productivity to not have to worry about caring for your dev environment.
But there are plenty of people who really just want to SSH into a beefy desktop (or directly edit on that desktop) and use their vim/emacs setups. These are the people who know how the whole build stack works, can debug the vscode/codespace magic, and can do amazing things not available in the VSCode plugin library.
So it's good to be able to do both. Allow power users to use their own tools, but don't require everyone to be a power user to sit down and write some business logic. Give nice rails to ride but don't require them.
I bet smaller/poorer companies may want programmers who are ready to fix tooling when it (inevitably) breaks
Additionally, it's not like developer set up is the only thing that test your ability to manage frustration. I would say developer set-up is just something you need to get through to start your actual job, and if that can be eased up then it's worth it imo.
For smaller/poorer companies, they could always hire an engineer that specializes in fixing broken tooling. Ultimately it's a company's job to hire competent engineers.
Thinking back to all the times I had to set up my workspace, I wonder how much did I really learn vs just Googling and getting quick fixes.
Google allows for normal desktop IDEs, plus having a web based one. The funny thing is, many people move to the web based one because it is so good.
But Google is also unique in how piper/citc[0] (our source control) works. It's effectively designed for web/cloud based style development, so the workflow for web based dev and desktop dev are effectively the same.
So web dev can work, but I don't think the existing tech that most of the software industry has is built well to support it.
[0] https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
There are some exceptions for some teams (like Chrome). And when I was there exceptions for people who did mobile (iOS/Android) development, though Google was moving pretty aggressively towards a world where even iOS code wasn't allowed on the employee's device.
The cloud environment builds and runs faster but the autocomplete, go-to-definition, etc. are pathetic.
It used to be entirely custom, but is currently being re-based on VSCode. Overall very similar feel to Codespaces, just tailed for how Google works.
There are other IDEs besides VS Code and shell based ones.
It can be a pain to set up, but is it more of a pain than setting up a dev environment the old fashioned way?
If you edit a file using some sync over ssh option, and want to run a test - how do you proceed? And how is this not the same thing as just checking out the git repo locally in the first place?
Well you have your files synced over ssh and also have a terminal over ssh which you use to run your tests. Am I missing something?
> And how is this not the same thing as just checking out the git repo locally in the first place?
It's not just avoiding cloning the git repo. It's avoiding building all the dependencies and getting them configured and running. For most of my projects that's not a big deal but I expect GitHub has several databases with specific configuration, memcached or redis, maybe custom patched builds, etc.
If you check out code in RubyMine, it'll try and install the dependencies in the Gemfile regardless, though.
If you’re looking to connect via SSH with your desktop IDE, we do have a workaround here: https://github.com/microsoft/vscode-dev-containers/blob/main... -- and I use this regularly for Jupyter Notebooks. :)
Jetbrains recently added remote capabilities to IntelliJ, I'm hoping it's going to be usable with codespaces soon.
Edit: Here is the CLI program that helped me set this up:
https://github.com/thoughtbot/rcm
https://docs.github.com/en/codespaces/customizing-your-codes...
I also have a very bespoke dotfiles config that predates Codespaces (https://github.com/wincent/wincent), so I made a thin wrapper around it that makes it work (https://github.com/wincent/dotfiles). For people with less complicated set-ups (ie. basically the entire universe), it is pretty straightforward.
Personally, I just `git init && git remote add ...` in my home directory and have a .gitignore that ignores everything by default, so I have to `git add -f` whenever I want to sync a config. Submodules work well for `.vim/bundle/the-plugin`. It's also nice for syncing important zsh customizations like this one: https://github.com/MatrixManAtYrService/home2/blob/master/.z...
There are also other strategies: https://dotfiles.github.io/
Seems it would solve all the issues, which are specifically that it mostly works only with VSCode or terminal apps, but with a remoting solution you could run anything you'd want.
I use Github Codespaces with a Vscode and vscode-neovim. Neovim is running locally.
I don't know how many people would want to use graphic applications over the network however, seems like it would be janky
VSCode editing over SSH works really well in contrast. I think Sublime has a plugin that ain’t bad. Emacs (TRAMP) is probably the worse of the three :/.
Terminal Vim or Emacs over Mosh is a pretty good option too.
Hear me out:
Toolings are, like the name suggests, a means to an end. If a web IDE is fast, works reasonably, and can continue improve itself. And the learning curve is friendly to engineers with different background and experience. Everyone should be comfortable to be nudged to use it. And if the tool additionally is a critical product of the organization, everyone should be comfortable to be mandated to use it, because it's now a very valuable source of dogfooding.
When I work for someone, I do the work they ask me and in return they pay me AND I get to learn something. For my learning process, I need the work part to be over fast. And that means I need to use stuff that I works best for me. I also need to be able to be able to fulfill my potential and spend time doing actual work (again, using tools that work for me). I don't want to spend a day setting up vscode and then realizing I can never edit as fast on it as vim what with the 300ms latency it adds to every keystroke
*it's always engineers making these statements or I only know engineers
"Tools are just means to an end" is just one of the many rhetorical devices that cushy managers use to eschew responsibility. Of course they're means to an end, but some means are better than others, and usually those with better means end up doing a better job.
Hiring is brutal, especially now, so good developers can afford to be picky and find another semi identical job in almost no time.
There should honestly be some good faith on both sides. It doesn't make sense for a job, in most cases, to be strict about an editor. But it's also not a good move for eng to be walking around talking about all the fragile things they'll quit abruptly for.
I feel like you're gradually making this hypothetical engineer seem more ridiculous. If someone quit over training or their unwillingness to try scrum, that would be bananas of course. There's an extremely large gap between that and mandating the main tool to use to do your job everyday.
That said, small things can look trivial from the outside and make your life a living hell at the same time.
In my experience (FAANG, startups in the valley and startups out of state/country), strong engineering talent is not particularly difficult to find, but finding a good fit is the tricky part. And I would wager that eagerness-to-rage-quit is a pretty good indicator of someone who will have trouble finding teams that they fit on.
A lot of people "rage-quit" basecamp over their no politics thing. I probably wouldn't have, but to each their own.
FWIW, I have also spent lots of time at FAANG, startups, etc and haven't had any trouble finding a great fit. I have also never looked more than a week or two for a job when I wanted to leave, so that does factor into my willingness to leave if I'm not enjoying my job.
If you look outside of FAANG (companies in the rest of the world with unattractive stock), finding good engineers is a problem and most codebases are filled with horror.
The plus side is that, once you're a good engineer, you get little stress/pressure and a lot of leverage on flexibility (eg. remote in cheap places, in pre covid times), even if you won't make as much as FANG engineers. Also, interviews don't require 2 months of preparation every time you want to jump ship.
I'm sure if you're paying FAANG money you can find plenty of good engineers happy to do backflips.
I am best placed to determine what tools I work with best to get work done. You wouldn't get an electrician in to do some work and then decide you can suddenly prescribe the tools they use.
I just don't have the patience for this sort of nonsense, there are a dozen clients who won't micromanage the operating system, editor and way of working that I'll be using to get the job done. With that said, obviously it's a two-way street and some willingness to be flexible needs to be shown on both sides. If $client wants me to sync my work with their $uniqueVersionControlSystem then fine, I'm willing to put in the extra effort to try and meet their specific needs within reason. But that doesn't extend to working inside a client-provided VM, using a client-prescribed OS image, using a client-specified IDE or making other such major changes to my established ways of doing business.
And in fact any client trying to micromanage this sort of stuff is a massive IR35 working practices red flag over here in the UK. You're better off terminating the agreement and signing something else as a SoftEng contractor.
No hard feelings or anything. I spend 5-8 hours a day working in my IDE. That's a huge chunk of my life and it matters a lot to me to work how I want. I've also found getting this aspect of my worklife right greatly impacts my overall productivity.
> Toolings are, like the name suggests, a means to an end.
It goes both ways. Employees are paid to accomplish an end, and I understand employees who are happy to walk away from an employer that micromanages the means to that end down to preferences in tooling. The reason you quit isn't because you miss Emacs (even if you do), the reason you quit is because your boss doesn't even trust you to choose a text editor.
That said, if the thing is your own product, yeah I get it: dogfooding is important. That said, I know of teams at Microsoft who do their (OS-agnostic) work on macOS... at some point accomplishing the end is more important than dogfooding, and you just let the workers use the tools that they're comfortable with.
I read that as you can ssh in and run a terminal-based editor, but that doesn't seem to offer much support for people looking to edit in NEdit or Intellij or Atom or Sublime or whatever else. Maybe some of those could be supported by forwarding an XSession out of the image, but that's going to suck hard for people on macOS whose editor is now the Linux/X version, with all of the attendant mismatch.
What do vim and emacs have to do with the shell?
This is another case of people thinking "shell" is synonymous with "TUI", which is false.
"Visual Studio Code is great. It’s the primary tool GitHub.com engineers use to interface with codespaces. But asking our Vim and Emacs users to commit to a graphical editor is less great. If Codespaces was our future, we had to bring everyone along.
Happily, we could support our shell-based colleagues through a simple update to our prebuilt image which initializes sshd with our GitHub public keys, opens port 22, and forwards the port out of the codespace.
From there, GitHub engineers can run Vim, Emacs, or even ed if they so desire."
I'm ready for the downvotes.
I'd love to use Codespaces or a self hosted alternative so that I don't need to carry around a heavy i7 laptop.
Being able to develop in the cloud from a browser opens up the possibility to develop from a thin Chromebook.
(Much better than eg PyCharm's attempt at remote development.)
https://marketplace.visualstudio.com/items?itemName=ms-vscod...
I wish this was even more modular so we could have a dumb compile farm, +local ide +run&debug on-device (I've recently got 90% closer by remote sshing with vs code to a pi vm)
env portability/consistency/using prebuilds as an example could have been done with in house CD/vagrant/packer ~10y ago. This post doesn't focus specifically on that, and there are additional benefits to the approach taken, but the notion that local dev is incompatible with these types of technologies is a consistent background theme in the writing
also, referring to emacs as non graphical and 'shell based' is a bit off to say the least.
The future is long term probably in the web, I'm just not sure we are there yet.
GitHub's approach seems utterly ridiculous, especially given that I've seen colleagues prevented from working because their Internet connection at some point ran through a part of the world that was sanctioned by Western governments.
Developer, it appears you are somewhere near Crimea! NO EDITOR FOR YOU!
The Visual Studio Code server isn't in Nixpkgs/NixOS yet, but there's an out-of-tree effort being maintained here for now: https://github.com/msteen/nixos-vscode-server
I don't actually doubt that this (and the 4 other glowing employee quotes) are real, but even assuming they are, I can understand people remaining skeptical about the sample size of 5 being broadly representative of the 100s (1000s?) of engineers at the company.
Also slightly hilarious that "This is the way" is a Star Wars callback referring to a character blindly following dogma before eventually realising the error of their ways...
On the other hand, there's quite a few "hints" littered throughout the article that their current setup is a bit of a beast, so I can definitely imagine some engineers being relieved to be able to get such a behemoth off their physical laps/desks: I guess if the local development experience is sufficiently brittle & frustrating, any alternative may seem welcome.
> Visual Studio Code is great. It’s the primary tool GitHub.com engineers use to interface with codespaces. But asking our Vim and Emacs users to commit to a graphical editor is less great. If Codespaces was our future, we had to bring everyone along.
snip
> From there, GitHub engineers can run Vim, Emacs, or even ed if they so desire.
I'm curious to know if this statement is true. Most experienced developers these days have a regular desk and a chair where they like to code and focus. How prevalent is the move-and-code scenario?
So I do see the appeal in thin client. My only issue is that the experience his closely tied to latency, so sometimes working over ssh is... irritating. Especially on mobile connection.
My usually setup is chrome dev tools and some node servers fired up, so quite easy to replicate.
But I still prefer to just move my laptop around.
I've been able to remove WSL from my PC: Docker desktop was gouging itself on resources. I can now also shut down my desktop and continue exactly where I was on my laptop, and visa-versa. I don't have to pull WIP commits back and forth between the two machines.
It's a major leap forward, and there is no need to use GitHub infra.
(Or who hosts it)
I have started to run some dev tasks on a remote Linux server, with Dockers, and then just configured automatic file sync on save over SSH.
Aside/HTH: The new Yarn has "Plug 'n Play"[1]. They claim that committing [their version of] node_modules is a reasonable thing to do, so that's one way to avoid install entirely. I don't trust that claim, yet (it smells like noisy diffs). You can instead re-enable the global cache[2], and then mirror your host cache directory into your container (whatever `yarn cache dir` gives you).
I have tried none of this myself because I'm not working on anything JS right now. pnpm[3] is what I've been using up til now, and is also a direct upgrade in terms of speed (but I suspect it won't work with mounting host directories).
[1]: https://classic.yarnpkg.com/en/docs/pnp/ [2]: https://yarnpkg.com/features/offline-cache#sharing-the-cache [3]: https://pnpm.io/
This only does code. You'd probably have to do those locally and SCP them over or, yeah, some mount magic.
> What happen when you need to work from a shitty internet network ? In train ?
You'd have to commit your work, but you'd be able to bring the environment up on your laptop. There are still merits to this, such as the repeatable nature of containers. Depending on where you've worked, you may have been through the hell of a huge list of setup steps when on-boarding a project: that's one thing that containers are really good at automating.
You may not be the target audience, though.
VS Code remotes allows you to drag and drop files from your desktop and they're placed where you want them in your remote environment. I don't understand why this would be any different.
The wrinkle is that the display server is running on your local machine, and connecting to the thread of execution on the remote server. Vim doesn't really have that concept of separating view from control, but tmux does play a somewhat similar roll.
A closer analogy would be using Plan9, and mounting most of your cloud machine locally. Vim would then just be your local front-end into those mount points.
Containers/docker add the ability to share your exact machine configuration with someone else, and have them create their own copy of it with a single command. It would be almost like sharing a VM image with your team. There's been a lot written about the pros/cons of containers, but they achieve similar goals to VMs, and so-happen to bring a lot to the table in this scenario.
I can't use docker in general because my internet is too crappy. That was annoying in 2012 when I first used it, but it's even worse now. I've always worked with VMs, since 2001 or earlier - virtualbox, wok, AWS, wok, and now proxmox.
It helps to know what your use case is, I don't run a ton of VMs, just one or two at a time that can use almost all of the resources of the server, and I generally do infrastructure and computing work in general.
I presume none of this counts as a homelab, though, since I don't program in rust.
You make an attractive case it for! :) Yeah, this is why I don't have a "home lab".
In my case, my homelab is my laptop (which I connect to from my desktop). I don't actually have a server blade lying around (although those can also be relatively cheap when big cloud upgrades their servers).
[1]: https://www.ebay.com/sch/i.html?_nkw=dell+optiplex+3040+micr... [2]: https://www.ebay.com/sch/i.html?_nkw=intel+nuc&_udhi=200&_ud...
Personally I use Gitea, which should be usable by someone used to Github. I don't switch workspaces too often, but I am sure some tools are available.
I have no idea what actually went into developing the feature, but I would have to suspect that Electron would have contributed to making the separation of an app client vs server easier than something totally "native".
[0]: https://news.ycombinator.com/item?id=28145556
With this setup, you can use whatever local tool you want and, as long as environment variables are respected, they’ll just work. The only requirement on your end is to either run nix-shell first, or to use direnv and ask it to load the nix env for you.
The downside is you still are compiling locally, so you still need sufficient CPU/memory resources, but that’s it.
Unfortunately most companies still aren’t paying any attention to Nix.
Yes, the parent comment looks a bit complicated, but that's just the one time setup, not something that needs to be done by every developer. And Codespaces will require some of that setup too; it's not like it'll magically know what packages and dependancies your project requires to build.
If not, then it misses the main benefit outlined in the article.
But yes, once a project commits to using CodeSpaces dev environment (even and especially ones with large code / heavyweight environments) there is a lot that can be done to optimise the spawn up time, that cannot be done as easily for local dev.
That sounds incredibly useful actually. Is that the main sales pitch for codespaces?
The only people who I don't think would instantly like this sort of thing are people like game developers who basically don't do any unit testing of their code and run the binaries they generate. For those users you could find quite a few ways around that.
[0] - https://www.quora.com/What-does-Googles-web-IDE-look-like
My friends, I’m here to tell you I was a Codespaces skeptic before order 66 and now I am not. Good engineers follow orders.
The nightmarish local development experience of either case are symptoms of deeper issues than local development itself being a problem.
We used to write COBOL using thin client on mainframes. Then we moved to development locally.
[1] https://github.com/svanderburg/disnix
[2] http://hydra.nixos.org/job/disnix/disnix-trunk/tarball/lates...
I've been using Sublime Text for a decade and it has kept the scope creep to a minimum. I've also used JetBrains IDEs and while there is a massive amount of complexity, it is well organized I think. I just get anxiety from launching VSCode, may be it is because of not being familiar with it, I don't know.
The configuration also syncs itself if you login with GitHub, so you're not going to keep having to do this or juggle files around.
In reality it's not that different from juggling around dotfiles and the various configurations that go into other software.
In VSCode (VSCodium to be more specific) I very rarely need to configure things.
From my doffiles git log - two config changes this year so far.
So by extension, codespaces is more risky than my current environment.
The question is, how much time do you spend on dev environment issues per year?
When I was at one of the FAANGs, our dev enviroment took about 1hr to install, and you had to redo it everytime you switched platform / version (about every week).
We spent ~50 hours per year managing our local dev enviroment. probably more in reality.
Definitely not little. But at the same time, I usually learn a lot about how stuff works during that. Which I like to think makes me better at helping other people debug weird shit.
Another fun one was when an org was rolling out CarbonBlack on all endpoints and decided to block all Java. Because apparently Java in the web browser is a security threat. They blocked 30 devs Java runtimes as a result. So there was an entire afternoon wasted for those guys.
To a corporation the risks of GitHub being down might be ok. Plus this will drastically increase on boarding devs to an existing project.
It's certainly reasonable to think that a SaaS can approach availability time of a single piece of hardware.
Furthermore, it appears that engineering github/github to work in codespaces helped their local provisioning as well.
> So by extension, codespaces is more risky than my current environment.
That only looks at the hardware. There are many more pieces to the puzzle.
This might be very resource intensive though.
Now githubs code is going to be developed on an online platform, which works by sharing code others have uploaded.
I have a feeling that this is not a good recipe.
Very curious to know more about this. What exactly are you talking about? Closest thing I could find is this[0], but you said "couple of years ago".
[0] https://www.zdnet.com/article/hacker-gains-access-to-a-small...
Another kid, got access to every single private github repo.
It was on hackernews, he declared it, made a write up about it and got given a pretty small bounty from github. Google it, otherwise I guess the net was scrubbed of it. I follow him on twitter but yeah just google more and you should find out all about it.
I was always surprised it wasn't a much bigger deal. Basically since then I think any company ( most that I work for ) are clinically insane to upload their proprietary code to the site.
On a related note - was it not just a couple of weeks ago people were concerned the codespaces system is using other peoples proprietary code? ( I havent kept up to that on that one so not sure if its still an issue)
Development environments being overly complex to setup, long compile times, those are symptoms of software bloat and bad design, but instead of addressing these fundamental problems, people want compilation to happen externally on a 32-core machine so they can sweep those problems under the rug. Okay, let's see how that turns out.
On GitHub, that instance type is $2.88/hour or $2,073 monthly per developer for a single instance.
(Granted, that's running 24/7 but still - wow, that's expensive for a single instance)
A powerful laptop is a drop in the ocean compared to how much cloud VMs cost per developer.
I guess lucky for them to be owned by Microsoft now and so that's just a cost of doing business. I have a hard time believing they could have even considered an approach like this if they were still paying for hardware out of pocket.
Kudos to the team for figuring it out though.
I do wonder how many VM's they have running though at different points of the day and how many spares they have prepared.
Companies typically buy laptops with an assumed 3 year lifespan; so, let's assume a high-end $3000 laptop, that would be ~$83/month. Of course, you need a computer to access Github Codespaces, so this isn't saving all of that money. Maybe companies can cut some costs by buying cheaper laptops?
Or if you want a more apples-to-apples; a Lenovo ThinkStation P620 runs ~$2800 for 32 threads and 64gb memory. DIY would also land somewhere around that $3K mark; a Threadripper 2950x (32 thread) is $1100; 64gb of memory is around $250.
This, of course, is the most expensive option they have; the more standard option would be 4/8; for which you would pay ~$85/mo (all the time, normal work hours), for the privilege of a machine that's far less powerful than any laptop any of our developers have. For god's sake, my M1 MacBook Air is an 8/16, with four of those cores far more powerful than any three year old Xeon Azure has running. And it was like $1400.
I understand maintaining dev machines isn't the easiest thing in the world. I have never once worked anywhere where it was such a problem that the company would justify tens of thousands of dollars in spend. Because it kind of is one-or-the-other; time invested in making Github Codespaces work for your company is time not invested in maintaining local dev environments. And Github's ideal end-state is companies totally rely on Codespaces, so the local dev environments languish.
Its not worth it.
The cost difference between laptop meant for remote development vs regular dev laptop is less than $1k. ($300 for CPU upgrade, $200 for RAM, $200 for SSD upgrade)?
If your dev costs you $100/h, you only have to save them +/- 5 hours of faffing around with branches/dependencies every month to make it worth it (assuming a 8 hour workday).
Big could, of course. But considering it takes new devs at my company ~3 business days to get up and running (and we're a React shop), it could be worthwhile.
Look, this is yet another product targeting naive VC funded startups. Milking the cows until they bleed.
Where we run into trouble is when we try to copy the shape of those big companies, and the kind of initiatives they fund, into startups, or mid-range companies. In quite a few ways, those top companies are better at development, but the base costs are only worthwhile if you have a money fountain of real revenue that grows far faster than your dev expenses. Trying to copy them with very different conditions is going to lead to tough problems, possibly company killing, and it's the kind of imitation we see all over the industry.
So yes, someone like Github is going to have eye popping expenses, because the alternatives to avoid those expenses just don't make sense to them. If you are not working in a multi billion dollar company, their practices can be interesting, but it makes as much sense to emulate them as it'd make for you to hire an entire Formula 1 pit crew to keep your commuter car in good shape.
Sure, you might not need to buy machines that are as beefy (although I bet you'll still need a decent machine to run all your chrome/electron windows) but enterprise maintenance on these machines is still a significant cost.
I definitely don't think one can look at this from the 'saving money' angle.
The issue is not so much senior devs, but new devs. If they start off with things like that, there' so much magic under the hood, they won't understand how anything works. They don't understand they don't own shit until it's too late.
I remember when I first saw the JAM stack, basically they use external services for *EVERYTHING*, claiming it's "The future™". They don't even own the data/database.
I'd rather make things better for users, rather than companies. Stallman's essay [The Right to Read](https://www.gnu.org/philosophy/right-to-read.html) gets more and more scary.
He/She was concerned (and with good reason) about the safety of data. Had to learn the bad way :(
It's just that like auth, logging, etc, CDNs have been neatly abstracted out and service-ified in such a way that it's often more economical to go third-party. But that choice is orthogonal to jamstack.
1. Development environments are running on Kubernetes, and VS Code is remoting into them. You don't have to pay a monthly subscription for that. VS Code remoting is free-as-in-beer and Kubernetes is free-as-in-speech.
https://code.visualstudio.com/docs/azure/kubernetes
2. The actual Kubernetes hosting is the product "GitHub Codespaces" running on Azure. GitHub certainly isn't paying a monthly fee to themselves for this; they are, technically, self-hosting it. I agree that you probably do want to have the option to run it yourself, but I think you do, given 1.
This will be coming to a cloud service near you
In some cities you do, and they're often controlled by organized crime.
(Whether this comment is a metaphor for SaaS is left as an exercise for the reader.)
(not entirely serious, but it has a very expensive monthly fee for what is in essence a better GC algorithm)
I don’t see why that would be a problem though? If I can pay a company to provide improved GC for my infra, where the cost savings made it beneficial, I would do it…
[1] https://superuser.com/a/1365224
I have seen multiple companies collapse because the proprietary Backend-as-a-Service they built their sand castle on top of pulled a vanishing act, however.
Which is to say, you wouldn't have heard of them, because they never got big enough to be known to anyone beyond a handful of clients and employees. I've personally been hired to try to rescue such a company, and was not successful.
Larger companies have also been known to buy out critical service vendors and in-house them when there was a risk that they might otherwise shutdown or pivot, too; that's less often an option fot smaller firms.
The objection here isn't to hosting per-se – most companies aren't in the business of maintaining PostgreSQL and should consider outsourcing that task early on. If your host of choice shuts down, that's a standard service, you can dump and restore your DB and move on. Annoying but NBD.
The danger with jamstack or BaaS or similar approaches is working yourself into a place where a proprietary service API is baked into your product as a critical, fundamental dependency; if and when the host shuts down, you're in extreme trouble. That's what I'd heavily caution against.
(There's an argument that when the proprietary service is run by an MS or AWS or Google "they won't go bankrupt" so there's no risk, but even then those companies shift priorities and shut down services not infrequently).
Applications using .NET Framework with WinForms are not just a dependency update and recompile away from being ported to .NET 6. WinForms got a whole new API and porting complex UI effectively boils down to a rewrite. At least if you can. Microsoft doesn't support third-party controls in the WinForms designer of Visual Studio for .NET 5+, so have fun trying out positions and compiling over and over.
> With operations in over 620 cities, it was paramount for us to identify a chat solution that would enable Uber employees to reliably communicate on desktop and mobile regardless of where they were in the world.
Did Uber really need their own chat solution? Are they not better served by a SaaS product, one of 5 that 90%+ of companies are using? Are they still using uChat today? (honest question, I don't know)
This is especially dangerous when talking about security sensitive features. It is highly inadvisable to roll your own crypto
Lot of companies shut down or fail for lots of reasons. I think a business should focus on what they do best and what's critical to their mission.
[0] https://eng.uber.com/uchat/
But at Uber's scale I can see the business need for a custom chat app. Most third party chat apps are bloated, both the cheap and expensive ones. Plus there's the need for custom integrations.
The nice thing about being a client is that they support all this stuff and it gets better over time. Also something like Slack is absurdly cheap compared to the usefulness and productivity it provides.
[0] https://en.wikipedia.org/wiki/Symphony_Communication
The implication that Symphony is a better solution for chatting than already exists / existed.
As a (non-trading) user of the platform, I strongly disagree.
I don't know what the traders think, but I'd hope there's a USP in there for them besides "it's cheaper than a Bloomberg seat."
Nowadays, there's a lot more support internally at Uber for buying off-the-shelf solutions instead of trying to reinvent wheels.
Sure, there's obviously a big difference between controlling the things that are critical to your business and wasting time outside your core competencies. And part of that difference is the risk profile: if your entire backend disappears, and takes the API your frontend is built around and all of the data with it, you're basically up shit creek. If your chat app of choice disappears, you can sign the company up for a different one tomorrow – it doesn't take 8 months of crunch to "port" your employees, and likely all of your critical business docs weren't in the form of chat history.
> Lot of companies shut down or fail for lots of reasons. I think a business should focus on what they do best and what's critical to their mission.
Agreed. But very few internet companies can claim that the data in their datastore and the entire mechanism by which their product accesses said data is not critical to their business, which is the distinguishing factor here.
And it isn't all about efficiency. Programming and CS is such a major part of my life, that I want to know the deeper parts, and I want others to understand it too. Programmers who do not seek that knowledge I unjustly judge for it.
I'm sure this has been said dozens of times throughout the years as we've built more and more tools of abstraction. As software gets more complicated, it's OK if not everyone fully understands every part of the stack. People specialize, and then become experts at their small scope.
Or even if they're not experts at their slice, not every company needs a 10x code wizard who understand the system up and down. Some companies just need someone lightly technical to mess with website templates or write simple scripts and services.
At this point Microsoft's strategy around GitHub, VS Code, Copilot, etc. should be pretty obvious, and we should all be running for the hills.
As someone not following this closely, what would that strategy be?
My understanding of EEE is that it involves taking some competitive product/standard, making a cool MS version of it, and then killing off the standard version - e.g. implementing a barely sufficient POSIX layer on Windows, getting customers who have "POSIX" as a purchasing requirement to switch, and then getting them onto the NT API.
Most of the examples on that wiki page have been unsuccessful (IE trying to EEE HTML, Outlook trying to EEE SMTP, etc.), because they weren't able to get enough people to rely on the extended version and stop using the standard thing.
What is the analogy here? Microsoft wants to embrace and extend... compiling software? and will kill off... existing ways of compiling software? How will this work?
VS Code is an amazing tool that's getting huge adoption because of how awesome it is, and how open source and community centric it is, etc. They've gotten a lot of mindshare and dev love, that's the embrace bit.
The next step is the set of closed source addons. Have you noticed that a lot of the new VS Code features are now in addons that are under a different license. This includes the new remote dev and python tooling. Still free to use and awesome tools, of course, but fully under MS exclusive and controlled. That's the extend.
I don't know what the longer-term plan is. I'm hoping that it won't lead to an extinguish. But if that's what they want, then they could e.g. cripple the open source version, moving all their dev effort into a closed-source, Azure/Github only web environment. Who knows?
Half the dev world is now used to VS code. Many junior developers have never used anything else. Whatever they do can have a big impact.
Microsoft is embracing free development tools that compete with purchased tools they sold. SourceSafe vs Git, Eclipse vs Visual Studio.. those are terrible examples but you get what I mean. So you make a free tier of VS, you buy git hub etc.
Extend.. you try to make your tools as ubiquitous as possible. You convince hobbyists and students that they should use your platform because it is more powerful /useful.. it will help you get a job later. This part of the strategy seems really hard because there are so many development tools that are open source, and lots of companies like IBM, Oracle, Google want to keep one of their own safely under their control.
Extinguish. You add features that only really work on your platform, or work better. Think about trying to develop for Android not on Android Studio. I haven't used VS for a Windows project in a long time but I assume it has similar advantages. Or Intel's compilers used to be the first to work with new instructions.
Ideally people would "have" to use your platform and you can start taxing them for that, either with fees or requirements that they implement your initiatives. Think about Google pushing Kotlin.. Android Studio kind of defaults to that now. Of how it supports Firebase, or defaults new projects in segments. I am not mad about that it or anything, it may just be a good idea. But it shows how if you control the dominant tech platform you can support your initiatives.
This isn't something that keeps me awake at night because a number of tech companies are working to promote their own semi-proprietary or open development platforms. But the fact that so many go to the trouble to do that shows it is a concern.
So for my argument, SourceSafe vs. Git is a great example. :-) You simply can't keep up with a proprietary tool that does its own stuff. You have to use Git. Microsoft tried having a custom version of Git for Windows OS development, and decided that they weren't even able to get themselves to fall for the trap - they ended up getting involved in upstream Git development and turning all their extensions into standard features.
There are definitely cases where if you control the platform you can control what people are doing. I'm arguing that Microsoft has no actual effective control of the platform here.
So it be great if their proprietary remote extensions fail and they are forced to open-source them and make them a standard, but clearly the intent right now is that they don't fail and it gives them a leverage over their competitors.
Now that they attracted people to use it, they embraced Linux containers and offered managed dev containers as a service over it called Codespaces running all on Azure.
At this point, it's all good, this is how a company profits on open source and standards.
But now they are extinguishing by adding proprietary remote extensions, see https://code.visualstudio.com/docs/remote/faq#_why-arent-the...
Those extensions being proprietary will lock you to VSCode and their own containers. As a user you might still use it for free, even though they may add future premium features see: https://code.visualstudio.com/docs/remote/faq#_will-you-char...
But other competitors will not be allowed to use them to compete, and they can't be bundled in anything else, so Sublime wouldn't be able to bundle them in or make use of the remoting features.
The remote extensions are key to the attractiveness of VSCode for use with cloud managed containers, so it's a pretty big differentiator.
On top of that, they purchased GitHub, where most open source software lives, and are now trying to move development on those open source projects to adopt their own proprietary extensions to offer contributors a way to quickly contribute using VSCode and GitHub Codespaces.
Sounds like EEE to me. Maybe it will fail to extinguish, but my guess is we will see a lot of people slowly relying more and more on proprietary extensions owned by Microsoft to the point where if you work for certain companies you might not have a choice but to use VSCode.
hmmmm.......
I just don't like the direction of moving the development environment to the cloud. It makes it easier to just consume third-party services and a) Don't understand how anything works, and b) Don't own anything.
If you know, fully aware, what the cloud implies, then sure, use it, but it's dangerous when the decision comes out of ignorance.
I guess there's not really much you can do about it though. People will do whatever is easier, and not everyone wants to know how things actually work.
I do think there's a serious autonomy question here, and I think companies do actually care a lot about maintaining their autonomy from other companies, and in many cases the incentives align a lot. The whole stack here seems like something that you can run internally / self-host:
- Codespaces are specially-configured containers. All the standard infrastructure for running containers (Docker, Kubernetes, etc.) is FOSS and is quite reasonable to run internally. It's not even like OpenStack where it's a knockoff product of what the major cloud vendors run: it's literally what the cloud vendors run.
- Shallow clones, pre-built running containers, etc. are all deployment practices, not products.
- You can ssh into a codespace.
The big problem here in my eyes is that it's reliant on VS Code, whose remoting stuff is not FOSS (and, having tried it at my own workplace pointing it at our self-hosted Kubernetes, it's been a pain to get it to work well with our authentication setup, proxies, etc. without access to the code and we have a number of open bugs filed). I think the challenge is for a free-software IDE to adopt the VS Code model of the IDE running locally but executing code (including running the LSP backend) remotely.
And, in a sense, Emacs already supports this just fine with TRAMP. It's just that the experience of using Emacs and the experience of using any modern IDE (VS Code, any of the JetBrains IDEs, whatever) are very different... and none of them are FOSS.
If you have a FOSS IDE, I think you can get this whole setup working well in a way where you have autonomy over the setup and where you can understand the details just fine. And even if you're using VS Code, you can set all of this up in a way where you're not contacting any external services.
Do you have more details on this? I thought VC Code just uses the language server protocol for remote editing, which supposed to be an open standard. What stops say neovim adding full LSP support?
edit:formatting
If you're not running VS Code, then this isn't relevant to you and I'd imagine you can get a good experience via Neovim using LSP + netrw.
Codespace stuff is a way out of insanity. It's a mediated common ground. You could standardize on Docker or Vagrant or whatever, but they come with a lot of pitfalls and the local dev story with them all kind of suck. Codespace on the other hand is a solution that mostly doesn't suck. It's actually a pretty decent experience.
FUD. Everything is neatly handled by rvm (or rbenv) and bundler. You can even use gemsets if you want to be really anal about it. It all goes in version-controlled user directories. I keep my main Mac machine, my work Windows machine, and the production Linux boxes all running the same versioned stack just fine.
I'm not mounting an attack on Rails. Rails' fine.
(Edit) I can't answer your reply. It seems there's many ways to read the quote you brought up. I'm clarifying that the meaning I intended wasn't an attack on Rails. Take it as you will. I'm a bit confused why my original comment, which I meant as a positive-to-neutral tone, is being perceived so negatively.
I literally quoted you saying that in my first post. But hey, what do I know?
Codespaces has the disadvantage of turning you into a renter.
Can't wait for the equivalent of abusive dmca to be able to put down your entire dev team.
It's really a way out of madness.
Bingo. The inevitable rug pull is going to be very, very expensive/catastrophic for a lot of people (though, profitable if you understand the underlying systems).
I have blown people's minds by showing them how to just call msbuild or git from the command line etc... I think it's changing since dotnet core took over in that space but had a good 10 year run where generally no one at various work places had any understanding of what visual studio was really doing and it did bite us in the ass at times.
I think that's one of the biggest issues I have with codespaces.
MS changed. Google changed. Apple changed. Why do one expect github to never turn evil ? Haven't learned enough from history ?
Don't let them control your entire stack! Don't let anybody control the entire of anything.
There is a difference this time though. This time, most of us will say it only once.
Fight this bullshit. Call your pro-breakup / anti-trust representatives and give them the reasons this is bad for you and your career. Tell them the narrative and give them ammunition to fight this.
Google, Amazon, Apple, and now Microsoft deserve regulations and/or dissolution.
This isn't a black-box system, it's github. Owning your artifacts is as simple as setting up a job to automatically clone the repo a couple times a day (assuming your entire company is working through the cloud interface and not a single person already has a local checkout).
Lots of companies put their entire databases in AWS - owned by a company that's already evil, no speculation required - where the egress charges make offsite backups a challenge. And western civilization hasn't collapsed yet.
If GitHub changes its policies one day or shuts down the service... just push to a new repo and switch over to regular workstations.
Isn't the use-case they're selling in the article approximately "the development environment setup is so fragile that it doesn't work on lots of machines?" I'd be willing to wager that after a few years of the whole team using this, the fragility will be much worse, so moving away will be that much harder.
What are you referring to?
It's 100% economic. "Everything has to be free" means there is no money to be made in selling actual applications. That means the industry pivots to SaaS since the cloud is DRM and by running apps remotely you can charge rent and make piracy impossible. As a bonus you get the user's data, which you can do anything you want with in most cases.
As a bonus SaaS can also claim to be open source if only the client code is open, or if all the code is open but the SaaS still holds all the data and has the network effect. So you get those open source virtue points for free while still locking up the data or the network.
Of course the other response to "everything has to be free" is surveillance capitalism.
I feel like there's an ever growing tension in software development where magic like this makes us more productive but also more vulnerable.
Maybe we need to develop better benchmarks for "knowing enough" so when the magic fails, we're not starting at zero trying to figure out what went wrong.
Why should you trust your computer?
Why should your career be safe?
People are fungible resources to be used when convenient and agreeable. Say or do the wrong thing, or have your utility dip below a threshold, and you can be removed.
It's happening.
There are open-source alternative(s) that support import of most things from github.
Any good CS/Engineering program should have the students build and use a dev environment they configured to their liking.
If the only dev environment used is wrapped into some IDE magic I would be very skeptical of the program.
I can navigate to any internal or dependency definition, out of the box.
But when I need to learn or explore a really large code base in Ruby, I haven't found anything better.
I usually run mine with at least 8GB max heap space (on a 32GB machine) and have a good experience with large Gradle projects.
Editing happens locally in the client (and is synced to the service in parallel), and so you shouldn’t notice any latency when typing.
That being said, while it has a lot of advantages from an enterprise perspective, I wouldn't want to live that way :)
Wait, what?!?
No, it doesn't. Their engineers' workstation software can read or edit their code just as easily as their engineers can. Going into the cloud just adds another liability, it doesn't take any one away.
If those files are accessible only via the web (or if you bother setting up ssh + pubkey, via ssh) that significantly reduces the surface area of possible attacks.
Don't hate on "not being impossible" for "being less likely."
All it changes is that the attacker will have to launch the browser or rewrite some part of it (what he can do, because the browser auto-updates with your access level), instead of simply taking the files.
Oh, and by the way, not all software running on your machine has complete access to the filesystem. Reading your code and changing your browser requires basically the same level of access.
Also, do keep in mind the target audience for Microsoft's enterprise products - if something lets you check off a bunch of boxes on some government or industry compliance checklist, then that thing has value to those customers, regardless of any actual security benefit.
You have N machines, M developers, with access to the internet and development environment E. What are the chances of someone stealing your code given N, M, and E.
Changes to N, M, and E change your expectation on the frequency and severity of attacks.
Though a lot of people seem to have issues with FUSE filesystems.