When I work on my own projects, I used to try to balance both worlds cuz I want these source code to be seen, but after some time observing GitHub insights on my repos I noticed barely any ppl even care to check out any sub directory like lib or src.. so I stopped caring about coding styles or formats, i do keep my repos zero warnings from linters though, for my own pleasure.
Then why build another incredibly slow dynamically typed monstrosity that breeds bugs and bad quality at industrial scale? Stop building JavaScript infrastructure.
C'mon it wasn't that bad. npm was a fun experiment on what could possibly go wrong with putting leftpad into a single package.
For me it's still by far the best runtime to do anything event-alike/http/json/web. It's quick to develop small things and quick to deploy (if you omit the whole transpilation shenanigans which of course help if you have dozens of engineers working together).
He took one of the fastest & most successful & invested in existing JS engines & bolted on a pretty decent standard library.
Great winning move, where a ton of the hard part of language optimization & growth is other people's problems, & Ryan could focus on making a great developer experience, which he and izs (npm author) did.
Reading your comment I can't help but think about that we're now starting to feed our layer lasagne that "mostly works" with code generated by a stochastic parrot which also only "mostly works" and turn the resulting products into some kind of fabric of the society. Ten years from now must be a fun time.
"When I started, all I had was swamp! Other kings said I was daft to build a castle on a swamp, but I built it all the same, just to show 'em! It sank into the swamp, so I built a second one. That sank into the swamp. I built a third one. It burned down, fell over, and then it sank into the swamp. But the fourth one stayed up! And that's what you're going to get, lad--the strongest castle on these islands!"
You laugh, but... AI only got real traction when we stopped teaching computers clever efficient ways to solve logic puzzles and started throwing massive amounts of data, memory, and compute at the problem.
I first encountered the term "programmer-archeologist" on an IRC server in the 2000s, since the server was named after a Vernor Vinge character.
>>> The Programmer-Archaeologist churns through this maddening nest of ancient languages and hidden/forgotten tools to repair existing programs or to find odd things that can be turned to unanticipated uses.
- lambda-the-ultimate.org
At the time, I was a freshman in college studying "Classics" - dead languages and history, like Latin and Ancient Greek.
I dropped out that same year, flabbergasted and excited that I could experience the same archeological thrill on a much shorter time scale. All I needed to learn was some system engineering...
So I did that. Been here ever since, loving every day that I got to make some beautiful jank soup of software work for another epoch.
P.S. I was on the IRC server trying to pirate some music and textbooks I couldn't afford to buy. Wound up gainfully employed throughout my 20s, able to retire in my 30s (programmer / data archeology pays well)
me trying to make sense of an ancient application built on a version of oracle so old I should have put its extended end of life date in school years ago and oh my god is that cobol wtf I thought they said this was fortran...
One was to dig through a system that was built a decade ago and modernize it. I just wrapped that up. The end result has eliminated multiple points of failure and unnecessary process, and has enabled my team to completely own the solution end to end.
The other task is to dig into and modernize a system that was originally started over 2 decades ago. I've got some other, minor tasks to get through this month and next and am excited to get onto this next task around August.
I'm ~20 years into this career and I am have more fun in the job today than I ever have before.
Ditto. Spaniard here, so we have volumes and literal tons of history. But GNU/Linux and using 15yo old software which had years of geekeery and culture such as Emacs in y2k was magical. Code, math puzzles, a chatbot, fortune files, games like nethack (playing slashem and Tome2 was more fun than reading LOTR at 16)...
It was live history working in my hands and in a usable way after Euclid and Newton.
No need to buy hardware and materials to do technical stuff, a computer was enough. And not even a good one. My cwm+tmux setup would still work in a Pentium 2.
They are. Patterson & Hennessy at their Turing Award lecture back in 2017, when they started talking about where future opportunities for increased speed were going to come from, the very first thing they took a (polite) dump on was how software was written very inefficiently: https://www.youtube.com/watch?v=3LVeEjsn8Ts&t=2183s
The old canard that programmer time was worth more than processing time that bad coders quote is less and less true by the year; SWEs who are able to wring better performance out of less hardware resources are going to become more valuable in the future.
> SWEs who are able to wring better performance out of less hardware resources are going to become more valuable in the future.
As much as I'd like for this prediction to come true, I think it's wishful thinking. Compute capacity continues to increase exponentially while (human) cognitive resources, for all practical purposes, do not, which means less efficient code will continue to be the more economically viable option as far as I can tell.
Even with AI, where we're essentially trading compute (used for running code) for extended cognitive resources (used for writing code), what incentive exists to channel those resources into writing faster code rather than writing the same code faster?
It’s a literal waste of energy. LLMs draw a tremendous amount of power. They enable problems to be solved in a low effort, careless sort of way, but this neglects the power efficiency of our own brains. We can think through and implement better solutions consuming only a tiny fraction of the power. In a world where we’re running up against climate change and finite resource constraints, that is valuable, it’s just not self-evident yet. We’ll first have to crash headlong into a wall to realize it.
We can use AI to help us design and build a functional fusion plant, which we can then use to power more datacenters running ever bigger LLMs to help us design more efficient datacenters to run bigger LLMs to help us build ... yeah this is going to be one hell of a century.
> I can only imagine the people working on the actual chips are screaming in silence somewhere at the complete waste of energy we've made.
As a dev, this bothers me a great deal. We have miracle machines on our desks and in our pockets, and 80% of the capabilities of that hardware are pissed away by terribly inefficient software.
I find it impossible to relate to this stance because no software I use works. Period. I haven't encountered a single piece of software that's not frustrating in the last decade. Tell me all about the archeology and cool shit but this stuff boils my blood at this point.
This is why I use Apple products now. I don’t care that it’s expensive, I don’t care that it’s locked down. I’m just relieved and somewhat delighted that everything just works. I still have Linux for my gaming desktop and it’s a constant reminder of how good I have it when I have to do things like restart pipe wire to change audio outputs, or having new kernels break stuff.
Some of the experiences are downright magical like being able to copy something from my MacBook and paste it on my phone.
Just to rain on your parade, my mac stops working after resuming from sleep and having a monitor connected (audio plays, nothing is movable or actionable). It also randomly sticks the launchpad icons permanently on screen that won't go away without a reboot.
The ctrl key randomly stops working, tested by pressing ctrl-C at random intervals. Sometimes it's perfect, sometimes not.
Here's a site of one person's experiences of things that have gone wrong with Apple software: https://annoying.technology/ it's been goint for multiple years now.
Software is just buggy in general. I mostly agree with the parent comment. The amount of software that I've never had a negative experience with is close to zero.
(GP above here) I find Apple software broken all the time too, except I can't fix it like Linux. I have to use OSX for work because my employer requires a MacBook and it's buggy as hell. I use Thinkpad/Linux at home because thanks but no thanks.
That reasoning is fine in a world of video games and music player software perhaps, but not when it comes to running mission critical systems like flight control on a 747 or 10,000 autonomous cars carrying passengers.
I worry about complacency with overcomplexity and over-reliance on software, and our fast-growing trigger finger to automate everything....
Personal responsibility and training backed by a human touch will not be beat by code any time soon in my opinion. Citing that code is almost always deeply influenced if not forged and maintained by human hands, any credible singularity is far from our currently over-marketed Ai reality at the moment in my opinion as well... Google can't even get it's assistant and spell check to work properly, and they've been one of the largest companies on the block for decades now.
There is no easy way to release the hand brake in the new BMW 1 series without starting the engine. They had a pin to pull in the trunk and older cars had a tool for this, but mine does not, and I could not find the pin in the dark of the night either.
> build on top of a unnecessary complicated foundation
Is there a hidden assumption in "unnecessary"? None of the tech unfolded in a deliberate, methodical manner. It was random people doing various things over time and a plethora of details.
To assume that the aggregate would have a tidy structure seems unrealistic.
I mean, there is copious engineering effort internal to, e.g. the Linux kernel.
And much goes into RFCs.
But is there much internet-wide engineering going on to squeeze out the entropy?
Repackaged: the engineering that occurs is more bottom-up than top-down. Google, for example, has enough heft to say: "QUIC: here is a bit of top-down engineering." But that example bolsters my general point, in my opinion.
I don’t think all that many of it is unnecessarily complicated — we quite literally has no other ways of managing all that complexity. Mathematics is the same way — complexity is infinite, and abstraction is the only thing that can make it remotely tameable.
The fact that I can pipe text into a network connection, and it just arrives at the other end, in perfect order, secured over the wire, and with nothing more than the network stack that’s implemented in every serious programmable environment imaginable, is still insane to me. I don’t even care that it’s mostly 40 year old technology technically.
It would be nice if all software were simple, elegant, and minimal. But I don’t know why we should expect it to be. The hardware that runs the software is complicated on a whole other level. You want and simple, elegant and minimal cpu? Video card? Sound card? You better be ready to give up a LOT to get that. And giving up the current experience is something end users really don’t want to do.
What absolutely shatters us is that we almost never have enduring & exposed bases. Software is endless layers of abstraction reaching upwards. Very few layers expose themselves at all. So we are all strangers to most software.
For software to be simple elegant & minimal, we would have to be able to deploy existing knowledge, be able to leverage the fundamentals we know. But that's not where we are, but we are all adrift:
> In the past year I think I have finally come to understand the ideals of Unix: file descriptors and processes orchestrated with C. It's a beautiful idea. This is not however what we interact with.
Back in the 1990's there were maybe 5 languages commonly used (C/C++/Assembler/Turbo Pascal/Visual Basic).
Now there's 40. Each one needs it's own networking library. 8 different kinds of databases - with each language needing it's own adapter. 5 different web frameworks for each one. 3 different json libaries.
It's a classic cartesian product. And instead of "I'll help fill in the gaps", its "I'll solve the problem by writing a new language!" -- which just makes the cartesian product bigger.
And that's fine. Choice is good. IMO the problem isn't that there are a lot of things to choose from, it's that there are a lot of things on top of each other. Stuff on top of stuff on top of stuff.
I personally wouldn't want to live in a world in which I could only use "one of these 5 languages", whatever those are. But I would love to live in a world where things weren't so deep in abstraction and complexity. These are two very different problems.
Imagine if every state had their own completely different traffic rules, and you had to be licensed in every state in order to drive there. E.g. In one state you had to drive on the left hand side but only on Tuesdays, Thursdays and Saturdays. In another state, it cost too much to put up traffic signs and signals, so, all drivers had to stop at all intersections unless you were going in a North or South direction.
One aspect is that it would make an interesting situation at the borders between states. And that's kinda sorta what we have now with programming languages.
> Stuff on top of stuff on top of stuff.
Couldn't that also be too many choices? Like if you specialize in JS instead of C, you're not going to have all the knowledge a C expert might know. Instead of utilizing proven, existing C libraries, you're going to write stuff from scratch in JS?
The difference is that states require you to follow the rules, but you can choose what language you want to use yourself. I don't need to know Java if I want to use Lisp, and you don't need to know Lisp if you want to use Java. Yes, stuff will need to be rewritten. If you want choice you trade efficiency for it. I'm okay with that.
If everyone uses the "proven" libraries and languages and whatever we'll have no innovation. I don't want to live in a rigid world where we somehow ended up choosing a set of things and nobody uses anything else; I want to see more software written in weird languages. Arguments that go like "but nobody is gonna learn a whole language to contribute to your stuff" are depressing - if it's good, I, like many others, would. Without experiments like these we'd still be in the 60s using COBOL because it's "industry proven" or "battle tested" or "blazingly fast" or whatever. That world is boring.
All software, frameworks, etc is a reflection of the creature that created it: a Human, with all our faults, affectations, and prejudices.
A lucky few have a come to Jesus moment and realize that nothing is perfect and that fighting the tide by swimming upstream is a fool's errand, they swim sideways to the shore.
I disagree, humans can create amazingly beautiful and simple things.
All software reflects the environment in which it exists, including the time pressures involved in its creation and the complex, messy reality it is trying to model.
We’re all familiar with a “90% MVP” that is small and elegant and solves the main use case sort of ok. Beyond that, reality has a lot of detail and edge cases and that’s when software gets nasty.
> The only thing that matters in software is the experience of the user.
I mean, sometimes software development is enjoyable as an end in itself, otherwise Shenzhen I/O wouldn't be a thing.
That said, it's a lot more enjoyable when the blocks you're building with are nice self-contained things that work well together like LEGO bricks. As opposed to a bunch of double-clawed hammers that are all invisibly connected with strings under the floor so that when you try to pick one up your drill automagically turns on as it falls off the shelf (I am borrowing some PHP-fractal-of-bad-design metaphor but it applies just as well to Spring Boot or whatever-the-f&^& somebody with too much time on their hands thinks is fun to inflict on me).
Abstractions are great so long as they don't leak. But software development seems to have become the art of not going insane while trying to maintain castles in the sky that are oozing spaghetti sauce from every crack.
So anyway, I agree with the author, but maybe for slightly different reasons.
I urge the author to never look into building a physical building. The codes and amount of work you have to do to legally start are already complicated.
But then you start buying screws and realize the entire screw isle at building supply store is bigger than many small grocers. And probably isn't complete. Heaven help you if you want to do something aesthetically pleasing like a pocket door.
Or, far worse, heaven help you if you want to fix a pocket door. Finding a compatible wheel mechanism is usually most easily done with, "rip it out and use a new mechanism."
Then there is car maintenance. Check engine light says something about the eco subsystem isn't working right? Here is a book mechanics use that list the common reasons for the code. Not standard across makes. Or models, even...
I could probably go on. Point is nothing is greenfield lacking in complexity. I agree we probably could do better with software. But having done embedded work, I think it is easy to be ignorant of exactly how much has to coordinate in a computer system.
Hey, I just fixed that one. Turned out to be a mystery evap hose under the rear seat with no visible signs of wear or failure.
Was it what any of the mechanics I talked to thought it was? Nope! Did anyone carry the part, or was anyone even able to identify the correct part number, even with a diagram? Nope! Did I have to hunt all over for some marine fuel line that happened to have the same inside diameter so that I could jury-rig a new part? Yep!
Diagnosis and repair actually felt a bit like working with third party libraries: all the documentation is wrong, the problem is never your first guess, and you end up rolling your own solution in the end.
At least you can look at, touch and feel; Weigh and stress test the screw pretty simply. Software has edge cases which spring forth at the worst times and updates without your control at all.
not really. Choose a wrong screw and it will look great.. until a year later, an ugly rust stain appears.
And the pipes/hoses! Having your software update cause your server to fail is _absolute joy_ compared to discovering that your improperly rated pipe burst and is how dumping gallons of water on the ground.
After years of salt the screw is now rusted stuck. Oops i just stripped the head. Oops the head just cracked off. Theres tools for this but it requires experience too
There's a pretentious phrase for the phenomenon: déformation professionelle.
This is the idea that people who have learned one thing discount the knowledge of experts in other domains.
Nothing can be as fantastically complicated as my own work, surely! Building a house is just nailing a few sticks together. Fixing a car is just undoing a bolt with a spanner. Running a farm is just driving around on a tractor.
> The only thing that matters in software is the experience of the user.
I have one disagreement with this... There is another thing that matters in software --- the experience of the developer making it.
IMHO, I agree that libraries, languages, tooling,... etc. are not art whose complexity should be admired. But they are a necessary intermediate tool for _developers_ to make their work easier.
We can apply the same logic to something like construction. You can build a house with hand tools and man power, just like you can write code in machine language. It's not enjoyable but you can do it. The work around developer tooling is like a builder working with scaffolding, an excavator, a crane, an electric drill... You spend effort to set it up, and before the final product is delivered, you have to tear it down. But it was still worth it because it serviced the developer in some transient way.
The software serves the business' bottom line period.
The user only matters in so far as the keep coming back and paying money (or attention, which is also money).
Developer experience only matters in so far as it keeps costs low, creating better margins.
If that sounds kind of gross to hear, it's because it is. Businesses are made up of people. Customers are people. Every interaction on both sides of the equation should take that into account. If you ruthlessly maximize like the human part doesn't exist, your optimization strategy will fail.
"The software serves the business' bottom line period."
Making big assumptions about what the software project here.
Given the article was mostly complaining about software developed for developers by developers, I don't think we can blame the suits for this one.
Sure but it's a bit misleadingly reductive to both say that "only the end product matters, intermediate steps should just keep adapting to be the most optimal in producing that product" while also complaining about the exact artifacts that are resulting from that adaptation of intermediate steps.
Tooling does not get complex because engineers are somehow magically attracted to complexity. They get complex because the world is complex. To "hate [the complexity of] the tooling" is really just indirectly saying you hate the complexity of the world... which, I mean, I guess is fair, but not that interesting.
The problem with that argument is the "developer experience" is also awful for a lot of "modern"[1] software. The JS ecosystem is notorious for this, especially huge dependency trees of tiny fragments which no one really understands, tools on top of tools, etc. A lot of that complexity isn't actually necessary.
[1] Whenever I hear that word, I think "is that the best adjective you can come up with to convince people to use it?" because it often has no other advantages.
For sure, but things are still evolving and evolution is messy.
FWIW, I'd probably be considered a JavaScript luddite these days lol. When I build my own projects, I write vanilla ES5 (yes, 5). No tooling. No frameworks. No compiler. Just edit and hit refresh.
But when I work with others I will respect their choice of framework. Because the goal of frameworks and libraries and encapsulation in general is to trade-off extra individual work in exchange for smoother between-developer "compatibility" (reduced communication friction). I don't need to understand someone else's code, and neither do they need to for mine.
> A lot of that complexity isn't actually necessary.
I don't understand this take. Client-side JavaScript is hard because of cross-browser / cross-platform / cross-device challenges. Server-side JavaScript is hard because of the nature of the language and the runtimes, the lack of standard library and the myriad attempts to "fix" it, ranging from inventing new languages to transpilers to runtime rewrites.
May be you mean to say, a lot of complexity could have been avoided? Unfortunately, that ship has sailed.
Client-side JavaScript is hard because of cross-browser / cross-platform / cross-device challenges.
Is it really? Or is it hard because people are under the impression that it's hard and thus jump into the quagmire of others "selling" grossly overcomplicated "solutions" hoping it'll be easier (when in fact it isn't)?
When I can replace something that a team of about a dozen needed several months to create using a few MB of frameworks that work only in the latest version of Chrome and somehow needs multiple seconds to load over a gigabit LAN (internal tool) with an HTML form and a few lines of JS written in an afternoon that will work in any browser newer than ~IE5 (and will mostly work even without JS at all), I don't think that's the answer.
I don't understand this take.
As the saying goes, "It is hard to get a man to understand something when his salary depends on him not understanding it."
I think that's ultimately what it comes down to: Developers justifying their existence. They're all enamored with the new and shiny, resume-padding crap that they forget the problem they were hired to solve and instead create more problems that they then solve to show how "smart" they are.
...then they all get thrown out when the company downsizes. That's why I ended up having to do some JS work. I don't normally work with JS but I can use it if I need to.
I never had a need to do server-side JS so I won't comment on that in specifics, but I suspect it's much of the same situation.
I think both of you recognize a noble & true goal of optimizing & empowering people.
I somewhat think there's a huge error here though, a massive mistake in computing that we have such a huge divide between user and developer. A couple small vim tricks or enough shell knowledge to be a little dangerous, or even someone ok at excel macros blends the line so much here.
We have been building more and more totalistic apps, which afford less & less view of computing & developer-ish sensibilities. It keeps getting harder to see because we are so entrenched in pro application anti computing paradigms, but I think some day consumeristic computing will be worn thin. I New-old dynamics can re-start, embracing systems that also try to offer more developer-like sensibilities & capabilities.
Douglas Engelbart's idea of augmenting intellect to me speaks of the need for people to get great at toolbending, I at taking some good existing tools & capabilities, and being greatly empowered to bend create & explore different configurations & possibilities. Soft systems, with malleability, where users can ad-hoc cross over the aisle & become developers: that's a strong culture, where softwares real hidden noble purpose lies.
And where we can stop hating, stop being adrift & trapped, amid unyielding prisons of hard computing.
Writing software - is an act of self-expression like it is for an artist or writer.
Everybody wants to write his/her masterpiece in his/her own way, and not always agree with the way others do it.
same with scientific community - two scientists cant agree on two ways to conduct scientific experiment, because they want to express themselves in this act
Actually, some people want to write his/her own masterpiece in their own way.
Others just want to write literally anything to get the job done and be on with it.
Others resent the fact they're being forced to write it in the first place.
There are likely many other archetypes I'm ignoring too. But even reconciling these 3 together is what leads to everything regressing to the mean amount of suckiness.
> Others just want to write literally anything to get the job done and be on with it.
this is not compatible in team-based software development. Once you want to collaborate with others, the #1 priority becomes not "anything that gets job done", but "anything that team can read, maintain, support, and continue developing"
if you are solo dev or writing bash scripts for your own workflow - then of course you can use anything literally, as long as it is on your computer and nobody sees it
We rarely replace the foundations, we just paper over them. I don't think this actually matters, except when it breaks? I'm all for building a clean, simple system, but for me Unix would actually be the main part I would rip out (my preferred architecture would look something like unikernels on the server and a qubes-style cluster-of-communicating-unikernels on the desktop). If you're against adding complexity and memorizing trivia, Unix and C create more of those than the rest of the stack put together.
> qubes-style cluster-of-communicating-unikernels on the desktop
Oh hey, yeah. A bunch of small utilities that communicate and interoperate. I can see it working out. You could even have them communicate with each other via some sort of universal interface.
> Oh hey, yeah. A bunch of small utilities that communicate and interoperate. I can see it working out. You could even have them communicate with each other via some sort of universal interface.
The key is to have that interface be narrow and well defined - something like gRPC. Allowing them to communicate via a huge, poorly specified, non-concurrency-safe swathe of shared state (filesystems with all their ad-hoc semantics, shared memory, signals, pipes, KAME sockets, namespaces, goodness knows what else two processes on a Unix system can do to fuck with each other) is a recipe for disaster.
> The key is to have that interface be narrow and well defined - something like gRPC.
You really can't use a mesh of unikernels for this then. You need an external system to enforce the types (like a compiler does for a single program).
At runtime, so far as the computer cares, there's really just arrays of bytes (or machine words depending on if your system is really byte-addressable or just faking it).
You can do input validation, but unix cli utilities can already do that. You'd need, rather than a bunch of unikernel utilities, an all encompassing environment that the utilities can register their constraints with, and the system would prevent them from being called if the constraints were not met.
Main contender is probably emacs, just because it's already ahead on the "all encompassing environment" part of this.
> At runtime, so far as the computer cares, there's really just arrays of bytes (or machine words depending on if your system is really byte-addressable or just faking it).
The problem isn't arrays of bytes. The problem is all the other semi-structured cruft that the OS has accumulated, the squillion different system calls you can make with almost-but-not-quite standardised effects.
> You can do input validation, but unix cli utilities can already do that. You'd need, rather than a bunch of unikernel utilities, an all encompassing environment that the utilities can register their constraints with, and the system would prevent them from being called if the constraints were not met.
Or you just don't give them the interfaces to mess with each other's internals. You have your hypervisor isolate them from each other except for a narrow communications channel obtainable in a narrowly specified way. Ideally the hypervisor also enforces that what you send/receive is well-formed gRPC (which is not hard), but that's not essential - the main benefit comes from just closing off all the other random OS-level ways that Unix processes communicate with each other.
Unfortunately, still in 2023, one's OS is still largely dependent upon C (and assembly for certain optimizations). Rust is dipping it's toe into the OS world yet, but isn't fully mainstream.
Also a lot of that C code in Unix, you know, actually like works and has worked for a long time! 50 year old C code in a Unix operating system isn't going to be unheard of here shortly.
> Unfortunately, still in 2023, one's OS is still largely dependent upon C (and assembly for certain optimizations).
We mostly don't actually use the OS though. We write code that gets deployed on some managed cloud system, and to the extent that there's anything in between (a unix process, kernel and virtual machine to run the code) we probably cargo cult it and don't worry about the details unless it breaks. Building unikernels (which is increasingly practical, with OCaml leading the way but other options available) would sidestep many of those layers of junk.
("Serverless" is another way to achieve the same thing, and may well be the one to succeed; even if serverless systems don't currently use unikernels, they could be transparently upgraded to them in the future, if they're strict enough about the interfaces they offer at the moment).
The other times this appeared on HN are informative.
But to be the devils advocate here, the unix abstraction is only beautiful if your living in a single process, single user, teletype environment. Add multiple users, network programming, multiple processes/threads, async IO, etc, etc, etc. and the whole thing starts to look like the layered monstrosity full of edge states, and broken promises the author is arguing against.
Although frankly, there aren't many better choices. NT being nearly 3 decades newer and with the forethought of VMS is actually much better at a systems programming level but you have to be willing to dedicate significantly more effort upfront to understand the more complex task/io/user/permissions model. In the end there is probably less of a learning curve on windows, but its steeper because you end up having to learn how to do all this stuff on the various *nix clones as well, just using bolt on interfaces that frequently vary slightly between the different flavors.
In a way this applies to much of the windows ecosystem now that vb6/delphi aren't popular and the similar attempts at web co-design utilities have largely died.
I suspect at least part of it is due to the dogmatic worshipping of abstraction and the period in the late 80s-early 2000s where performance was still growing exponentially. Then there's the common notion that "more is better" when it comes to complexity, and along with that the idea that "growth" is somehow a good thing. The vast majority of software simply does not need constant growth nor change.
Look at the demoscene for an example of the exact opposite culture.
You only have to configure this once. Don't install too many things. Just what you need. This level of complexity is needed because this is not TempleOS in which all software is made by one person. Most software exists to bridge the Tower of Babel lost in translation scenario.
Suckless.org is merely a subjective catalog. Not a bad one, but for a tiling Window Manager, which I have to recompile every time I want to change its settings and need to somewhat know C for that - because what if I don't - that hardly sucks less.
NixOS is also a subjective take on packaging, heavily luring people into a functional programming world, where one set of problems is merely replaced by another set of problems. I tried NixOS, so no, thank you. There are better ways to isolate packages and have multiple versions of libraries. The simplest of which that come to mind are things like Flatpak and containers. There's almost 0 usecase for NixOS - it merely replaces one set of problems with another. And forces you to learn something, that does not justify the time it'd take to learn it. I feel like people who built it just wanted to be smart, not create things that are seemingly simple and beautiful (which is what true artists strive for, I think). I learned Vim in two weeks. I gave NixOS the same time and it was almost a waste of it. Not a complete waste, however, because, I recall, I discovered an extremely useful, but unrelated thing (zfs), but I don't remember how come it worked out this way.
I can’t help but wonder if right now, we are unwittingly building software in such a way, that in 100 years from now, our decisions will result in some kind of crisis, that the generations to come will wonder why we didn’t take more care with what we were doing.
Operating systems are great abstractions. If the author ever wanted to build an actual application on top of kernel APIs they would appreciate things like boost or libc more. While not always perfect, the alternative of implementing purpose built abstractions is not worth it for 99% of projects long term.
The author should look into golang which amazingly gives you one fully static binary (without dynamic linking glibc) and builds directly on top of kernel APIs. They will find reasons to rant about go itself though, I'm sure.
One of the constant lessons I preach to my engineers is to make everything as simple as possible. You can always complicate it later if absolutely needed. Once you make 20 simple things and start to mix them together, you are already in a combinatorial explosion beyond man’s comprehension. The good old devs know this instinctively. The young ones haven’t learned the hard way. The old ones who never learned the lesson must be avoided at all costs.
That's true. I think that in reality, the complex has to "live" somewhere.. It is inherently present in the world.
If the engineer simplifies their solution by relying on a dependency that abstracts away all of the complexity that doesn't mean the complexity is just gone. Should the engineer ever have to deal with that complexity, should that dependency break, they might be screwed.
On the other hand, if they over engineered a more complex solution themselves, the engineer would at least know how to deal with it and fix it if it breaks. I would say usually the issue is lack of time to generate quality work (even if complex) and lack of time to adequately document it, mainly due to the "agile engineering" trend.
It amounts to this: if there exists complexity, there must exist responsibility for that complexity (in my example in the form of the one engineer who created the complex component).
I also like language constructs which have constrained definitions, eg in python list and other comprehensions vs for loops. The comprehensions are very limited in their side effect outcomes and intent, the for loops are much more open ended. Don't use a more open ended construct when a more narrow one can be used. This is sort of analogous to the using narrowest scope possible rule of thumb, use the narrowest conceptual space when authoring software.
143 comments
[ 3.5 ms ] story [ 216 ms ] threadAnd not much has changed since then.
Previous discussion:
https://news.ycombinator.com/item?id=3055154
I hate almost all software (2011) - https://news.ycombinator.com/item?id=28181632 - Aug 2021 (247 comments)
I hate almost all software (2011) - https://news.ycombinator.com/item?id=15142316 - Aug 2017 (8 comments)
I hate almost all software (2011) - https://news.ycombinator.com/item?id=13586596 - Feb 2017 (144 comments)
I hate almost all software. (2011) - https://news.ycombinator.com/item?id=10553408 - Nov 2015 (2 comments)
“I hate almost all software” — Ryan Dahl - https://news.ycombinator.com/item?id=3055154 - Sept 2011 (292 comments)
https://www.linkedin.com/in/tinyclouds
Looks like he's still all in on Javascript with Deno? Interesting.
I wonder what, if anything, has changed in his perspective.
For me it's still by far the best runtime to do anything event-alike/http/json/web. It's quick to develop small things and quick to deploy (if you omit the whole transpilation shenanigans which of course help if you have dozens of engineers working together).
Great winning move, where a ton of the hard part of language optimization & growth is other people's problems, & Ryan could focus on making a great developer experience, which he and izs (npm author) did.
It's quite amazing that we've put layer after layer of leaky abstractions on top of each other and managed to make systems that mostly work.
I don't hate all the software. I think it's awesome. It can be hell to develop, though.
>>> The Programmer-Archaeologist churns through this maddening nest of ancient languages and hidden/forgotten tools to repair existing programs or to find odd things that can be turned to unanticipated uses. - lambda-the-ultimate.org
At the time, I was a freshman in college studying "Classics" - dead languages and history, like Latin and Ancient Greek.
I dropped out that same year, flabbergasted and excited that I could experience the same archeological thrill on a much shorter time scale. All I needed to learn was some system engineering...
So I did that. Been here ever since, loving every day that I got to make some beautiful jank soup of software work for another epoch.
P.S. I was on the IRC server trying to pirate some music and textbooks I couldn't afford to buy. Wound up gainfully employed throughout my 20s, able to retire in my 30s (programmer / data archeology pays well)
One was to dig through a system that was built a decade ago and modernize it. I just wrapped that up. The end result has eliminated multiple points of failure and unnecessary process, and has enabled my team to completely own the solution end to end.
The other task is to dig into and modernize a system that was originally started over 2 decades ago. I've got some other, minor tasks to get through this month and next and am excited to get onto this next task around August.
I'm ~20 years into this career and I am have more fun in the job today than I ever have before.
It was live history working in my hands and in a usable way after Euclid and Newton.
No need to buy hardware and materials to do technical stuff, a computer was enough. And not even a good one. My cwm+tmux setup would still work in a Pentium 2.
The problem is that they're not layers of abstraction, they're just layers.
An actual layer of abstraction should allow you to replace or combine the lower layers. But we just have layers without abstractions.
I can only imagine the people working on the actual chips are screaming in silence somewhere at the complete waste of energy we've made.
https://www.st.com/resource/en/errata_sheet/es0182-stm32f405... https://www.nxp.com/docs/en/errata/IMXRT1170CE.pdf https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
The old canard that programmer time was worth more than processing time that bad coders quote is less and less true by the year; SWEs who are able to wring better performance out of less hardware resources are going to become more valuable in the future.
As much as I'd like for this prediction to come true, I think it's wishful thinking. Compute capacity continues to increase exponentially while (human) cognitive resources, for all practical purposes, do not, which means less efficient code will continue to be the more economically viable option as far as I can tell.
Even with AI, where we're essentially trading compute (used for running code) for extended cognitive resources (used for writing code), what incentive exists to channel those resources into writing faster code rather than writing the same code faster?
As a dev, this bothers me a great deal. We have miracle machines on our desks and in our pockets, and 80% of the capabilities of that hardware are pissed away by terribly inefficient software.
It just barely hangs together enough to sort of function most of the time.
Software that fully works is too expensive to build.
I'm just saying the remaining 10% can be pretty impressive considering the complexity and cruft developers deal with to make it.
[1] https://en.m.wikipedia.org/wiki/Sturgeon's_law
Some of the experiences are downright magical like being able to copy something from my MacBook and paste it on my phone.
The ctrl key randomly stops working, tested by pressing ctrl-C at random intervals. Sometimes it's perfect, sometimes not.
Software is just buggy in general. I mostly agree with the parent comment. The amount of software that I've never had a negative experience with is close to zero.
I worry about complacency with overcomplexity and over-reliance on software, and our fast-growing trigger finger to automate everything....
Personal responsibility and training backed by a human touch will not be beat by code any time soon in my opinion. Citing that code is almost always deeply influenced if not forged and maintained by human hands, any credible singularity is far from our currently over-marketed Ai reality at the moment in my opinion as well... Google can't even get it's assistant and spell check to work properly, and they've been one of the largest companies on the block for decades now.
I have seen too much software to ever trust software with an automobile.
Automobile is as broken as software and hardware.
Things will start falling apart one day.
Is there a hidden assumption in "unnecessary"? None of the tech unfolded in a deliberate, methodical manner. It was random people doing various things over time and a plethora of details.
To assume that the aggregate would have a tidy structure seems unrealistic.
And much goes into RFCs.
But is there much internet-wide engineering going on to squeeze out the entropy?
Repackaged: the engineering that occurs is more bottom-up than top-down. Google, for example, has enough heft to say: "QUIC: here is a bit of top-down engineering." But that example bolsters my general point, in my opinion.
What absolutely shatters us is that we almost never have enduring & exposed bases. Software is endless layers of abstraction reaching upwards. Very few layers expose themselves at all. So we are all strangers to most software.
For software to be simple elegant & minimal, we would have to be able to deploy existing knowledge, be able to leverage the fundamentals we know. But that's not where we are, but we are all adrift:
> In the past year I think I have finally come to understand the ideals of Unix: file descriptors and processes orchestrated with C. It's a beautiful idea. This is not however what we interact with.
Now there's 40. Each one needs it's own networking library. 8 different kinds of databases - with each language needing it's own adapter. 5 different web frameworks for each one. 3 different json libaries.
It's a classic cartesian product. And instead of "I'll help fill in the gaps", its "I'll solve the problem by writing a new language!" -- which just makes the cartesian product bigger.
I personally wouldn't want to live in a world in which I could only use "one of these 5 languages", whatever those are. But I would love to live in a world where things weren't so deep in abstraction and complexity. These are two very different problems.
One aspect is that it would make an interesting situation at the borders between states. And that's kinda sorta what we have now with programming languages.
> Stuff on top of stuff on top of stuff.
Couldn't that also be too many choices? Like if you specialize in JS instead of C, you're not going to have all the knowledge a C expert might know. Instead of utilizing proven, existing C libraries, you're going to write stuff from scratch in JS?
If everyone uses the "proven" libraries and languages and whatever we'll have no innovation. I don't want to live in a rigid world where we somehow ended up choosing a set of things and nobody uses anything else; I want to see more software written in weird languages. Arguments that go like "but nobody is gonna learn a whole language to contribute to your stuff" are depressing - if it's good, I, like many others, would. Without experiments like these we'd still be in the 60s using COBOL because it's "industry proven" or "battle tested" or "blazingly fast" or whatever. That world is boring.
This is Hacker News after all.
A lucky few have a come to Jesus moment and realize that nothing is perfect and that fighting the tide by swimming upstream is a fool's errand, they swim sideways to the shore.
All software reflects the environment in which it exists, including the time pressures involved in its creation and the complex, messy reality it is trying to model.
We’re all familiar with a “90% MVP” that is small and elegant and solves the main use case sort of ok. Beyond that, reality has a lot of detail and edge cases and that’s when software gets nasty.
I do too. In fact, I'm writing my own framework that replaces most of libc.
It builds a better foundation, and I'm better off for it.
I'd love to see your work if you're willing to share it here!
This framework is not meant to be shared really; it's more building my own software stack from scratch, kind of like [1].
[1]: https://www.youtube.com/watch?v=443UNeGrFoM
I mean, sometimes software development is enjoyable as an end in itself, otherwise Shenzhen I/O wouldn't be a thing.
That said, it's a lot more enjoyable when the blocks you're building with are nice self-contained things that work well together like LEGO bricks. As opposed to a bunch of double-clawed hammers that are all invisibly connected with strings under the floor so that when you try to pick one up your drill automagically turns on as it falls off the shelf (I am borrowing some PHP-fractal-of-bad-design metaphor but it applies just as well to Spring Boot or whatever-the-f&^& somebody with too much time on their hands thinks is fun to inflict on me).
Abstractions are great so long as they don't leak. But software development seems to have become the art of not going insane while trying to maintain castles in the sky that are oozing spaghetti sauce from every crack.
So anyway, I agree with the author, but maybe for slightly different reasons.
But then you start buying screws and realize the entire screw isle at building supply store is bigger than many small grocers. And probably isn't complete. Heaven help you if you want to do something aesthetically pleasing like a pocket door.
Or, far worse, heaven help you if you want to fix a pocket door. Finding a compatible wheel mechanism is usually most easily done with, "rip it out and use a new mechanism."
Then there is car maintenance. Check engine light says something about the eco subsystem isn't working right? Here is a book mechanics use that list the common reasons for the code. Not standard across makes. Or models, even...
I could probably go on. Point is nothing is greenfield lacking in complexity. I agree we probably could do better with software. But having done embedded work, I think it is easy to be ignorant of exactly how much has to coordinate in a computer system.
And most software ppl are quite useless at group formation and maintenance, hence the perpetual drama and tantrums.
That kind of troubleshooting is about as software as it gets.
Was it what any of the mechanics I talked to thought it was? Nope! Did anyone carry the part, or was anyone even able to identify the correct part number, even with a diagram? Nope! Did I have to hunt all over for some marine fuel line that happened to have the same inside diameter so that I could jury-rig a new part? Yep!
Diagnosis and repair actually felt a bit like working with third party libraries: all the documentation is wrong, the problem is never your first guess, and you end up rolling your own solution in the end.
Quickly slapping some Vaseline on the has cap gasket has worked in my Ford.
A couple of German cars later, that CEL is always much much more expensive.
https://en.wikipedia.org/wiki/Stress_corrosion_cracking
And the pipes/hoses! Having your software update cause your server to fail is _absolute joy_ compared to discovering that your improperly rated pipe burst and is how dumping gallons of water on the ground.
This is the idea that people who have learned one thing discount the knowledge of experts in other domains.
Nothing can be as fantastically complicated as my own work, surely! Building a house is just nailing a few sticks together. Fixing a car is just undoing a bolt with a spanner. Running a farm is just driving around on a tractor.
I have one disagreement with this... There is another thing that matters in software --- the experience of the developer making it.
IMHO, I agree that libraries, languages, tooling,... etc. are not art whose complexity should be admired. But they are a necessary intermediate tool for _developers_ to make their work easier.
We can apply the same logic to something like construction. You can build a house with hand tools and man power, just like you can write code in machine language. It's not enjoyable but you can do it. The work around developer tooling is like a builder working with scaffolding, an excavator, a crane, an electric drill... You spend effort to set it up, and before the final product is delivered, you have to tear it down. But it was still worth it because it serviced the developer in some transient way.
The software serves the user. Period.
The user only matters in so far as the keep coming back and paying money (or attention, which is also money).
Developer experience only matters in so far as it keeps costs low, creating better margins.
If that sounds kind of gross to hear, it's because it is. Businesses are made up of people. Customers are people. Every interaction on both sides of the equation should take that into account. If you ruthlessly maximize like the human part doesn't exist, your optimization strategy will fail.
Making big assumptions about what the software project here. Given the article was mostly complaining about software developed for developers by developers, I don't think we can blame the suits for this one.
All they care about is does it do what I want+expect it to do when I want it done while it solving my problem? That makes money.
There's lots of scenarios where the end user is in fact another developer, but I think the spirit is talking about the general populous.
Tooling does not get complex because engineers are somehow magically attracted to complexity. They get complex because the world is complex. To "hate [the complexity of] the tooling" is really just indirectly saying you hate the complexity of the world... which, I mean, I guess is fair, but not that interesting.
[1] Whenever I hear that word, I think "is that the best adjective you can come up with to convince people to use it?" because it often has no other advantages.
FWIW, I'd probably be considered a JavaScript luddite these days lol. When I build my own projects, I write vanilla ES5 (yes, 5). No tooling. No frameworks. No compiler. Just edit and hit refresh.
But when I work with others I will respect their choice of framework. Because the goal of frameworks and libraries and encapsulation in general is to trade-off extra individual work in exchange for smoother between-developer "compatibility" (reduced communication friction). I don't need to understand someone else's code, and neither do they need to for mine.
I don't understand this take. Client-side JavaScript is hard because of cross-browser / cross-platform / cross-device challenges. Server-side JavaScript is hard because of the nature of the language and the runtimes, the lack of standard library and the myriad attempts to "fix" it, ranging from inventing new languages to transpilers to runtime rewrites.
May be you mean to say, a lot of complexity could have been avoided? Unfortunately, that ship has sailed.
Is it really? Or is it hard because people are under the impression that it's hard and thus jump into the quagmire of others "selling" grossly overcomplicated "solutions" hoping it'll be easier (when in fact it isn't)?
When I can replace something that a team of about a dozen needed several months to create using a few MB of frameworks that work only in the latest version of Chrome and somehow needs multiple seconds to load over a gigabit LAN (internal tool) with an HTML form and a few lines of JS written in an afternoon that will work in any browser newer than ~IE5 (and will mostly work even without JS at all), I don't think that's the answer.
I don't understand this take.
As the saying goes, "It is hard to get a man to understand something when his salary depends on him not understanding it."
I think that's ultimately what it comes down to: Developers justifying their existence. They're all enamored with the new and shiny, resume-padding crap that they forget the problem they were hired to solve and instead create more problems that they then solve to show how "smart" they are.
...then they all get thrown out when the company downsizes. That's why I ended up having to do some JS work. I don't normally work with JS but I can use it if I need to.
I never had a need to do server-side JS so I won't comment on that in specifics, but I suspect it's much of the same situation.
I somewhat think there's a huge error here though, a massive mistake in computing that we have such a huge divide between user and developer. A couple small vim tricks or enough shell knowledge to be a little dangerous, or even someone ok at excel macros blends the line so much here.
We have been building more and more totalistic apps, which afford less & less view of computing & developer-ish sensibilities. It keeps getting harder to see because we are so entrenched in pro application anti computing paradigms, but I think some day consumeristic computing will be worn thin. I New-old dynamics can re-start, embracing systems that also try to offer more developer-like sensibilities & capabilities.
Douglas Engelbart's idea of augmenting intellect to me speaks of the need for people to get great at toolbending, I at taking some good existing tools & capabilities, and being greatly empowered to bend create & explore different configurations & possibilities. Soft systems, with malleability, where users can ad-hoc cross over the aisle & become developers: that's a strong culture, where softwares real hidden noble purpose lies.
And where we can stop hating, stop being adrift & trapped, amid unyielding prisons of hard computing.
Is a infrequent commonly one time thing that pales in comparison to operating it.
Is that how software development works today?
Writing software - is an act of self-expression like it is for an artist or writer.
Everybody wants to write his/her masterpiece in his/her own way, and not always agree with the way others do it.
same with scientific community - two scientists cant agree on two ways to conduct scientific experiment, because they want to express themselves in this act
Others just want to write literally anything to get the job done and be on with it.
Others resent the fact they're being forced to write it in the first place.
There are likely many other archetypes I'm ignoring too. But even reconciling these 3 together is what leads to everything regressing to the mean amount of suckiness.
this is not compatible in team-based software development. Once you want to collaborate with others, the #1 priority becomes not "anything that gets job done", but "anything that team can read, maintain, support, and continue developing"
if you are solo dev or writing bash scripts for your own workflow - then of course you can use anything literally, as long as it is on your computer and nobody sees it
Oh hey, yeah. A bunch of small utilities that communicate and interoperate. I can see it working out. You could even have them communicate with each other via some sort of universal interface.
The key is to have that interface be narrow and well defined - something like gRPC. Allowing them to communicate via a huge, poorly specified, non-concurrency-safe swathe of shared state (filesystems with all their ad-hoc semantics, shared memory, signals, pipes, KAME sockets, namespaces, goodness knows what else two processes on a Unix system can do to fuck with each other) is a recipe for disaster.
You really can't use a mesh of unikernels for this then. You need an external system to enforce the types (like a compiler does for a single program).
At runtime, so far as the computer cares, there's really just arrays of bytes (or machine words depending on if your system is really byte-addressable or just faking it).
You can do input validation, but unix cli utilities can already do that. You'd need, rather than a bunch of unikernel utilities, an all encompassing environment that the utilities can register their constraints with, and the system would prevent them from being called if the constraints were not met.
Main contender is probably emacs, just because it's already ahead on the "all encompassing environment" part of this.
The problem isn't arrays of bytes. The problem is all the other semi-structured cruft that the OS has accumulated, the squillion different system calls you can make with almost-but-not-quite standardised effects.
> You can do input validation, but unix cli utilities can already do that. You'd need, rather than a bunch of unikernel utilities, an all encompassing environment that the utilities can register their constraints with, and the system would prevent them from being called if the constraints were not met.
Or you just don't give them the interfaces to mess with each other's internals. You have your hypervisor isolate them from each other except for a narrow communications channel obtainable in a narrowly specified way. Ideally the hypervisor also enforces that what you send/receive is well-formed gRPC (which is not hard), but that's not essential - the main benefit comes from just closing off all the other random OS-level ways that Unix processes communicate with each other.
Also a lot of that C code in Unix, you know, actually like works and has worked for a long time! 50 year old C code in a Unix operating system isn't going to be unheard of here shortly.
We mostly don't actually use the OS though. We write code that gets deployed on some managed cloud system, and to the extent that there's anything in between (a unix process, kernel and virtual machine to run the code) we probably cargo cult it and don't worry about the details unless it breaks. Building unikernels (which is increasingly practical, with OCaml leading the way but other options available) would sidestep many of those layers of junk.
("Serverless" is another way to achieve the same thing, and may well be the one to succeed; even if serverless systems don't currently use unikernels, they could be transparently upgraded to them in the future, if they're strict enough about the interfaces they offer at the moment).
But to be the devils advocate here, the unix abstraction is only beautiful if your living in a single process, single user, teletype environment. Add multiple users, network programming, multiple processes/threads, async IO, etc, etc, etc. and the whole thing starts to look like the layered monstrosity full of edge states, and broken promises the author is arguing against.
Although frankly, there aren't many better choices. NT being nearly 3 decades newer and with the forethought of VMS is actually much better at a systems programming level but you have to be willing to dedicate significantly more effort upfront to understand the more complex task/io/user/permissions model. In the end there is probably less of a learning curve on windows, but its steeper because you end up having to learn how to do all this stuff on the various *nix clones as well, just using bolt on interfaces that frequently vary slightly between the different flavors.
In a way this applies to much of the windows ecosystem now that vb6/delphi aren't popular and the similar attempts at web co-design utilities have largely died.
Look at the demoscene for an example of the exact opposite culture.
With the rise of the Web/Mobile computing things have gotten exponentially worse. And i don't even want to think about where we are headed with AI/ML.
You only have to configure this once. Don't install too many things. Just what you need. This level of complexity is needed because this is not TempleOS in which all software is made by one person. Most software exists to bridge the Tower of Babel lost in translation scenario.
NixOS is also a subjective take on packaging, heavily luring people into a functional programming world, where one set of problems is merely replaced by another set of problems. I tried NixOS, so no, thank you. There are better ways to isolate packages and have multiple versions of libraries. The simplest of which that come to mind are things like Flatpak and containers. There's almost 0 usecase for NixOS - it merely replaces one set of problems with another. And forces you to learn something, that does not justify the time it'd take to learn it. I feel like people who built it just wanted to be smart, not create things that are seemingly simple and beautiful (which is what true artists strive for, I think). I learned Vim in two weeks. I gave NixOS the same time and it was almost a waste of it. Not a complete waste, however, because, I recall, I discovered an extremely useful, but unrelated thing (zfs), but I don't remember how come it worked out this way.
The author should look into golang which amazingly gives you one fully static binary (without dynamic linking glibc) and builds directly on top of kernel APIs. They will find reasons to rant about go itself though, I'm sure.