I'm not sure the car comparison is a good one, given that their inflated sizes prevented us from actually saving fuel... Most of our modern industrial complex is by and large wasteful on resources, computing is no different.
"As engineers, we can, and should, and will do better. We can have better tools, we can build better apps, faster, more predictable, more reliable, using fewer resources (orders of magnitude fewer!). We need to understand deeply what we are doing and why. We need to deliver: reliably, predictably, with topmost quality. We can—and should–take pride in our work. Not"
Right. Many of my performance enhancements added by caring Devs are kept secret from the PMs, because they are out of scope, don't have tickets, and/or may increase complexity.
The problem is there are engineers who will take your paycheck and build crappier software more quickly, because there's barely any quality regulation for software. That's the difference between software and other traditional engineering disciplines. An electrical engineer has their license at stake.
They're mostly independent components, controlled by software, talking over a shared bus to other components. It's already bad and has been for some time.
I read half way through, enough to get the sense. What the guy is missing is the difference between “it bothers me” and it matters.
He uses a car analogy (how much efficiency we squeezed out of cars vs software) but misses the point - if a car was available that consumed half the gas or went twice as fast, I’d buy it!
If a computer / OS was available to me that was so much faster / more efficient than what I use… turns out I don’t really care and neither does anyone.
I am writing this on a 6 year old iPhone X that I got after my wife upgraded to the iPhone 15 at the same time my toddler drowned my Pixel 6 pro. The iPhone X is noticeably laggier than the 15 or the pixel but it doesn’t matter enough to go upgrade. And ditto for my 10+ year old PC. Point being, if I as a consumer am not bothered by the perf I get to even just drive down to the store, manufacturers would be dumb to over index on it
Obvious this is “just me” but if people really cared about a phone that boots in a second instead of 30(his example) there would be a market for it but the fact is nobody reboots their phone and it doesn’t matter. On the other hand if I had to wait 30 sec to drive my car and another manufacturer reduced that to 1, I’d switch.
TLDR - yes it’s inefficient and no it doesn’t matter. Those old systems he’s talking about were “efficient” because they had to run on hardware of the day, and increased requirements meant decreased market size - ie it really mattered! I am sure someone bemoaned the bloat of win95 that couldn’t run on his beloved 286.
I believe what the author is bemoaning is not the business or customer realities, but the "art of engineering" being traded in for the bottom line. In many ways, this is simply the way all disenchantment works, the hard truth is that art doesn't really matter for its own sake. It needs to sell, and what makes it sell is not what makes one love the process of building. The elegance, the simple beauty, none of those "matter" to the market. Unfortunately, writing up articles stating how much you can't do what you love probably won't garner much sympathy (clearly), but I suppose that's why we have passion projects instead.
One of the key differences between art and engineering is that the former eschews compromise while the later requires it. An engineer that over-indexes on a single dimension of what they are working on is seldom a good engineer.
Efficiency is one dimension. If I couldn't run calculator on anything but the latest Pixel / iPhone, that would be clearly insane. But it's not the sole dimension - if my ambition is to create something that works well for millions or billions users, hand-crafting Assembly may not be the right tradeoff.
People care but they have other concerns like compatibility that force them to surrender to bad software.
Look how bad the self checkout experience is. You might say “if people cared, they would shop elsewhere”. But people want to buy food more than boycott a store with bad software.
// Look how bad the self checkout experience is. You might say “if people cared, they would shop elsewhere”.
I would say that bad checkout matters - in the way that I am talking about mattering - because it does effect a non-trivial amount of behavior.
First, I am much more likely to opt for self-checkout at Whole Foods vs in my town's supermarket (that I otherwise like) because the WF experience is pretty smooth while at the other store, the machine blocks until help arrives every X items. Does that matter? Well I think so, because (1) some # of people will go to WF over this store just because of that (2) some # of people will stand in a cashier line - costing the store money - because of this (3) some people will opt for delivery because of this - undermining the "local store" value proposition in the long run.
The above impact exists because in the bad case, self-checkout is nearly unusable. If it was just a matter of every item scan taking 2 seconds instead of 1, or needing to get human help once a year instead of several time per checkout - nobody would care.
Until a critical mass of movers forms, no change will be made. And there's other inertia, including lack of signal from customers, existing practices, vendor monopolies.
Your reasoning assumes a fluid system with perfect observability into everyone's interests. Consumers can't even articulate their own interests fully -- it's impossible for vendors to respond properly.
Quality and high performing software exists when vendors elect to produce it. There are intrinsic motives and extrinsic.
Most quality products exist despite consumer demands not because of them. Don't fall for the "money incentivises everything" trap.
// Quality and high performing software exists when vendors elect to produce it.
Can you give me an example of where this exists separately from "and we can sell more units before its performant"? I am open to that being the case, but wasn't able to think of a good example. Can you?
He lost me immediately with the car analogy, as if the auto industry, its products, and the ways they're used by consumers are in any way reasonable.
Everyone needs a car capable of reaching 120mph right? Moms and dads need giant utility vehicles to feel safe on the streets surrounded by giant pickup trucks. How is it "efficient" to drive trucks that are 2-3x the curb weight with less cargo capacity?
The problems occur due to business decisions not engineering
Funnily enough, the author is the creator of DataScript, a language which has been the source (directly or indirectly) of some of the biggest bloat/performance problems I've encountered in client side programming.
> Your desktop todo app is probably written in Electron and thus has a userland driver for the Xbox 360 controller in it, can render 3D graphics and play audio and take photos with your web camera.
Can Electron not "tree-shake" parts of the browser engine that are unused? Either by static analysis or manual configuration? Seems like a real missed opportunity to trim down on bloat...
The "build" step of an Electron app doesn't build Chromium, so this wouldn't be very feasible. Building Chromium requires an insane amount of computing power.
According to Google, a Chromium build on normal hardware takes 6+ hours.
And alas, even if it was feasible to custom build based on what you need, it would have to be done via configuration--since there's no way to know at compile time which language features will be used, since your app could (and probably does) include remote scripts.
Yeah that sounds about right - I use a chromium on Android fork and according to the lead developer, it takes about 3 hours for a release to compile and that is after optimizing the process as much as possible.
I'm not familiar with Chrome's architecture enough to say, but I would be surprised if all these capabilities get paged in and initialized. There definitely has to be some work to setup the web platform bindings, to let JS think this stuff is here ready to go, but I hope it's backed by late bound code.
And in the web browser, I believe v8 is using snapshots of the js runtime to avoid much of the work of initializing the js side of things: it just forks a new copy of itself.
This is one of the prime strengths of alternatives like Tauri, that use a shared library model rather than having a static library. With Electron you have a ton of initialization to do for a very excellent runtime, but then you never get to reap that existing work again. Where-as on the web, we open new pages and tabs all the time, and avoid the slow first load & get to enjoy the very fast latter instance loads. That multi-machine capable vm pays off! With Tauri, the shared library may well already be in memory and initialized. Only the first consumer of the shared library has to pay the price.
Nope, it doesn’t do this. A while back I looked into doing a “hello world” version of that - trying to remove the Print feature. To do this, I had to build Chromium, which did actually take like 5 or 6 hours on an M1 MacBook.
From what I could tell of the config files, Chromium is pretty modular. It looks like you could just delete a few lines and have it avoid compiling entire subsystems. But I didn’t ultimately achieve my goal because I couldn’t get it to compile with those changes. IIRC it hit a linker error and I couldn’t figure out what to prune next, and I wanted to get back to actually building my product. (I ended up switching to Tauri anyway)
Part of me wants to revisit that project though. It would be so great if there were custom minimal builds of Chromium for Electron apps.
No, it can't. Chrome is deliberately designed to be entirely non-modular. The codebase itself is somewhat modularized in the sense that related functionality is grouped into different directories and build targets, but the Chrome team have no interest in allowing it to be cut down to just the needed functionality.
The main reason for this is politics. The Chrome team is ideologically wedded to the idea that everything should be a web app running on Chrome. They see desktop and mobile apps as the "enemy" to be wiped out by making the web do more and more stuff, until Chrome is the universal OS. Classical ChromeOS is the pinnacle of this vision - a computer that only runs a web browser and nothing else.
Chrome's architecture reflects this, um, purity of vision. It is not reusable in any way and the Chrome team do not care to make that easier. Projects that make it embeddable or reusable are all third party forks that have to maintain expensive patch sets: CEF, Electron, etc, all pay high maintenance costs because they have to extensively patch the Chrome codebase to make it usable in non-Chrome apps. The patches are often not accepted upstream.
This problem also affects V8. Several projects that were originally using V8 are trying to migrate to JavaScriptCore or other JS engines because V8 doesn't have a stable API and building it is extremely painful. It's a subsystem of Chrome, so you have to build Chrome to get V8.
This is a pity. There's a ton of great code in the Chrome codebase, and it can be built in a modular way (with millions of small DLLs). It's slower to start up when you do that due to the dynamic linking overhead, but it does work. Unfortunately for as long as the Chrome guys see native apps as a bug to be fixed and not a reality to be embraced, we will continue to have dozens of apps on our laptops which statically link an Xbox 360 gamepad controller.
There are a few possible solutions for this.
One is to not write Electron apps. Java went through a modularization process in version 9 and since then you can bundle much smaller subsets of the platform with your app. I guess other platforms have something similar. Obviously, native apps also don't have this issue both because the platform comes with the hardware you buy and because they tend to be more modular to begin with. But, people like writing Electron apps because it gives you the benefits of web development without many of the downsides (like the ultra-aggressive sandboxing). The nature of the web platform is that it always lags what the hardware can actually do by many years due to the huge costs of sandboxing everything so thoroughly, whereas Electron apps can just call into native modules and do whatever they need, but you can still use your existing HTML/JS skills.
Another would be for an alternative to Electron to arise. There are experiments in using system WebViews for this. That doesn't make the web platform more modular, but at least means it's a single install is being reused. You could also imagine a fork of the web platform designed for modularity, for example, in which renderer features are compiled out if you don't need them or even bringing back renderer plugins.
Another is to just tackle the issue from an entirely different angle, for example, by opportunistically reusing and merging Electron files on disk and in memory between different apps. If you ship your Electron app to Windows users using Conveyor [1] (or in the Windows Store using MSIX) then this actually happens. Windows will reuse disk blocks during the install rather than download them, and if two apps are using the same build of Electron the files will be hard-linked together so only one copy will be in memory at once at runtime.
But fundamentally the issue here is one of philosophy. Chrome wants to rule the world and has a budget to match, yet their approach to platform design just does not scale. For as long as that is the case there will be lots of complaining about bloat.
>The Chrome team is ideologically wedded to the idea that everything should be a web app
I'm not surprised, and doubt the Firefox team is any different in that regard.
So far at least the Chrome team hasn't removed from Chrome the ability to go to a new web page in response to code external to Chrome, so we have that to be thankful for at least. Yay?
(The desktop code I maintain achieves that effect by invoking /opt/google/chrome/google-chrome with the desired URL as an argument.)
The comment I wrote at a higher level in this same thread is, ultimately, a criticism of NOT using a statically compiled, statically linked language (because a form of tree shaking like original commenter suggested is already part of the linking step there, except for DLLs).
And yet, full disclosure and admitting to cognitive dissonance, for a (hobbyist) C++ game engine I'm currently working on that targets Emscripten for a web build and native for Debug build, I'm considering not even having a native-Release build at all.
The idea being if it's web-first and the native build being only for developer use for debugging, I could do things like supporting only one desktop graphics API (e.g. just DirectX) and optimizing the native graphics pipeline for simplicity over performance. End users would/could just use the web version.
Granted this is a bit different because I wouldn't be distributing a browser too a la Electron; it would just use the browser the end user already has. Just thought it's interesting that it's easy for me to criticize others, but with the choice of how to spend my own limited developer time (my free time) it's looking like this way of doing it makes the most sense.
I kind of at least partly agree with Chrome. My problem with it is that web apps are missing a lot of features you would want in a non-cloud environment for offline use.
But if something is proprietary and cloud linked anyway I would much rather it all go through the web. That way open platforms still have a chance.
If banking apps and and Google payments and proprietary IoT devices controllers were all on the web, then a Linux phone might actually be viable!
How would it be a "Linux phone" if the only software you could run was web apps? That which defines an OS is its APIs and unique capabilities, anything can act as a bootloader for Chrome.
Unless they actually convince Linux to block non web apps, you could run whatever you want on a FOSS platform, you'd just be able to run proprietary web stuff in addition to native free software.
I'm guessing eventually the native free software would move more and more into the browser too, but that's fine as long as you can still run the stuff that hasn't been moved yet or isn't interested in moving.
This is an important distinction between statically compiled and interpreted languages which is often lost in discussions that focus on developer usability.
For many years I worked on my own game engine which combined a C++ core with an interpreted scripting language. I had developed a system of language bindings which allowed the interpreted language portion to call functions from C++. The engine quickly grew to a multiple-gigabyte executable (in the debug build), and no matter how much I tried to optimize the size it was still unconscionably huge.
One of the reasons I eventually gave up on the project was I realized I was overlooking a simple mathematical truth. The size was NxM, where N is the number of bindings and M the size of each binding. I was focusing on optimizing M, the size each binding added to the executable, while not just ignoring N but actually increasing it every time I added bindings for a new library I wanted to call from the game engine.
There were diminishing returns to how much I could improve M because I was relying on compiler implementations of certain things I was doing (and I was using then-new next generation C++ features that weren't well optimized); it would be a lot easier to simply reduce N. And the easiest way to do that would be some sort of tree shaking.
Unfortunately due to the nature of interpreted code it isn't known at compile time which things will/will not be ultimately called. That determination is a runtime thing, by calls via function pointer, by interpretation of scripts that start out as strings at compile time (or even strings entered by the user during runtime).
From a compile time perspective, static usage of every bound function, feature or library already exists - it is the C++ side of the cross-language binding. That's enough to convince the linker to keep it and not discard it.
In fact, the mere presence of the bindings caused my game executable to grow more per each included library than would a similar C++ -only, all-statically linked program. If a library provided 5 overloads of a function to do a similar thing with different arguments, an all- C or C++ application that uses only one of them would need only include that version in the compiled executable; the others would be stripped out during the linking step.
Since I don't necessarily know ahead of time which overload(s) I'm going to end up using from the interpreted language side of the engine, I would end up binding all 5. Then my executable grew simply from adding the capability to use the library, whether or not I make use of it, but moreover if I did use it my executable grew even more than an equivalent C/C++ - only user of the library because I also incur costs for all the unused overloads.
You can see why something like Electron would have the same problem. Unused functions can't be automatically stripped out because that information isn't known at compile time. To do it by static analysis the developer of the Electron app would have to re-run the entire build from source process of the Electron executable to combine that with static analysis of the app's Javascript to inform the linker what can be stripped out of the final executable.
And it bears mentioning neither such a static analysis tool for Electron app Javascript nor the compiler/linker integrations for it currently exist. In theory they could exist but would still have trouble with things like eval'd code.
Manual configuration would be possible but necessarily either coarse-grained or too tedious to expect most developers (of Electron itself or users of Electron) to go into that much detail. That is, you may have manual configuration to include or not include Xbox 360 controller, but probably not for "only uses the motion controls" while not including other controller features.
Either way you wouldn't be able to add-back support for it Javascript written after build time turned out to actually need the function or feat...
Everything has a cost. In the cases where the computing cost or degraded user experience is high enough efficiency is optimized for (see ML models for example). In other cases it's not because the end user doesn't actually want that at the cost of fewer features. Cars used to be gas guzzlers until fuel costs and environmental concerns caused customers to want something different.
A lot of engineers forget that they are in fact paid to build a product for customers.
edit: It also seems OP has never had to wait for older Windows or Linux machines to fully boot. Modern versions boot much faster because customers wanted that. Phones are on 24/7 so customers don't care if it takes longer to boot once every few months.
That's nothing. I remember when booting a game required loading it from cassette tape. Me and my friends would go round to each other's houses, start the computer loading a game, go outside to play a football match and when we got back it'd be nearly ready to start.
I mean, there's booting, and then there's loading a program...
My Commodore 64 from the early 80s booted to a full BASIC REPL in around 1 second. Yes, loading from removable media was slow, but the computer was ready to fully use in less time than it takes "modern" systems to even POST. Totally ridiculous.
I don't know where the POST code is run, maybe not the CPU ? Also, i'd imagine this sort of code to not parallelise the checks because it try very hard to be bug-free, so we are looking at a piece of code scanning the whole RAM linearly, I don't think you reach anything near the max bandwith of your stick of RAM in this scenario.
The problem here is that it does that at every boot, it should do it once, you maybe have disable fast-boot options or equivalent in the bios ?
A huge amount of boot time seems to be because BIOS waits several seconds(!) by default for various hardware to get into a stable state after powering up.
Since when customers have any meaningful choice or way to express preferences? It's long been a supply-driven market: vendors make what they want, customers buy what's being put on the market.
I feel like it's gotten even worse with pervasive telemetry and A/B testing. Now customers get more of what they "engage" with even if that's not what they want.
>Since when customers have any meaningful choice or way to express preferences? It's long been a supply-driven market: vendors make what they want, customers buy what's being put on the market.
By your logic Tesla and Apple shouldn't have grown like they did since customers wouldn't pick them over existing incumbents. Customers do have a choice in aggregate and they express that choice. Some people who don't agree with that choice try to say customer's have no choice but in the end they do and the person saying it is upset they're a minority.
Partially true, but let's not pretend that when trends are set by the companies of the size of Google or Apple, they are still following the customer's will the same way some small, or even medium sized company does.
They have enough power to shape the landscape itself, and as such they can drastically steer the outcome of "customer needs"
Neither of them used to be the giants they are but overall they made solutions that their customers (who are, btw, advertisers in Google's case) wanted and bought over larger existing competitors. Apple's market cap has increased 500x since the late 90s.
Obviously they entered the market with simply good products, but that doesn't change the fact that now they have a lot of power that comes simply from their size.
Im not saying its their only advantage, just that it plays big role.
But also, common now. Today's platform lockins are much more impactful then anything we had in the 90'. And you don't compete with Google's product anymore, but rather you compete with ecosystem that half of the world bought in. Which makes it practically impossible to be a viable alternative
> A lot of engineers forget that they are in fact paid to build a product for customers.
In a way yes but I think in many ways software became much more than that. In many ways it became a crucial part of human lives as electricity or medical services. And you can say doctors/engineers are being paid to develop new medicine or a new power sources but these have many many levels of government control just because how sensitive and important the matter is, which is not the case with software (yet?).
I am getting really tired of these extremely low effort "manifestos". They complain about the state of software, but make no attempt at understanding why it is that way or what tradeoffs had to be managed. There are just these vague claims that "we should be better" because the problem is that we're just "lazy". It's the writing that is lazy, not the industry. The software industry is arguably one of the most productive spaces that the world has ever seen.
If you're going to criticize the state of software, you have to:
1. Demonstrate you understand the systemic incentives in place that encourage this behavior,
2. persuasively argue why this condition needs to be improved,
3. and propose concrete policies or incentives that counteract these tendencies.
Otherwise it's just whining about something you don't really understand, and the proposed "solution" is to just start all over because this time it'll be different I swear.
The author appears to raise valid points. There are explanations that they did not visit, which is okay by me. It does read like a manifesto but gets the conversation started.
> The software industry is arguably one of the most productive spaces that the world has ever seen.
I would argue based on personal experience that majority of software that comes out of the industry is unmaintainable garbage. The reason it is acceptable in most cases is that the barrier to entry is low and there are few consequences of things going wrong. Quality goes up significantly when consequences become real. At any rate, I would not call the software industry "productive".
> The reason it is acceptable in most cases is that the barrier to entry is low and there are few consequences of things going wrong
Then it is acceptable. You're making a moral value judgement because it's not up to your personal standards, but that's not relevant for whether or not the software accomplishes what it set out to do.
> At any rate, I would not call the software industry "productive".
This is indefensible, given that basically the entire world runs on software these days. Again, I think you're making some sort of subjective value judgement about the perceived "quality" of the software, while ignoring what it has enabled.
Whining about a problem comes before fixing the problem. Demanding people must come up with a solution on their own before they share a problem doesn't help fix problems.
And my point is that this problem has been whined to death, with absolutely no concrete suggestions on how to fix it. All I've ever read is complaints about how some-or-another software should be better, and surprise! the author knows absolutely nothing about the codebase or the tradeoffs being managed. It's lazy armchair software engineering.
It obviously hasn't, there is still so much low hanging fruit everywhere.
And don't think this problem is unfixable, we will fix it at some point via standardization and each whiner is a data point that helps us get closer to standardization. It happened to cars, planes, boats, tanks etc, those fields started out with a lot of whacky ideas that didn't fit but then designs got standardized and the only thing left to do was to optimize those standard designs.
Edit: But for example, whining gave is git, a fast standard version control system that everyone now uses. When enough people whine about something then it gets fixed, bigger problems needs more of it.
The problem is that the whining on display in the post is completely rudderless. It has no idea what it's actually complaining about, just a general feeling of "wow things are complicated aren't they?". Then it asserts a priori that it could be simpler. Out of all the different examples that it complained about, not once did they propose a solution in any of those scenarios.
If your argument is that things are more complex then they need to be, prove it. Because it's like an old man looking at a Honda and going "things were so much simpler back in my day!!"
We need more of these articles. Teams has no business being as slow and awful as it is. No reason to try multiple times to do a simple task in any OS and app. Sony cameras computer software UX is inexcusable. Overall software quality is dismal. And society is ever more dependent on it.
> Teams has no business being as slow and awful as it is
Why not? This is part of the problem I'm talking about: we have too many armchair software engineers that know nothing about extremely complex codebases making sweeping generalizations about how they "should be". Their solutions are invariably "let's get rid of most features" or "let's just start over and Do It Better This Time."
I didn't say it in the post I wrote about C [1], but this is a big reason why I use C: I will have a hard time bloating my software. I can add features, yes, but adding a singular feature struggles to add even 100 kb to the executable.
I don't work for anyone right now, but I do have a "work" machine. This machine is beefy.
But I still run Neovim and tmux instead of an IDE. [2]
I don't run a typical Linux distro; I use a heavily-modified Gentoo, and that includes using OpenRC over systemd. [3]
I don't use a full desktop; I run a TWM called Qtile. [4]
All of this is so my machine is not bloated. When my machine boots up, and I just barely log in, it is running only 40 processes (including the terminal and htop I use to check).
As of right now, I'm engineering software. Truly engineering; I am spending the effort to effectively mitigate all of C's problems, while keeping the software lean and fast. I hope to someday build a business on that software.
I guess I'll see if there even is a market for non-bloated, sleek software anymore.
C++ is huge and highly interconnected. No matter whether you like it or dislike it, and how experienced you are with software development, you either invest years into becoming good at it, or stay away from C++ jobs.
If I was going to go back and time and offer younger myself career advice, I would probably say "just stick with C++". It was the most popular programming language when I started my career. It will be the most popular programming language when I end my career.
Think about it. Is there any widely-deployed OS not written in C/C++? Any web browsers? Spreadsheets? CAD software? Text editor?
Having as much experience as you do and still conflating "widely used because it's super good" and "widely used because of network effects and inertia" is really puzzling.
Surely you understand that your argument is disingenuous, right? Nobody wants to start over a new OS or a browser in another language due to the huge costs that no corporation nowadays is willing to shoulder (due to their own interests). Otherwise a lot of people would very definitely write OS-es and browsers in Rust, D, V even, Zig, Nim and a good amount of others.
C/C++ were there first, that's all there is to it, it's that simple.
Besides, I struggled with sysadmin-type tasks until I bit the bullet and installed Linux from Scratch and then Gentoo. It was one of my best educational investments.
I would say truly engineering would be contextualizing these decisions along a spectrum of tradeoffs and positioning your project on that spectrum according to the constraints of its creators and users. That may put you to one extreme of the continuum, but that doesn't mean people whose constraints land them elsewhere are not doing "true engineering."
"I am going to build the strongest, lightest bridge in the world" is a marvel. "I am going to build a light-enough strong-enough bridge for a cost my client can afford" is engineering.
Trade-offs also get a bit meta with "standardization" one way or another. Sure you could get peak performance if you made custom length and width screws but it boosts costs and makes it harder to maintain with the one-offs. Going with trade-offs is itself a tradeoff which means monoculture breeds a pattern of ignoring the tradeoffs.
Hear! Hear! Engineering is a practical discipline - and it is a discipline. It's building for purpose. I appreciate Tonsky's point, and agree with much of it, but engineering means turning out into the world things that are fit for purpose, however that's defined. Purpose is +100 and all the rest is something of less than 100. Mathematics is mathematics, science is science and engineering is engineering.
I agree, except that engineering is"fit for purpose at the smallest cost."
For me personally, I am banking on having multiple business customers for my software. If I consider the cost amortized, I can spend a little more to engineer something better.
But yes, it still has to be fit for purpose. My customers will provide a document that explains what they want to do with my software and on what platforms. If I sign a contract, that means I am committing to supporting their purposes on those platforms.
The ethical considerations and professional obligations that go into creating a bridge aren't exactly translated into the act of creating software. If we mirrored those in a bridge building analogy the "cost my client afford" would mean the engineers would often find themselves greenlighting a bridge that would fail in sensible conditions in our current world.
No. A civil engineer's response would (should?) be immediately that the client cannot build the bridge for that cost, simply because it will kill someone as a first order effect. That's the ethical consideration. The downsides of janky rendering or poor memory management aren't quite the same unless you actually add up the power wastage and apply it as an impact on global warming. But, those are not a first order effect, and although they should be, they don't have the same influence on the solution.
There's plenty of software that can kill someone as a first-order effect. If you allow second-order, there's even more.
Avionics. Air traffic control. Industrial control. Automotive software. 911 dispatch. Medical instruments. Medical information. Military command and control. Mechanical engineering analysis software.
That's not what I'm saying. If a web page takes 2s to render rather than a theoretical minimum of "immediately", for most software that most people write, no one will die, or be hurt, or even feel too bad as a first-order effect. Obviously, if one is writing flight control software, or somesuch, then the stakes are higher, and one ought to be aware of that. That's the practicality calculation. If you're going to kill/hurt/really upset someone, then what you're doing is not fit for purpose (unless that is the purpose :) )
> As of right now, I'm engineering software. Truly engineering; I am spending the effort to effectively mitigate all of C's problems, while keeping the software lean and fast.
Can you expand on that?
Which exact problems of C's are you working on solving? Do you mean the language itself (writing a new dialect of C), or the ecosystem (e.g. impossibility of static linking with glibc)? Or something else entirely?
Since you care about these things, why not try a language that has scoped threads and bounds checking built in, like Rust or Ada?
Actually, I had written more about Rust and Ada, but then I read your blog, and it says you like C better anyway because you find it more fun, so nevermind.
You might also like to use D in its "Better C" mode, which at least offers bounds-checked arrays and sliced, as well as some other features, while being very similar to C.
Not to pick on you—this is something I and most other engineers also struggle with—but I think this sums up the reason for most of the complaints in this thread. That is: builders who prioritize marketing outcompete those who prioritize engineering quality.
Making something people want enough to pay for is very difficult; if you are trying to do that while also imposing a bunch of other constraints on yourself that have no clear marketing benefits, your odds of success go down a lot.
Engineering excellence can sometimes be used successfully as a form of marketing in itself and you could well pull this off, as your content is super engaging and well-written.
But I would suggest to every engineer to consider Chesterton’s fence. Why is all the winning software in every category slow and bloated and seemingly unconcerned with engineering excellence? Is it because everyone involved, the engineers that build it and the customers that buy it, are all technical philistine morons? Or is it because the products that win in the market prioritize… winning in the market, and all their competitors that don’t prioritize that for whatever reason fall by the wayside?
Here is how the main factor incentivizing this sloppy engineering behavior is justified: everything is cast in terms of making money. Revenue is good -- fine. Therefore no revenue is bad -- doesn't follow and you need to justify that somehow.
No revenue generally means the project eventually gets abandoned or relegated to spare time because people gotta eat. So yeah I'd say it's a bad thing.
I'm all for quality engineering. I'm just saying that if your whole plan for a new product is "quality engineering", then you have no plan at all. The horse needs to come before the cart.
Even couching the output as a "product" begs the question. It comes with a whole set of connotations around what's expected in and from such an effort.
What you're describing is a tragedy of the commons from the perspective of social and civilizational benefit. I agree that's what it is, but I think we should be more careful before justifying it.
That's fine for theoretical discussions. The poster I was replying to implies that their goal is to sell a product and make money, so I was speaking to that.
> But I would suggest to every engineer to consider Chesterton’s fence.
I agree.
In fact, I believe I have considered Chesterton's Fence; I have a plan.
I'd like your opinion on my plan.
> Making something people want enough to pay for is very difficult.
Yes, absolutely.
To fight this, I have spent years, 3.5 years, figuring out what people hate about build systems, and I have designed mine to address those.
While that doesn't guarantee success, I think it may improve my odds. Do you agree?
> That is: builders who prioritize marketing outcompete those who prioritize engineering quality.
Yes, and it sucks.
I hate marketers. I hate marketing. I hate the fact that I'm going to do it. But I have to.
So my marketing with be focused on giving people something for just paying attention.
> Engineering excellence can sometimes be used successfully as a form of marketing in itself and you could well pull this off, as your content is super engaging and well-written.
Thank you for your compliment! I hope so, and my plan is to emphasize this.
I'm going to post four articles on HN, each a week apart, and each will be designed to give readers something substantial, with a blurb about marketing at the end that will be clearly marked as marketing.
* The first will be new FOSS licenses designed to better protect contributors from legal liability.
* The second will be a post on language design and psychology.
* The third will be a deep dive into Turing-completeness and what it means.
* The fourth will be the source code as a Show HN, along with an offer for early adopters.
> Or is it because the products that win in the market prioritize… winning in the market, and all their competitors that don’t prioritize that for whatever reason fall by the wayside?
It's awful, but you are right.
So this marketing push is all I will do for four weeks: preparing and posting, and responding to comments. This is when I will "prioritize winning in the market."
It's a start for sure. But personally I'd suggest changing your mindset a bit from the idea that you'll come out of the code cave for a month to grit your teeth and do marketing. Granted, this is much better than just staying in the cave. But really to run a successful business I think you need to accept at a deep level that marketing, making money, and growing the business is now your main job, and you will permanently need to spend at least as many thought cycles on that as you do on programming.
To wit, while posting those articles and a Show HN sounds like a good plan and you should definitely do that, what will you do if they all fall flat and get no traction? It's a distinct possibility, and I hope you won't just give up.
I'd think more about what you're going to do every week for the next year to get users rather than putting all your chips on an HN launch that may or may not pan out. Even if you do rock the HN launch, you're probably going to have the "trough of sorrow"[1] to contend with after, so I'd think more about how you can make marketing a repeatable part of your rhythm in the long run.
> while posting those articles and a Show HN sounds like a good plan and you should definitely do that, what will you do if they all fall flat and get no traction?
You got me. I was planning on giving up. :P
I am in a place where the only cost to switching projects and trying again in three years is the opportunity cost, so because I'm bad at constant marketing, that's what I was going to do if I got zero traction.
If I got only some traction, I'd weigh my options.
I was only going to worry about long-term weekly marketing if the launch went well enough.
Because I'll be frank: I have no idea how to do constant marketing that doesn't bother people or waste their time. If I would waste their time, I'd rather just throw my own time away and switch projects.
> Even if you do rock the HN launch, you're probably going to have the "trough of sorrow"[1] to contend with after
Good blog post, and yes, I agree. I am expecting the trough as I build up the MVP.
The issue with your strategy imho is that failing to get traction with an HN launch is not that much of a negative signal. Getting to the front page is a bit of a crapshoot and not achieving it doesn't necessarily mean no one is interested in your product--it could mean you got unlucky or just need to iterate on your messaging.
If you haven't gotten it in many users' hands yet, it might be a good idea to try recruiting like 50-100 users first, either one-by-one through email reachouts or in smaller communities where it's less hit-or-miss, like niche subreddits. If some of these users like the product and stick with it, start giving you feature requests, etc., that tells you that you're on to something. Conversely, if you can't get even a small group of users to try it and stick with it using that approach, it's much more of a negative signal than a failed HN launch and probably indicates that something needs to change.
Whatever route you decide to take, I wish you the best with it!
Also:
"I have no idea how to do constant marketing that doesn't bother people or waste their time. If I would waste their time, I'd rather just throw my own time away and switch projects."
That's noble of you, but I would cut yourself some slack. Ideally you would market in a way where you don't bother people or waste their time, but getting users often requires trying things where you risk getting close to that line. Sometimes you might cross over it, but that's just something to learn from.
Trying to market a product while never bothering anyone the least little bit is a bit like trying to be a comedian without offending anyone or to find a romantic partner without enduring some awkward dates. I think it just goes with the territory.
I think if you succeed, it will be because other developers are investing in you personally. Because they like what you write and how you think, and thus trust you to write good software.
I think patio11 is someone who has used a similar marketing strategy to great success. People hire him because of the quality of his writing demonstrating his understanding.
> While that doesn't guarantee success, I think it may improve my odds. Do you agree?
Not your parent commenter but I wouldn't agree, no. Build systems are one of these things most companies want to setup once and want to never deal with them again, and only do so if they have zero choice.
`earthly.dev` had an article some weeks ago that addressed why one of their products never took off even though it's really good and addresses grievances in the space. The TL;DR was that existing customers of other CI/CD tools did not want to migrate to a new system, especially since it doesn't offer a migration from their existing solution.
> Why is all the winning software in every category slow and bloated and seemingly unconcerned with engineering excellence?
1. They are extremely aggressive about coming to market fast. That's a good thing, yes.
2. They then let the project ossify because they view development as a cost center. Which is a bad thing.
3. Their endgame after being successful enough and can afford enough lawyers, they start proactively buying or shutting down potential competitors.
The "marketing wins over quality (and slower) engineering" does apply, yes, but not throughout the entire lifecycle. It applies only in the very first phase.
>, but this is a big reason why I use C: I will have a hard time bloating my software.
Your perspective is interesting because I'm old enough to remember when the C Language was considered bloat compared to just writing it in assembly language.
Examples of 1980s programs in assembly was WordPerfect, Lotus 123, MS-DOS 1.0. SubLogic Flight Simulator (before Microsoft bought it) was also in assembly.
Back then, industry observers were saying that MS Word and MS Excel being written in "bloated" C was a reason that Microsoft was iterating on new features faster and porting to other architectures sooner than competitors WordPerfect and Lotus 123 because they stayed with assembly language too long. (They did eventually adopt C.)
I see this "bloat-vs-lean" tradeoff in my own software I write for my private use. I often use higher-level and "bloated" C#/Python instead of the leaner C/C++ because I can finish a particular task a lot faster. In my case I'm more skilled in C++ than C# and prefer leaner C++ executables but those positives don't matter when C# accomplishes a desired task in less time. I'm part of the bloated software problem!
is there not, a minor hint of irony here, that when I think of bloated software my mind goes almost immediately to everything Microsoft creates (except VSCode, which is somehow the most performant software they've created despite being written in a language that is theoretically slow)
C probably was bloated then. But today it’s the backbone of everything, whereas python will never be that, at least on hardware at the levels we can foreseeable build.
Languages like C brought a massive benefit to accessibility. Devolving “software is slow” to “yeah but C vs assembly” is such a ridiculous crutch argument. Assembly is not remotely approachable to the majority of programmers. C, rust, zig, c++, Java, C# are all approachable languages that are fast and have great fast libraries and frameworks to work with.
All I can see in the “I can finish the see sharp program faster” argument is that “python vs c++.jpeg” from the 2000’s where half the python was importing libraries, but they wrote the C++ from scratch, and everyone who knew nothing about C++ moved this image around like it was some hilarious joke of C++.
> Your perspective is interesting because I'm old enough to remember when the C Language was considered bloat compared to just writing it in assembly language.
Both perspectives are correct for their respective times. Compilers were much dumber in 80. These days your GUI desktop program written in assembly language would probably run slower than written in C and compiled with modern gcc O2.
Using Arch with GNOME, VIM/NEOVIM. Happy value between usability and resource usage. The programmers on Linux usually care about efficiency. Christian Hegert is improving recently a lot in GNOME. This allows me to use a ThinkPad X220 which is running circles around some modern laptops with Windows. I understand that people prefer the small footprint of C. I like C. My personal preference is C++ (low- and high-level, more safety and flexibility).
The problem is greed - i.e. capitalism. The blog mentions Electron and therefore Chrome and JavaScript. They are awful combination and allow companies to save on programmers. Programmers which handle C, C++, Rust or Python are a small group. Java, C# and JavaScript consume more resources, allow using a lot more stuff quicker (like a drug) and most importantly are forgiving mistakes. So the industry decides to waste the resources on our computers because they don’t need to buy and maintain them. The customer pays twice, for the software and the next computer with more soldered main-memory. The key point is - the software companies don’t pay for hardware or the environmental damage! I’m doing myself a little JavaScript and some Java. Efficiency? Nobody ask for that and I should not spend time on it. There is now law stating that managed languages need to waste resources but a side-effect.
Remember Steve Jobs?
I don’t appreciate what Apple does. But he banned Flash for a reason. Resources! Okay. Also bugs. And for the same reason they should ban Electron. Apple devices run faster with less resources because Apple saves on hardware. Why build devices with huge batteries, when you can achieve more runtime with less material and take the same money? The EU needs to enforce side-loading on the iPhone. But I also think Apple should be allowed to keep Blink (and therefore Electron) banned from AppStore. I don’t want see my battery dying because some corporate manager decided to drain it. But if you need Blink? Side load.
High keyboard focused usage with focus on the application-window. I specifically like to enter just “terminal” and the terminal open, either a new window or already open on. I also Alt+Tab but with many open applications finding is more convenient. Another thing is that it is also easy to use with the mouse and the overview makes that pleasant (good for novice users).
They eliminated the unappropriate “desktop metaphor” from Windows 95 and the “system tray”.
The negative side is that GNOME often removes options or hides them - which drives away experienced users (the people needed to pull in new users) and some developers to forks. The are right to not support every bewildering option but the needed ones must in place despite the UX people don’t use them itself.
Some of this is right. Do stuff automatically right and don’t provide unneeded options to avoid complexity. Some not! Also provide required preferences e.g. “Do not suspend on LID close” because some people don’t want that! The GNOME people assumed it was needed as option because of problems with suspend (S3) in the past. Maybe people used it to bypass issues but that isn’t the use case of the preference. Another thing is “When you’ve five clock applets” I want to know what is missing from the first one which made all the others necessary. GNOME learned that people don’t need an “Emacs” as UI-Shell but somewhat went into the other extremes.
PS: And GNOME just looks good by default. I don’t need themes because it is fine.
Jobs also said native apps were unnecessary and we can all just use web apps. (Despite working on an app store at the time.) So even the prophet himself isn't such a great example.
I think there might be a market mostly with other engineers like you. Lots of people seem to really love this stuff, look at the forth crowd and how much they adore the language.
For me... I don't notice any slowness or excessive battery drain in VS Code, so I just use that. Perhaps there's something deeply wrong with my psychology, but I'm just not very interested in simplicity for its own sake. It's cool, but not an approach I'd want to use every day in real life.
I wonder how much of this is just because when a project manager comes along and says "as a system administrator, I want to be able to log all of the keystrokes of the users and reports who's slacking to the boss", you say, "ok, that feature will take about two weeks to implement" and they can't argue with you because it's C so they just go away and leave you alone.
I think I lost the lede when the author implied text editors were simple and easy. My understanding is that text editing uses sophisticated data structures and is actually a really difficult problem to solve. Especially when combined with lock free replication used in multi-user editing. This is not easy or simple and comparing it to a terminal editor felt disingenuous.
I can relate to the disenchantment but I found the argument here to fall flat.
Yes 100%. Plain text editing, especially with any sort of highlighting or parsing, is hard. But I'm also struggling to think of a common and modern "text editor" with high latency like that. When I think "text editor" I think of something only capable of saving and loading plain text, with some plugin support. Stock VSCode is exceptional at this. So is neovim. So is Microsoft Notepad. So is Sublime Text. So is Nova.app.
He has SOME editor in mind, I'm just curious what it is.
> lock free replication used in multi-user editing
I understand that author was referring to the concept of a minimal version that does just text editing and nothing more. Add features to anything and it will stop being simple and easy.
However, the author might also be generalizing and referring to "MS Word" as a text editor, which a lot of people unfortunately do. This is where I would disagree with the author's premise.
You can only pick two out of the three from the triad of optimization (time, monetary cost, quality).
Industry prioritizes functional solutions (requirements) over efficiency. If efficiency is one of the requirements, it will be addressed (e.g. video games). Optimizing for efficiency takes additional effort. The article argues that the software industry is stuck with inefficient tools and practices. Engineers can and should do better, aiming for better apps, delivered faster and more reliably with fewer resources. However, economy dictates that as you optimize two variables from the triad, the third will get de-prioritized.
Edit: I can see the downvotes but no idea why. Would you care to explain?
>I can see the downvotes but no idea why. Would you care to explain?
I dunno, but this has been taught in PM since PM. I remember learning it in the 90s and it still holds true. If you want it fast and a large scope, you have to throw bodies and planning at it, costing money. If you want it cheap with a big scope, you have to wait for that small team to finish it (time). If you want it fast and cheap, you have to limit your scope.
Time/Cost/Scope, pick 2. Perhaps people are taking issue with quality vs scope, but quality is a part of scope for certain.
I don't think this can even be blamed on what the industry prioritizes, but on what customers reward. How many users pick one solution over another because it's quicker? Outside of very specific use cases practically never happens. These types of complaints ultimately come down to wanting users to care about different things. It's not dissimilar from complaints about nobody having fashion sense anymore or buying too much processed food.
Yes, the engineers who verbally bat for efficiency/correctness in these threads have no qualms about practically abandoning efficient sublime to less efficient vscode (in general, of course; not that there aren't exceptions) :)
> How many users pick one solution over another because it's quicker?
Isn’t that what drives the most sales in upgrading a phone? Idk many people that care about the microscopic improvements to the camera or UI, most of the time it’s something along the lines of “my iPhone 8 doesn’t run fast enough anymore, time to upgrade”. Same with consoles. I remember one of the big pitches of next gen consoles being “look! No more loading screens!”. And even ChatGPT 3.5 vs ChatGPT 4. There are tons of people that will use crappier output because it’s faster. Speed is still absolutely a selling point, and people do care.
> I remember one of the big pitches of next gen consoles being “look! No more loading screens!”. And even ChatGPT 3.5 vs ChatGPT 4.
Those are the very specific use case I mentioned.
How many users are gonna swap to a different chat client because of this or a different word processor because it start 1s faster? As a parallel comment points out, even developers swapped away from the very quick Sublime to Atom and now VSCode. At work hiring at some point became a huge pain because we swapped from Greenhouse to the recruiting tools built into Workday. It was super slow. I hated it, our recruiting team hated it, but it was purchased because it checked all the boxes and integrated with all our other stuff in Workday. The comparison to engine efficiency made me think that if we really want faster software, we need the equivalent of a gasoline tax for software, but what's the negative externality we are preventing?
I think you’re right that there’s a fundamental tension between those three factors but it’s begging the question of whether we’re seeing those fundamental limits versus something else. For example, I’ve seen multiple teams deliver apps using React frameworks which have no advantages in any of those points - they’re notably slower than SSR, use more resources on both the clients and servers, and don’t ship noticeably faster because while some functions are easier that’s canceled out by spending enough time on toolchain toil to make a J2EE developer weep.
That suggests that while those three factors are part of the explanation they’re not sufficient to explain it alone. My theory would be that we’re imbalanced by the massive ad-tech industry and many companies are optimizing for ad revenue and/or data collection over other factors such as user satisfaction, especially with the effects of consolidation reducing market corrective pressure.
How much programmer time has been wasted by gdbs byzantine UX? In a rational world, would we collectively invest time into building extremely good debuggers?
It's a tragedy of the commons, where everybody would benefit from better tooling, everybody wastes time dealing with poor tools, but nobody is willing to put time/effort/money into making better tooling.
From what I've seen the faster solution is also usually the simplest, which makes it faster to deliver and easier to maintain. We're far away from what Knuth warned about (micro-optimizing assembly). It's more at the level of high-level design, technology choice, etc.
Like at my last job, we had something like 50 microservices, Kafka, rabbit, redis, varnish caching, etc. For under 1k external requests/second and some batch processes that ran for a few million accounts. If we cut out all of the architecture to "scale", you could've run the whole thing on a laptop if not a raspberry pi. And then a real server could scale that 100x.
The company was looking at moving to a "cloud native" serverless architecture when I left.
Agreed. My argument is that software from market/customer perspective is more like a movie, which is supposed to delight them. And industry rightly priorities the capability/delight factor over mere efficiency for efficiency's sake. Only engineers are interested in efficiency for efficiency's sake (also - will the engineers pay for more efficient software? Nope. Why did engineers abandon Sublime for vscode? So when anyone wears a customer hat, they tend to prioritize factors other than efficiency/correctness more unless we are talking about life-saving medical saving devices or such).
The impossible trinity (also known as the impossible trilemma or the Unholy Trinity) is a concept in international economics and international political economy which states that it is impossible to have all three of the following at the same time:
- a fixed foreign exchange rate
- free capital movement (absence of capital controls)
- an independent monetary policy
Programming languages. Developer tooling. Debuggers. Cross platform libraries. Everything could be orders of magnitude better than what we have today. But our world is ruled by pragmatism, and duck-taping mediocre technologies together results in barely "good enough" solutions that make money and solve real world problems.
Even open source suffers from the same problem. Making really high quality software takes 10 times the effort (if not more) and nobody is willing to make that investment up front. And so half-baked software becomes popular and now we spend years or decades and untold hours trying to turn bad software into pretty good software without completely breaking backwards compatibility.
The root cause is probably the incrementalist approach we take to developing software. We start with something small because of time constraints or because we don't really understand the problem yet. Once the software has users who demand bug fixes and feature additions and this results in software organically growing instead of being intelligently designed for a purpose. As a result, you get stuck in a local maximum.
And that's how we spend 500,000 hours debating strategies for removing the GIL (global interpreter lock) in Python when the first version of Python was made 30 years ago as a side project during a holiday break.
> The root cause is probably the incrementalist approach we take to developing software.
I would blame the "unix philosophy" and "worse is better" approaches of the past, but I bet they were more symptomatic than causal, and their equivalents in other digital realms pop up all the time: IBM vs clones, unix wars, protocol wars, at various times its fights between 'official' (described as stuffy) vs 'pragmatic' (described as lax/crappy) definitions of stuff.
I'd hazard a guess that since (for those of us who are young and therefore spew confident sounding incorrect speculation like this comment) we still have so much of the old 'it works, ship it' hacker groups of the 70s in our past, then the overreach of the CASE / UML / XML fever of the late 90s which we have in turn overreacted against by going too far in the 'look its a containerized k8s pod running behind a reverse proxy that runs some react and uses leftpad to graphql your (must always be online) user information record in this headless electron because SHIP IT' direction.
PS: our historical 'its good enough' precedent didn't help, we've been trapped on 'very fast PDP-11s' for decades. Even BWK and the other forerunners of our modern C + unixlike stack weren't able to get us un-stuck from that stack and so plan 9 etc. failed to catch on. The Lindy Effect is a double-edged sword for sure.
This makes me wonder, having seen another thread on this recently, how much of this is due to layer after layer of abstractions piled onto one another. Given the demand for backwards compatibility, & the never-ending efforts that show up to port everything old to any new platform (with all the support that has to be baked in to make that happen in many cases), how likely is it that this can be avoided, or prevented from getting worse in the future?
You can run DOS applications on any computer architecture and on any operating system. Modern software is much worse at backwards compatibility than software from the past.
The reason for that of course is that DOS stopped changing almost 30 years ago. But, yes, Dosbox and other DOS emulators are great virtual machines if you do not want to have to rewrite your software every six months because some API changed, and they run almost everywhere.
I remember a few years back when a hacker decompiled the GTAV PC loading sequence and found the software loading ~ 1GB json file in memory to read a single param, which added an additional 2-3 min to the load.
Let’s keep the standard for performance high and continue to highlight performance issues.
This article doesn't even touch on the main pain point: bugs. Virtually all software is just as buggy as can be. I dread every time I need to take a piece of software down a non-happy/non-common path. It almost always fails. Working around and dealing with bugs is just a normal, every day part of modern society now.
Simple example, I sold my car to Carvana the other day and just baaaarely pulled it off using Chrome and Firefox. In Chrome the upload image wizard would get a JS exception. That part of the app miraculously worked in FF, but virtually the entire rest of the site was mired in issues as it's obvious Carvana devs don't test in FF. I pulled off the transaction by bouncing between the two.
Even worse, most non-technical people think they did something wrong when they encounter a bug.
Software that is bloated and slow but stable and rock solid? I'd gladly take it at this point.
On one hand I cannot believe what we have actually works as well as it does. Duct tape and bubblegum everywhere at all levels and it actually still works. I’m amazed everyday at what humanity has accomplished with that in mind. On the other hand I can’t help but notice just how damn buggy everything is, and I’m not sure if everything is actually more buggy or if I’m just losing patience with big business software development as I age having seen how the sausage is made. I can’t help being angry at some illusory product person telling people to ship feature X or else with it having a glaring bug in the UX that is easily caught.
It can be hard to tell what is "duct tape" and what is "speed tape" in software these days. An aircraft mechanic doesn't need to know the structural differences between the two - just use speed tape to be safe. Similarly, a pure programmer doesn't have to know the difference between ufw and Windows Firewall - just use the latter and move on.
However, an engineer (mechanical or software) better understand the differences in both situations
You're 100% right. Ever since watching the Jonathan Blow talk on the end of the world I have started paying attention to it and it's amazing how many absolutely shitty software experiences we just accept on the daily.
I understand all the points about financial costs and opportunity costs and pragmatism and the rest, and I partially agree with it, but it's hard not to sometimes feel like we've accepted living in a half built world.
I disabled my ad blocker and cleared out all caches and it still persisted. Even if it was, the JS exception was unhandled. Carvana is built in React, the absolute bare minimum would be an ErrorBoundary at the top of all flows (in this case, a modal).
That may be true. But software is also much more ubiquitous and essential these days. At least in the 90s you could be confident your car wasn't buggy.
Yeah, and I shit you not, many vehicles have had software updates to solve oil consumption issues. People walk away from those service encounters just shaking their head wondering how in the fuck software was causing their car to lose/burn oil. Truth is, several different reasons.
The number of vehicle software updates are staggering.
American cars in the 90s were terrible. The domestic automakers wouldn't be in business today if people actually bought based on reliability. Now you can buy almost any brand and reasonably expect it to last over 100k miles.
When the networking stack got worked out by Windows 98, everything was pretty smooth sailing. Until the drive-bys started. Then we moved to Windows NT and complexity, and albeit security, increased significantly. Windows XP was stable as fuck until the root kits started flowing. We moved through Vista, hopefully avoiding it all together until we landed at the fucking dream that was Windows 7. I would have stopped right there and been happy.
Then something got pushed over the cliff. Windows 10 and the UWP...like seriously WTF!? Satan's very own dumpster fire.
Agreed. I remember as a teenager when the first generation Macbook Airs came out, my friend realized you could reliably crash them (brand new, the demo models sitting out at the mac store) just by opening all the apps on the dock in quick succession. Took just a few seconds of clicking and the machine would crash and reboot.
And it's not like things have gotten any better, now I'm an adult with young children and if one of them gets their hands on my phone or laptop, they seem to be just as reliably be able to lock up, crash, freeze modern devices all the same. These kids are not physically damaging the phone in any way, they're just pushing buttons too quickly or in unexpected orders. That's the state of modern tech that we're in.
To be honest, I can understand why this hasn't been fixed. As mentioned in the article and throughout these comments, we've just come to expect to need to restart every now and again. In this case, people are likely to blame such a problem on the child, and a restart fixes the issue anyway. I just would've hoped for better by now in the lifecycle of these operating systems.
But if the software keeps getting modified and expanded it will not be stable. Bugs will be added. And that also causes bloat. I think they go very much hand in hand (and very much correlates with the frequency of upgrades... usually upgrades that no user asked for anyway).
Not sure I agree with that. It's possible to add new features with minimal or even no bugs. It requires good engineering, good test coverage, and often good manual QA and even alpha/beta rounds. But it can be done. Is it done today? Sure, some organizations absolutely follow this approach. I would guess, especially in the commercial space, these things are often lacking.
I've been working on an idea recently: "low interest rate architecture."
High complexity, ugly / unplanned, high rent (cloud costs, etc.), bloated, only polished on the surface, and designed to gather as much market share as possible as fast as possible and we'll fix it "later."
Examples: Electron, Kubernetes, Helm, expensive managed cloud, lots of domain specific languages and other high cognitive load systems demanding high staffing requirements, total abandonment of labor saving tech like WYSIWYG design, etc.
This is why I'm an embedded programmer on small MCUs. Give me C99 and a datasheet and I'll give you the world in 64kb.
Though times are changing in that world too. Sometimes you have to use a library. And more and more those libraries require an RTOS. Just about to make the plunge into Zephyr so I can use the current Nordic BLE SDK.
Having hard limits to RAM and flash is a great way to prevent bloat. Management is happy to let engineers grind for a month reducing code size of it means the code will fit into a cheaper MCU with less flash and save a 10¢ on the BOM. Pure software has no such incentive to minimize resources because the user buys the HW separately. If anything, some SW companies have an incentive to add bloat if they're the same company that sells you a new phone when your old one becomes too slow.
It’s a trade off. Would you rather have more software with more features or less software that runs fast and efficiently?
The market has clearly spoken in favor of the former.
After all, the goal of engineering isn’t to produce some abstract beautiful art but rather to create something valuable for regular humans.
Also the car and building analogies are poor because they are essentially the same design that has been optimized for decades. Indeed the only “new features” in cars are usually software driven and probably just as inefficient.
Has the market spoken? I agree that it's largely true but I am not aware of anyone selling software with an emphasis on performance. It's always been about new, shiny features. You can't really say the market decided if one half of the equation isn't even on the market
Demand software with features and performs efficiently.
If anything, the last several years in the software industry have proven hordes of mediocre developers really isn't a substitute for understanding how computers work.
The phenomena we're seeing now is VC money has dried up for all but those focused on efficiency, and the rats are scrambling to run to the other side of the ship before it sinks.
Except, in my experience, they don't have more features. Has Slack fundamentally changed in the last years, or Zoom, Teams etc? Why haven't these apps been optimized by now?
Same for web apps, why is Facebook, Gmail slow? I could accept it for new products but not for nearly unchanged products after all these years.
Btw: Gmail HTML was fast, so it can't be an infra problem, it's bad frontend programming.
> It also animates empty white boxes instead of showing their content because it’s the only way anything can be animated on a webpage with decent performance.
That's not the only reason, although it is one. Two of the other big reasons for this are:
1. The content is often loaded async while the boxes render, and this hides the latency of doing a network fetch.
2. Animating text looks terrible. Even if you had a perfectly fast computer, you wouldn't want to animate boxes filled with text and images, because it looks really bad.
I've noticed a similar trend in my current organization. In the beginning, when we were smaller, the details mattered. Things couldn't be slow, animations had to be smooth, scrolling speed mattered, loading times mattered. We aimed to build the most efficient product, anything to increase user efficiency.
Then as the team grew, the values changed to favor anything that improved developer efficiency. More abstractions, more layers, more frameworks. The tradeoff of saving one day of developer work was worth the cost of millions of user seconds collectively. I think the difference was just the visibility - management can see the costs of things on the development side, but they can't see the benefits of a slightly faster launch time, or better caching, or smoother scrolling. They aren't measurable, and once an org gets to the point where all it cares about are measurable numbers, I think this is a natural course.
Now just wait for the deplorable impact of LLM on code quality and performance in the years ahead. There's a new wave of programmers who are "GPT whisperers" and spend most of their time hooked to a chatbot (sometimes right in their IDE) that programs for them.
Of course, that's until AI gets to the point where it can fix everything we (or it) programmed wrong, including AI itself which is highly inefficient just like the OP predicts.
I don't think it's a given that LLM-driven coding is a negative for code quality and performance. In my experience coding with ChatGPT, it often reminds me to think about error handling, edge cases, and performance issues.
It also over-prioritizes readability if anything, using descriptive variable names and documenting every line with a comment. It's maybe not as good as the best engineers at considering all these factors, but I'd say it's better than the average developer, and may be better than the best engineer too when that engineer is tired and in a hurry.
It's not just about being measurable, it's about being profitable. Capitalism results in efficient production, not efficient products, because it's only interested in extracting profit from the production system.
Cars were optimised only after external (oil) shocks were applied, and even then very unevenly (American cars continue to be very inefficient). In fact, inefficient products are often more profitable in practice, as customers have to replace them more often; the crappy iPhone cable that splits after a year means Apple can charge you again (and again) for its replacement. White goods now break more often, but they're built more cheaply so they can make per-unit profit higher than it would otherwise be. What matters is efficiency producing, not after-sale use.
Capitalism in software means churning out new automation as quickly as possible, letting consumers pick up the resulting waste of energy and time. The production chain gets more and more standardized and optimized: you can now swap React developers, or Kubernetes admins, like you can swap warehouse workers, with all that it entails in terms of salary pressure. Some of that automation is effectively self-justifying, in the same way accountants make accountancy terms obscure so they can justify their jobs; but that's about it. Everything else is about profit.
429 comments
[ 4.6 ms ] story [ 300 ms ] threadA few previous discussions about this blog post:
https://news.ycombinator.com/item?id=18012334 (Sep 2018)
https://news.ycombinator.com/item?id=21929709 (Jan 2020)
https://news.ycombinator.com/item?id=31798580 (Jun 2022)
I agree with the author's call to action though:
"As engineers, we can, and should, and will do better. We can have better tools, we can build better apps, faster, more predictable, more reliable, using fewer resources (orders of magnitude fewer!). We need to understand deeply what we are doing and why. We need to deliver: reliably, predictably, with topmost quality. We can—and should–take pride in our work. Not"
Not if we have project managers we can't.
The problem is there are engineers who will take your paycheck and build crappier software more quickly, because there's barely any quality regulation for software. That's the difference between software and other traditional engineering disciplines. An electrical engineer has their license at stake.
You’ll usually pay them better than the engineer who delivers a better quality product in more time.
Every divorce starts with a marriage. Any delay starts with a planning.
Yeah, but I'll bet those car designers didn't close the requisite number of story points in their sprint! So really, who's laughing now?
He uses a car analogy (how much efficiency we squeezed out of cars vs software) but misses the point - if a car was available that consumed half the gas or went twice as fast, I’d buy it!
If a computer / OS was available to me that was so much faster / more efficient than what I use… turns out I don’t really care and neither does anyone.
I am writing this on a 6 year old iPhone X that I got after my wife upgraded to the iPhone 15 at the same time my toddler drowned my Pixel 6 pro. The iPhone X is noticeably laggier than the 15 or the pixel but it doesn’t matter enough to go upgrade. And ditto for my 10+ year old PC. Point being, if I as a consumer am not bothered by the perf I get to even just drive down to the store, manufacturers would be dumb to over index on it
Obvious this is “just me” but if people really cared about a phone that boots in a second instead of 30(his example) there would be a market for it but the fact is nobody reboots their phone and it doesn’t matter. On the other hand if I had to wait 30 sec to drive my car and another manufacturer reduced that to 1, I’d switch.
TLDR - yes it’s inefficient and no it doesn’t matter. Those old systems he’s talking about were “efficient” because they had to run on hardware of the day, and increased requirements meant decreased market size - ie it really mattered! I am sure someone bemoaned the bloat of win95 that couldn’t run on his beloved 286.
Efficiency is one dimension. If I couldn't run calculator on anything but the latest Pixel / iPhone, that would be clearly insane. But it's not the sole dimension - if my ambition is to create something that works well for millions or billions users, hand-crafting Assembly may not be the right tradeoff.
Look how bad the self checkout experience is. You might say “if people cared, they would shop elsewhere”. But people want to buy food more than boycott a store with bad software.
I would say that bad checkout matters - in the way that I am talking about mattering - because it does effect a non-trivial amount of behavior.
First, I am much more likely to opt for self-checkout at Whole Foods vs in my town's supermarket (that I otherwise like) because the WF experience is pretty smooth while at the other store, the machine blocks until help arrives every X items. Does that matter? Well I think so, because (1) some # of people will go to WF over this store just because of that (2) some # of people will stand in a cashier line - costing the store money - because of this (3) some people will opt for delivery because of this - undermining the "local store" value proposition in the long run.
The above impact exists because in the bad case, self-checkout is nearly unusable. If it was just a matter of every item scan taking 2 seconds instead of 1, or needing to get human help once a year instead of several time per checkout - nobody would care.
Your reasoning assumes a fluid system with perfect observability into everyone's interests. Consumers can't even articulate their own interests fully -- it's impossible for vendors to respond properly.
Quality and high performing software exists when vendors elect to produce it. There are intrinsic motives and extrinsic.
Most quality products exist despite consumer demands not because of them. Don't fall for the "money incentivises everything" trap.
Can you give me an example of where this exists separately from "and we can sell more units before its performant"? I am open to that being the case, but wasn't able to think of a good example. Can you?
It bothers me often means "it matters to me."
Sometimes it is a signal of something bigger. Other times... it's just me. And it is hard to know the difference.
Everyone needs a car capable of reaching 120mph right? Moms and dads need giant utility vehicles to feel safe on the streets surrounded by giant pickup trucks. How is it "efficient" to drive trucks that are 2-3x the curb weight with less cargo capacity?
The problems occur due to business decisions not engineering
So, I play the original one in DOSBox-x. Maybe it is nostalgia, but I love vibrant pixels and the Sound Blaster music.
Can Electron not "tree-shake" parts of the browser engine that are unused? Either by static analysis or manual configuration? Seems like a real missed opportunity to trim down on bloat...
According to Google, a Chromium build on normal hardware takes 6+ hours.
And alas, even if it was feasible to custom build based on what you need, it would have to be done via configuration--since there's no way to know at compile time which language features will be used, since your app could (and probably does) include remote scripts.
Yeah that sounds about right - I use a chromium on Android fork and according to the lead developer, it takes about 3 hours for a release to compile and that is after optimizing the process as much as possible.
And in the web browser, I believe v8 is using snapshots of the js runtime to avoid much of the work of initializing the js side of things: it just forks a new copy of itself.
This is one of the prime strengths of alternatives like Tauri, that use a shared library model rather than having a static library. With Electron you have a ton of initialization to do for a very excellent runtime, but then you never get to reap that existing work again. Where-as on the web, we open new pages and tabs all the time, and avoid the slow first load & get to enjoy the very fast latter instance loads. That multi-machine capable vm pays off! With Tauri, the shared library may well already be in memory and initialized. Only the first consumer of the shared library has to pay the price.
From what I could tell of the config files, Chromium is pretty modular. It looks like you could just delete a few lines and have it avoid compiling entire subsystems. But I didn’t ultimately achieve my goal because I couldn’t get it to compile with those changes. IIRC it hit a linker error and I couldn’t figure out what to prune next, and I wanted to get back to actually building my product. (I ended up switching to Tauri anyway)
Part of me wants to revisit that project though. It would be so great if there were custom minimal builds of Chromium for Electron apps.
The main reason for this is politics. The Chrome team is ideologically wedded to the idea that everything should be a web app running on Chrome. They see desktop and mobile apps as the "enemy" to be wiped out by making the web do more and more stuff, until Chrome is the universal OS. Classical ChromeOS is the pinnacle of this vision - a computer that only runs a web browser and nothing else.
Chrome's architecture reflects this, um, purity of vision. It is not reusable in any way and the Chrome team do not care to make that easier. Projects that make it embeddable or reusable are all third party forks that have to maintain expensive patch sets: CEF, Electron, etc, all pay high maintenance costs because they have to extensively patch the Chrome codebase to make it usable in non-Chrome apps. The patches are often not accepted upstream.
This problem also affects V8. Several projects that were originally using V8 are trying to migrate to JavaScriptCore or other JS engines because V8 doesn't have a stable API and building it is extremely painful. It's a subsystem of Chrome, so you have to build Chrome to get V8.
This is a pity. There's a ton of great code in the Chrome codebase, and it can be built in a modular way (with millions of small DLLs). It's slower to start up when you do that due to the dynamic linking overhead, but it does work. Unfortunately for as long as the Chrome guys see native apps as a bug to be fixed and not a reality to be embraced, we will continue to have dozens of apps on our laptops which statically link an Xbox 360 gamepad controller.
There are a few possible solutions for this.
One is to not write Electron apps. Java went through a modularization process in version 9 and since then you can bundle much smaller subsets of the platform with your app. I guess other platforms have something similar. Obviously, native apps also don't have this issue both because the platform comes with the hardware you buy and because they tend to be more modular to begin with. But, people like writing Electron apps because it gives you the benefits of web development without many of the downsides (like the ultra-aggressive sandboxing). The nature of the web platform is that it always lags what the hardware can actually do by many years due to the huge costs of sandboxing everything so thoroughly, whereas Electron apps can just call into native modules and do whatever they need, but you can still use your existing HTML/JS skills.
Another would be for an alternative to Electron to arise. There are experiments in using system WebViews for this. That doesn't make the web platform more modular, but at least means it's a single install is being reused. You could also imagine a fork of the web platform designed for modularity, for example, in which renderer features are compiled out if you don't need them or even bringing back renderer plugins.
Another is to just tackle the issue from an entirely different angle, for example, by opportunistically reusing and merging Electron files on disk and in memory between different apps. If you ship your Electron app to Windows users using Conveyor [1] (or in the Windows Store using MSIX) then this actually happens. Windows will reuse disk blocks during the install rather than download them, and if two apps are using the same build of Electron the files will be hard-linked together so only one copy will be in memory at once at runtime.
But fundamentally the issue here is one of philosophy. Chrome wants to rule the world and has a budget to match, yet their approach to platform design just does not scale. For as long as that is the case there will be lots of complaining about bloat.
[1] <...
I'm not surprised, and doubt the Firefox team is any different in that regard.
So far at least the Chrome team hasn't removed from Chrome the ability to go to a new web page in response to code external to Chrome, so we have that to be thankful for at least. Yay?
(The desktop code I maintain achieves that effect by invoking /opt/google/chrome/google-chrome with the desired URL as an argument.)
And yet, full disclosure and admitting to cognitive dissonance, for a (hobbyist) C++ game engine I'm currently working on that targets Emscripten for a web build and native for Debug build, I'm considering not even having a native-Release build at all.
The idea being if it's web-first and the native build being only for developer use for debugging, I could do things like supporting only one desktop graphics API (e.g. just DirectX) and optimizing the native graphics pipeline for simplicity over performance. End users would/could just use the web version.
Granted this is a bit different because I wouldn't be distributing a browser too a la Electron; it would just use the browser the end user already has. Just thought it's interesting that it's easy for me to criticize others, but with the choice of how to spend my own limited developer time (my free time) it's looking like this way of doing it makes the most sense.
But if something is proprietary and cloud linked anyway I would much rather it all go through the web. That way open platforms still have a chance.
If banking apps and and Google payments and proprietary IoT devices controllers were all on the web, then a Linux phone might actually be viable!
I'm guessing eventually the native free software would move more and more into the browser too, but that's fine as long as you can still run the stuff that hasn't been moved yet or isn't interested in moving.
For many years I worked on my own game engine which combined a C++ core with an interpreted scripting language. I had developed a system of language bindings which allowed the interpreted language portion to call functions from C++. The engine quickly grew to a multiple-gigabyte executable (in the debug build), and no matter how much I tried to optimize the size it was still unconscionably huge.
One of the reasons I eventually gave up on the project was I realized I was overlooking a simple mathematical truth. The size was NxM, where N is the number of bindings and M the size of each binding. I was focusing on optimizing M, the size each binding added to the executable, while not just ignoring N but actually increasing it every time I added bindings for a new library I wanted to call from the game engine.
There were diminishing returns to how much I could improve M because I was relying on compiler implementations of certain things I was doing (and I was using then-new next generation C++ features that weren't well optimized); it would be a lot easier to simply reduce N. And the easiest way to do that would be some sort of tree shaking.
Unfortunately due to the nature of interpreted code it isn't known at compile time which things will/will not be ultimately called. That determination is a runtime thing, by calls via function pointer, by interpretation of scripts that start out as strings at compile time (or even strings entered by the user during runtime).
From a compile time perspective, static usage of every bound function, feature or library already exists - it is the C++ side of the cross-language binding. That's enough to convince the linker to keep it and not discard it.
In fact, the mere presence of the bindings caused my game executable to grow more per each included library than would a similar C++ -only, all-statically linked program. If a library provided 5 overloads of a function to do a similar thing with different arguments, an all- C or C++ application that uses only one of them would need only include that version in the compiled executable; the others would be stripped out during the linking step.
Since I don't necessarily know ahead of time which overload(s) I'm going to end up using from the interpreted language side of the engine, I would end up binding all 5. Then my executable grew simply from adding the capability to use the library, whether or not I make use of it, but moreover if I did use it my executable grew even more than an equivalent C/C++ - only user of the library because I also incur costs for all the unused overloads.
You can see why something like Electron would have the same problem. Unused functions can't be automatically stripped out because that information isn't known at compile time. To do it by static analysis the developer of the Electron app would have to re-run the entire build from source process of the Electron executable to combine that with static analysis of the app's Javascript to inform the linker what can be stripped out of the final executable.
And it bears mentioning neither such a static analysis tool for Electron app Javascript nor the compiler/linker integrations for it currently exist. In theory they could exist but would still have trouble with things like eval'd code.
Manual configuration would be possible but necessarily either coarse-grained or too tedious to expect most developers (of Electron itself or users of Electron) to go into that much detail. That is, you may have manual configuration to include or not include Xbox 360 controller, but probably not for "only uses the motion controls" while not including other controller features.
Either way you wouldn't be able to add-back support for it Javascript written after build time turned out to actually need the function or feat...
A lot of engineers forget that they are in fact paid to build a product for customers.
edit: It also seems OP has never had to wait for older Windows or Linux machines to fully boot. Modern versions boot much faster because customers wanted that. Phones are on 24/7 so customers don't care if it takes longer to boot once every few months.
My Commodore 64 from the early 80s booted to a full BASIC REPL in around 1 second. Yes, loading from removable media was slow, but the computer was ready to fully use in less time than it takes "modern" systems to even POST. Totally ridiculous.
My BIOS takes like 20 seconds to POST, and that's apparently normal. (then ~10 seconds for the OS to boot)
[1] https://en.wikipedia.org/wiki/DDR5_SDRAM
By your logic Tesla and Apple shouldn't have grown like they did since customers wouldn't pick them over existing incumbents. Customers do have a choice in aggregate and they express that choice. Some people who don't agree with that choice try to say customer's have no choice but in the end they do and the person saying it is upset they're a minority.
They have enough power to shape the landscape itself, and as such they can drastically steer the outcome of "customer needs"
Neither of them used to be the giants they are but overall they made solutions that their customers (who are, btw, advertisers in Google's case) wanted and bought over larger existing competitors. Apple's market cap has increased 500x since the late 90s.
Obviously they entered the market with simply good products, but that doesn't change the fact that now they have a lot of power that comes simply from their size. Im not saying its their only advantage, just that it plays big role.
But also, common now. Today's platform lockins are much more impactful then anything we had in the 90'. And you don't compete with Google's product anymore, but rather you compete with ecosystem that half of the world bought in. Which makes it practically impossible to be a viable alternative
In a way yes but I think in many ways software became much more than that. In many ways it became a crucial part of human lives as electricity or medical services. And you can say doctors/engineers are being paid to develop new medicine or a new power sources but these have many many levels of government control just because how sensitive and important the matter is, which is not the case with software (yet?).
If you're going to criticize the state of software, you have to:
1. Demonstrate you understand the systemic incentives in place that encourage this behavior,
2. persuasively argue why this condition needs to be improved,
3. and propose concrete policies or incentives that counteract these tendencies.
Otherwise it's just whining about something you don't really understand, and the proposed "solution" is to just start all over because this time it'll be different I swear.
> The software industry is arguably one of the most productive spaces that the world has ever seen.
I would argue based on personal experience that majority of software that comes out of the industry is unmaintainable garbage. The reason it is acceptable in most cases is that the barrier to entry is low and there are few consequences of things going wrong. Quality goes up significantly when consequences become real. At any rate, I would not call the software industry "productive".
Then it is acceptable. You're making a moral value judgement because it's not up to your personal standards, but that's not relevant for whether or not the software accomplishes what it set out to do.
> At any rate, I would not call the software industry "productive".
This is indefensible, given that basically the entire world runs on software these days. Again, I think you're making some sort of subjective value judgement about the perceived "quality" of the software, while ignoring what it has enabled.
It obviously hasn't, there is still so much low hanging fruit everywhere.
And don't think this problem is unfixable, we will fix it at some point via standardization and each whiner is a data point that helps us get closer to standardization. It happened to cars, planes, boats, tanks etc, those fields started out with a lot of whacky ideas that didn't fit but then designs got standardized and the only thing left to do was to optimize those standard designs.
Edit: But for example, whining gave is git, a fast standard version control system that everyone now uses. When enough people whine about something then it gets fixed, bigger problems needs more of it.
If your argument is that things are more complex then they need to be, prove it. Because it's like an old man looking at a Honda and going "things were so much simpler back in my day!!"
Why not? This is part of the problem I'm talking about: we have too many armchair software engineers that know nothing about extremely complex codebases making sweeping generalizations about how they "should be". Their solutions are invariably "let's get rid of most features" or "let's just start over and Do It Better This Time."
I don't work for anyone right now, but I do have a "work" machine. This machine is beefy.
But I still run Neovim and tmux instead of an IDE. [2]
I don't run a typical Linux distro; I use a heavily-modified Gentoo, and that includes using OpenRC over systemd. [3]
I don't use a full desktop; I run a TWM called Qtile. [4]
All of this is so my machine is not bloated. When my machine boots up, and I just barely log in, it is running only 40 processes (including the terminal and htop I use to check).
As of right now, I'm engineering software. Truly engineering; I am spending the effort to effectively mitigate all of C's problems, while keeping the software lean and fast. I hope to someday build a business on that software.
I guess I'll see if there even is a market for non-bloated, sleek software anymore.
[1]: https://gavinhoward.com/2023/02/why-i-use-c-when-i-believe-i...
[2]: https://gavinhoward.com/2020/12/my-development-environment-a...
[3]: https://gavinhoward.com/2023/06/an-apology-to-the-gentoo-aut...
[4]: https://gavinhoward.com/2023/09/lessons-learned-as-a-user-3-...
Yep
Think about it. Is there any widely-deployed OS not written in C/C++? Any web browsers? Spreadsheets? CAD software? Text editor?
Surely you understand that your argument is disingenuous, right? Nobody wants to start over a new OS or a browser in another language due to the huge costs that no corporation nowadays is willing to shoulder (due to their own interests). Otherwise a lot of people would very definitely write OS-es and browsers in Rust, D, V even, Zig, Nim and a good amount of others.
C/C++ were there first, that's all there is to it, it's that simple.
Besides, I struggled with sysadmin-type tasks until I bit the bullet and installed Linux from Scratch and then Gentoo. It was one of my best educational investments.
"I am going to build the strongest, lightest bridge in the world" is a marvel. "I am going to build a light-enough strong-enough bridge for a cost my client can afford" is engineering.
But a sign of considering tradeoffs is that every project seems to be just a little different because every circumstance is just a little different.
When you have a software monoculture or monocultures, it's likely that tradeoffs are being ignored.
For me personally, I am banking on having multiple business customers for my software. If I consider the cost amortized, I can spend a little more to engineer something better.
But yes, it still has to be fit for purpose. My customers will provide a document that explains what they want to do with my software and on what platforms. If I sign a contract, that means I am committing to supporting their purposes on those platforms.
Avionics. Air traffic control. Industrial control. Automotive software. 911 dispatch. Medical instruments. Medical information. Military command and control. Mechanical engineering analysis software.
Lots of software can kill people.
that's what i was trying to draw attention to.
Can you expand on that?
Which exact problems of C's are you working on solving? Do you mean the language itself (writing a new dialect of C), or the ecosystem (e.g. impossibility of static linking with glibc)? Or something else entirely?
My blog post about C has a blurb about how I'm really writing in a partially memory-safe portable dialect.
I have bounds checks, structured concurrency (to mitigate use-after-free and double-frees), and a bunch of other stuff.
Actually, I had written more about Rust and Ada, but then I read your blog, and it says you like C better anyway because you find it more fun, so nevermind.
You might also like to use D in its "Better C" mode, which at least offers bounds-checked arrays and sliced, as well as some other features, while being very similar to C.
https://dlang.org/spec/betterc.html
Try supporting unicode
After I get to MVP and actually try marketing, we'll see what happens.
Making something people want enough to pay for is very difficult; if you are trying to do that while also imposing a bunch of other constraints on yourself that have no clear marketing benefits, your odds of success go down a lot.
Engineering excellence can sometimes be used successfully as a form of marketing in itself and you could well pull this off, as your content is super engaging and well-written.
But I would suggest to every engineer to consider Chesterton’s fence. Why is all the winning software in every category slow and bloated and seemingly unconcerned with engineering excellence? Is it because everyone involved, the engineers that build it and the customers that buy it, are all technical philistine morons? Or is it because the products that win in the market prioritize… winning in the market, and all their competitors that don’t prioritize that for whatever reason fall by the wayside?
I'm all for quality engineering. I'm just saying that if your whole plan for a new product is "quality engineering", then you have no plan at all. The horse needs to come before the cart.
What you're describing is a tragedy of the commons from the perspective of social and civilizational benefit. I agree that's what it is, but I think we should be more careful before justifying it.
> But I would suggest to every engineer to consider Chesterton’s fence.
I agree.
In fact, I believe I have considered Chesterton's Fence; I have a plan.
I'd like your opinion on my plan.
> Making something people want enough to pay for is very difficult.
Yes, absolutely.
To fight this, I have spent years, 3.5 years, figuring out what people hate about build systems, and I have designed mine to address those.
While that doesn't guarantee success, I think it may improve my odds. Do you agree?
> That is: builders who prioritize marketing outcompete those who prioritize engineering quality.
Yes, and it sucks.
I hate marketers. I hate marketing. I hate the fact that I'm going to do it. But I have to.
So my marketing with be focused on giving people something for just paying attention.
> Engineering excellence can sometimes be used successfully as a form of marketing in itself and you could well pull this off, as your content is super engaging and well-written.
Thank you for your compliment! I hope so, and my plan is to emphasize this.
I'm going to post four articles on HN, each a week apart, and each will be designed to give readers something substantial, with a blurb about marketing at the end that will be clearly marked as marketing.
* The first will be new FOSS licenses designed to better protect contributors from legal liability.
* The second will be a post on language design and psychology.
* The third will be a deep dive into Turing-completeness and what it means.
* The fourth will be the source code as a Show HN, along with an offer for early adopters.
> Or is it because the products that win in the market prioritize… winning in the market, and all their competitors that don’t prioritize that for whatever reason fall by the wayside?
It's awful, but you are right.
So this marketing push is all I will do for four weeks: preparing and posting, and responding to comments. This is when I will "prioritize winning in the market."
Does this have a chance of working?
It's a start for sure. But personally I'd suggest changing your mindset a bit from the idea that you'll come out of the code cave for a month to grit your teeth and do marketing. Granted, this is much better than just staying in the cave. But really to run a successful business I think you need to accept at a deep level that marketing, making money, and growing the business is now your main job, and you will permanently need to spend at least as many thought cycles on that as you do on programming.
To wit, while posting those articles and a Show HN sounds like a good plan and you should definitely do that, what will you do if they all fall flat and get no traction? It's a distinct possibility, and I hope you won't just give up.
I'd think more about what you're going to do every week for the next year to get users rather than putting all your chips on an HN launch that may or may not pan out. Even if you do rock the HN launch, you're probably going to have the "trough of sorrow"[1] to contend with after, so I'd think more about how you can make marketing a repeatable part of your rhythm in the long run.
1 - https://andrewchen.com/after-the-techcrunch-bump-life-in-the...
> while posting those articles and a Show HN sounds like a good plan and you should definitely do that, what will you do if they all fall flat and get no traction?
You got me. I was planning on giving up. :P
I am in a place where the only cost to switching projects and trying again in three years is the opportunity cost, so because I'm bad at constant marketing, that's what I was going to do if I got zero traction.
If I got only some traction, I'd weigh my options.
I was only going to worry about long-term weekly marketing if the launch went well enough.
Because I'll be frank: I have no idea how to do constant marketing that doesn't bother people or waste their time. If I would waste their time, I'd rather just throw my own time away and switch projects.
> Even if you do rock the HN launch, you're probably going to have the "trough of sorrow"[1] to contend with after
Good blog post, and yes, I agree. I am expecting the trough as I build up the MVP.
If you haven't gotten it in many users' hands yet, it might be a good idea to try recruiting like 50-100 users first, either one-by-one through email reachouts or in smaller communities where it's less hit-or-miss, like niche subreddits. If some of these users like the product and stick with it, start giving you feature requests, etc., that tells you that you're on to something. Conversely, if you can't get even a small group of users to try it and stick with it using that approach, it's much more of a negative signal than a failed HN launch and probably indicates that something needs to change.
Whatever route you decide to take, I wish you the best with it!
Also:
"I have no idea how to do constant marketing that doesn't bother people or waste their time. If I would waste their time, I'd rather just throw my own time away and switch projects."
That's noble of you, but I would cut yourself some slack. Ideally you would market in a way where you don't bother people or waste their time, but getting users often requires trying things where you risk getting close to that line. Sometimes you might cross over it, but that's just something to learn from.
Trying to market a product while never bothering anyone the least little bit is a bit like trying to be a comedian without offending anyone or to find a romantic partner without enduring some awkward dates. I think it just goes with the territory.
I'll think about what to do.
I think patio11 is someone who has used a similar marketing strategy to great success. People hire him because of the quality of his writing demonstrating his understanding.
I also think I can't replicate patio11's success! He's a much better marketer and writer than me.
Not your parent commenter but I wouldn't agree, no. Build systems are one of these things most companies want to setup once and want to never deal with them again, and only do so if they have zero choice.
`earthly.dev` had an article some weeks ago that addressed why one of their products never took off even though it's really good and addresses grievances in the space. The TL;DR was that existing customers of other CI/CD tools did not want to migrate to a new system, especially since it doesn't offer a migration from their existing solution.
Here it is, hope you find it insightful: https://earthly.dev/blog/shutting-down-earthly-ci/
1. They are extremely aggressive about coming to market fast. That's a good thing, yes. 2. They then let the project ossify because they view development as a cost center. Which is a bad thing. 3. Their endgame after being successful enough and can afford enough lawyers, they start proactively buying or shutting down potential competitors.
The "marketing wins over quality (and slower) engineering" does apply, yes, but not throughout the entire lifecycle. It applies only in the very first phase.
Your perspective is interesting because I'm old enough to remember when the C Language was considered bloat compared to just writing it in assembly language.
Examples of 1980s programs in assembly was WordPerfect, Lotus 123, MS-DOS 1.0. SubLogic Flight Simulator (before Microsoft bought it) was also in assembly.
Back then, industry observers were saying that MS Word and MS Excel being written in "bloated" C was a reason that Microsoft was iterating on new features faster and porting to other architectures sooner than competitors WordPerfect and Lotus 123 because they stayed with assembly language too long. (They did eventually adopt C.)
I see this "bloat-vs-lean" tradeoff in my own software I write for my private use. I often use higher-level and "bloated" C#/Python instead of the leaner C/C++ because I can finish a particular task a lot faster. In my case I'm more skilled in C++ than C# and prefer leaner C++ executables but those positives don't matter when C# accomplishes a desired task in less time. I'm part of the bloated software problem!
It's all tradeoffs; and finding the sweet spot for every particular insurance.
I believe the article is complaining that people just blow past the sweet spot.
In your case, C# is the sweet spot. In my case, I expect customers will want speed; I'm building an interpreter for a programming language.
Languages like C brought a massive benefit to accessibility. Devolving “software is slow” to “yeah but C vs assembly” is such a ridiculous crutch argument. Assembly is not remotely approachable to the majority of programmers. C, rust, zig, c++, Java, C# are all approachable languages that are fast and have great fast libraries and frameworks to work with.
All I can see in the “I can finish the see sharp program faster” argument is that “python vs c++.jpeg” from the 2000’s where half the python was importing libraries, but they wrote the C++ from scratch, and everyone who knew nothing about C++ moved this image around like it was some hilarious joke of C++.
Both perspectives are correct for their respective times. Compilers were much dumber in 80. These days your GUI desktop program written in assembly language would probably run slower than written in C and compiled with modern gcc O2.
The problem is greed - i.e. capitalism. The blog mentions Electron and therefore Chrome and JavaScript. They are awful combination and allow companies to save on programmers. Programmers which handle C, C++, Rust or Python are a small group. Java, C# and JavaScript consume more resources, allow using a lot more stuff quicker (like a drug) and most importantly are forgiving mistakes. So the industry decides to waste the resources on our computers because they don’t need to buy and maintain them. The customer pays twice, for the software and the next computer with more soldered main-memory. The key point is - the software companies don’t pay for hardware or the environmental damage! I’m doing myself a little JavaScript and some Java. Efficiency? Nobody ask for that and I should not spend time on it. There is now law stating that managed languages need to waste resources but a side-effect.
Remember Steve Jobs? I don’t appreciate what Apple does. But he banned Flash for a reason. Resources! Okay. Also bugs. And for the same reason they should ban Electron. Apple devices run faster with less resources because Apple saves on hardware. Why build devices with huge batteries, when you can achieve more runtime with less material and take the same money? The EU needs to enforce side-loading on the iPhone. But I also think Apple should be allowed to keep Blink (and therefore Electron) banned from AppStore. I don’t want see my battery dying because some corporate manager decided to drain it. But if you need Blink? Side load.
They eliminated the unappropriate “desktop metaphor” from Windows 95 and the “system tray”.
The negative side is that GNOME often removes options or hides them - which drives away experienced users (the people needed to pull in new users) and some developers to forks. The are right to not support every bewildering option but the needed ones must in place despite the UX people don’t use them itself.
https://ometer.com/free-software-ui.html
Some of this is right. Do stuff automatically right and don’t provide unneeded options to avoid complexity. Some not! Also provide required preferences e.g. “Do not suspend on LID close” because some people don’t want that! The GNOME people assumed it was needed as option because of problems with suspend (S3) in the past. Maybe people used it to bypass issues but that isn’t the use case of the preference. Another thing is “When you’ve five clock applets” I want to know what is missing from the first one which made all the others necessary. GNOME learned that people don’t need an “Emacs” as UI-Shell but somewhat went into the other extremes.
PS: And GNOME just looks good by default. I don’t need themes because it is fine.
For me... I don't notice any slowness or excessive battery drain in VS Code, so I just use that. Perhaps there's something deeply wrong with my psychology, but I'm just not very interested in simplicity for its own sake. It's cool, but not an approach I'd want to use every day in real life.
I wonder how much of this is just because when a project manager comes along and says "as a system administrator, I want to be able to log all of the keystrokes of the users and reports who's slacking to the boss", you say, "ok, that feature will take about two weeks to implement" and they can't argue with you because it's C so they just go away and leave you alone.
I can relate to the disenchantment but I found the argument here to fall flat.
He has SOME editor in mind, I'm just curious what it is.
I understand that author was referring to the concept of a minimal version that does just text editing and nothing more. Add features to anything and it will stop being simple and easy.
However, the author might also be generalizing and referring to "MS Word" as a text editor, which a lot of people unfortunately do. This is where I would disagree with the author's premise.
Sure, if you add non-existing features then the real argument assessing apps without those features would fall flat
https://100r.co/site/weathering_software_winter.html
Industry prioritizes functional solutions (requirements) over efficiency. If efficiency is one of the requirements, it will be addressed (e.g. video games). Optimizing for efficiency takes additional effort. The article argues that the software industry is stuck with inefficient tools and practices. Engineers can and should do better, aiming for better apps, delivered faster and more reliably with fewer resources. However, economy dictates that as you optimize two variables from the triad, the third will get de-prioritized.
Edit: I can see the downvotes but no idea why. Would you care to explain?
I dunno, but this has been taught in PM since PM. I remember learning it in the 90s and it still holds true. If you want it fast and a large scope, you have to throw bodies and planning at it, costing money. If you want it cheap with a big scope, you have to wait for that small team to finish it (time). If you want it fast and cheap, you have to limit your scope.
Time/Cost/Scope, pick 2. Perhaps people are taking issue with quality vs scope, but quality is a part of scope for certain.
https://www.projectmanager.com/blog/triple-constraint-projec...
Isn’t that what drives the most sales in upgrading a phone? Idk many people that care about the microscopic improvements to the camera or UI, most of the time it’s something along the lines of “my iPhone 8 doesn’t run fast enough anymore, time to upgrade”. Same with consoles. I remember one of the big pitches of next gen consoles being “look! No more loading screens!”. And even ChatGPT 3.5 vs ChatGPT 4. There are tons of people that will use crappier output because it’s faster. Speed is still absolutely a selling point, and people do care.
Those are the very specific use case I mentioned.
How many users are gonna swap to a different chat client because of this or a different word processor because it start 1s faster? As a parallel comment points out, even developers swapped away from the very quick Sublime to Atom and now VSCode. At work hiring at some point became a huge pain because we swapped from Greenhouse to the recruiting tools built into Workday. It was super slow. I hated it, our recruiting team hated it, but it was purchased because it checked all the boxes and integrated with all our other stuff in Workday. The comparison to engine efficiency made me think that if we really want faster software, we need the equivalent of a gasoline tax for software, but what's the negative externality we are preventing?
That suggests that while those three factors are part of the explanation they’re not sufficient to explain it alone. My theory would be that we’re imbalanced by the massive ad-tech industry and many companies are optimizing for ad revenue and/or data collection over other factors such as user satisfaction, especially with the effects of consolidation reducing market corrective pressure.
It's a tragedy of the commons, where everybody would benefit from better tooling, everybody wastes time dealing with poor tools, but nobody is willing to put time/effort/money into making better tooling.
There's plenty of FOSS developers that work for intrinsic rewards and likely produce better software.
https://www.sciencedirect.com/science/article/abs/pii/S02637...
https://www.researchgate.net/publication/305462896_Project_m...
More recent publications replace "quality" with "scope" since quality cannot be controlled directly and is seen as a result.
Like at my last job, we had something like 50 microservices, Kafka, rabbit, redis, varnish caching, etc. For under 1k external requests/second and some batch processes that ran for a few million accounts. If we cut out all of the architecture to "scale", you could've run the whole thing on a laptop if not a raspberry pi. And then a real server could scale that 100x.
The company was looking at moving to a "cloud native" serverless architecture when I left.
The impossible trinity (also known as the impossible trilemma or the Unholy Trinity) is a concept in international economics and international political economy which states that it is impossible to have all three of the following at the same time:
Even open source suffers from the same problem. Making really high quality software takes 10 times the effort (if not more) and nobody is willing to make that investment up front. And so half-baked software becomes popular and now we spend years or decades and untold hours trying to turn bad software into pretty good software without completely breaking backwards compatibility.
The root cause is probably the incrementalist approach we take to developing software. We start with something small because of time constraints or because we don't really understand the problem yet. Once the software has users who demand bug fixes and feature additions and this results in software organically growing instead of being intelligently designed for a purpose. As a result, you get stuck in a local maximum.
And that's how we spend 500,000 hours debating strategies for removing the GIL (global interpreter lock) in Python when the first version of Python was made 30 years ago as a side project during a holiday break.
I would blame the "unix philosophy" and "worse is better" approaches of the past, but I bet they were more symptomatic than causal, and their equivalents in other digital realms pop up all the time: IBM vs clones, unix wars, protocol wars, at various times its fights between 'official' (described as stuffy) vs 'pragmatic' (described as lax/crappy) definitions of stuff.
I'd hazard a guess that since (for those of us who are young and therefore spew confident sounding incorrect speculation like this comment) we still have so much of the old 'it works, ship it' hacker groups of the 70s in our past, then the overreach of the CASE / UML / XML fever of the late 90s which we have in turn overreacted against by going too far in the 'look its a containerized k8s pod running behind a reverse proxy that runs some react and uses leftpad to graphql your (must always be online) user information record in this headless electron because SHIP IT' direction.
PS: our historical 'its good enough' precedent didn't help, we've been trapped on 'very fast PDP-11s' for decades. Even BWK and the other forerunners of our modern C + unixlike stack weren't able to get us un-stuck from that stack and so plan 9 etc. failed to catch on. The Lindy Effect is a double-edged sword for sure.
https://pavelfatin.com/typing-with-pleasure/
Let’s keep the standard for performance high and continue to highlight performance issues.
Simple example, I sold my car to Carvana the other day and just baaaarely pulled it off using Chrome and Firefox. In Chrome the upload image wizard would get a JS exception. That part of the app miraculously worked in FF, but virtually the entire rest of the site was mired in issues as it's obvious Carvana devs don't test in FF. I pulled off the transaction by bouncing between the two.
Even worse, most non-technical people think they did something wrong when they encounter a bug.
Software that is bloated and slow but stable and rock solid? I'd gladly take it at this point.
There are still computer systems designed with high reliability and accuracy. See flight control systems.
New technology typically doesn't work on duct tape and bubblegum. So it has to be good. Current technology is going to be just right enough to work.
It can be hard to tell what is "duct tape" and what is "speed tape" in software these days. An aircraft mechanic doesn't need to know the structural differences between the two - just use speed tape to be safe. Similarly, a pure programmer doesn't have to know the difference between ufw and Windows Firewall - just use the latter and move on.
However, an engineer (mechanical or software) better understand the differences in both situations
I understand all the points about financial costs and opportunity costs and pragmatism and the rest, and I partially agree with it, but it's hard not to sometimes feel like we've accepted living in a half built world.
Not testing on Firefox just makes sense given how niche it has become. Not worth the effort to go beyond Chromium.
The number of vehicle software updates are staggering.
Then something got pushed over the cliff. Windows 10 and the UWP...like seriously WTF!? Satan's very own dumpster fire.
And it's not like things have gotten any better, now I'm an adult with young children and if one of them gets their hands on my phone or laptop, they seem to be just as reliably be able to lock up, crash, freeze modern devices all the same. These kids are not physically damaging the phone in any way, they're just pushing buttons too quickly or in unexpected orders. That's the state of modern tech that we're in.
To be honest, I can understand why this hasn't been fixed. As mentioned in the article and throughout these comments, we've just come to expect to need to restart every now and again. In this case, people are likely to blame such a problem on the child, and a restart fixes the issue anyway. I just would've hoped for better by now in the lifecycle of these operating systems.
High complexity, ugly / unplanned, high rent (cloud costs, etc.), bloated, only polished on the surface, and designed to gather as much market share as possible as fast as possible and we'll fix it "later."
Examples: Electron, Kubernetes, Helm, expensive managed cloud, lots of domain specific languages and other high cognitive load systems demanding high staffing requirements, total abandonment of labor saving tech like WYSIWYG design, etc.
Though times are changing in that world too. Sometimes you have to use a library. And more and more those libraries require an RTOS. Just about to make the plunge into Zephyr so I can use the current Nordic BLE SDK.
Having hard limits to RAM and flash is a great way to prevent bloat. Management is happy to let engineers grind for a month reducing code size of it means the code will fit into a cheaper MCU with less flash and save a 10¢ on the BOM. Pure software has no such incentive to minimize resources because the user buys the HW separately. If anything, some SW companies have an incentive to add bloat if they're the same company that sells you a new phone when your old one becomes too slow.
https://archive.ph/KiNbP
The market has clearly spoken in favor of the former.
After all, the goal of engineering isn’t to produce some abstract beautiful art but rather to create something valuable for regular humans.
Also the car and building analogies are poor because they are essentially the same design that has been optimized for decades. Indeed the only “new features” in cars are usually software driven and probably just as inefficient.
Demand software with features and performs efficiently.
If anything, the last several years in the software industry have proven hordes of mediocre developers really isn't a substitute for understanding how computers work.
The phenomena we're seeing now is VC money has dried up for all but those focused on efficiency, and the rats are scrambling to run to the other side of the ship before it sinks.
Same for web apps, why is Facebook, Gmail slow? I could accept it for new products but not for nearly unchanged products after all these years.
Btw: Gmail HTML was fast, so it can't be an infra problem, it's bad frontend programming.
That's not the only reason, although it is one. Two of the other big reasons for this are:
1. The content is often loaded async while the boxes render, and this hides the latency of doing a network fetch.
2. Animating text looks terrible. Even if you had a perfectly fast computer, you wouldn't want to animate boxes filled with text and images, because it looks really bad.
Then as the team grew, the values changed to favor anything that improved developer efficiency. More abstractions, more layers, more frameworks. The tradeoff of saving one day of developer work was worth the cost of millions of user seconds collectively. I think the difference was just the visibility - management can see the costs of things on the development side, but they can't see the benefits of a slightly faster launch time, or better caching, or smoother scrolling. They aren't measurable, and once an org gets to the point where all it cares about are measurable numbers, I think this is a natural course.
Of course, that's until AI gets to the point where it can fix everything we (or it) programmed wrong, including AI itself which is highly inefficient just like the OP predicts.
It also over-prioritizes readability if anything, using descriptive variable names and documenting every line with a comment. It's maybe not as good as the best engineers at considering all these factors, but I'd say it's better than the average developer, and may be better than the best engineer too when that engineer is tired and in a hurry.
Cars were optimised only after external (oil) shocks were applied, and even then very unevenly (American cars continue to be very inefficient). In fact, inefficient products are often more profitable in practice, as customers have to replace them more often; the crappy iPhone cable that splits after a year means Apple can charge you again (and again) for its replacement. White goods now break more often, but they're built more cheaply so they can make per-unit profit higher than it would otherwise be. What matters is efficiency producing, not after-sale use.
Capitalism in software means churning out new automation as quickly as possible, letting consumers pick up the resulting waste of energy and time. The production chain gets more and more standardized and optimized: you can now swap React developers, or Kubernetes admins, like you can swap warehouse workers, with all that it entails in terms of salary pressure. Some of that automation is effectively self-justifying, in the same way accountants make accountancy terms obscure so they can justify their jobs; but that's about it. Everything else is about profit.