To put a finer point on it, software today is absolute garbage. I've been screaming about this for decades.
All of this bloated 'shitware' today is the result of it having been written by people who a) have no deeper understanding of what the computer is actually doing; also known as typical Python/Java/etc/etc/etc programmers, and/or b) simply not giving a damn about conservation of resources--as further evidenced by all of the other extremely wasteful and destructive habits they hold in their personal lives, and in their societies in general.
After all, this is the same civilization that's burning through increasingly vast quantities of oil at an astounding rate, despite the fact that previously existing abundant and cheap oil is nearly depleted, with no possibility of replenishment or replacement. So is it any surprise that foolish developers also burn through CPU and memory with reckless abandon?
Really, the problems we face aren't just in software; they're more about the foundations of our entire Western 'civilization.' Such problems generally tend to be rather intractable, in the historical view.
I'm working to construct, in my own computing life, something of a 'personal oasis', which is increasingly removed and estranged from all of the horrible things I see Other People out there having to suffer in their personal computing lives, thanks to talentless 'developers' who Just Don't Fucking Care. Some of these pricks actually have the audacity to call themselves 'engineers', even.
They seem to believe civilization is always and is necessarily a good thing, and is not a bad things. No! Like many things, civilization is both good and bad.
Buddy, oil isn't running out anytime soon. If you want better programmers, maybe we should have more than a handful of universities that have programs whose sole purpose is churning out the same type of developers you're shitting on.
> this bloated 'shitware' today is the result of it having been written by people who a) have no deeper understanding of what the computer is actually doing; also known as typical Python/Java/etc/etc/etc programmers, and/or b) simply not giving a damn about conservation of resources
Much of the blame lies with bloated frameworks, such as Electron, rather than with the developer who uses the framework. You've covered this under Option B.
This topic cropped up recently on a Show HN. Someone built a Slack client for Windows 3.1. [0] It uses a tiny fraction of the memory used by the official Electron-based client, of course.
> To put a finer point on it, software today is absolute garbage. I've been screaming about this for decades.
That statement contradicts itself. Is this satire?
> All of this bloated 'shitware' today is the result of it having been written by people who a) have no deeper understanding of what the computer is actually doing; also known as typical Python/Java/etc/etc/etc programmers, and/or b) simply not giving a damn about conservation of resources--as further evidenced by all of the other extremely wasteful and destructive habits they hold in their personal lives, and in their societies in general.
Everything has a cost / benefit associated with it. Pretending otherwise show how little you know about engineering.
> I'm working to construct, in my own computing life, something of a 'personal oasis', which is increasingly removed and estranged from all of the horrible things I see Other People out there having to suffer in their personal computing lives, thanks to talentless 'developers' who Just Don't Fucking Care. Some of these pricks actually have the audacity to call themselves 'engineers', even.
I dunno how far your head has to be up your own behind to actually believe this? It not like most developers these days work on large software projects that are normally poorly costed, estimated, planned usually with stifling restrictions because certain "enterprise" technology is mandated by some architect who hasn't written a line of code in a couple of decades. Most people like a consistent regular income which allows them to support themselves and their family. I suspect these concerns are more important then incoherent ramblings of some guy on HN.
> 'That statement contradicts itself. Is this satire?'
My comment was the first one on this article. 473 comments later, it looks like over 90% of the other commenters agree with my observations, considering 90% of the other posts could have been written by me also. So the real question is, what do we know that you don't?
I started programming in 1991. Since the 90s I've watched as software has been steadily circling the drain. Many others have noticed it too, as shown by this discussion. The growing discontent is palpable. Where have you been during this time?
> 'Everything has a cost / benefit associated with it. Pretending otherwise show how little you know about engineering.'
Typical excuse from a lazy, talentless hack, excusing his shit code. What you are doing is not engineering.
> 'It not like most developers these days work on large software projects that are normally poorly costed, estimated, planned usually with stifling restrictions because certain "enterprise" technology is mandated by some architect who hasn't written a line of code in a couple of decades. Most people like a consistent regular income which allows them to support themselves and their family. I suspect these concerns are more important then incoherent ramblings of some guy on HN.'
Typical, common, ordinary, everyday selfishness. "Well I thought about taking a stand for what's right, but decided against it Because Reasons." Pathetic.
What are you doing that is actively making the world of computers, or even the world in general, a better place? Is that something you care about at all? Or do you prefer to just cash that check while making bullshit excuses?
I live in a cabin in the woods just because I'm not going to swim in the shark pit that you call home. Nor am I going to actively fuck up the world just to make a paycheck. It's called having principles.
Did you notice the subtle contradiction in your post?
> they're more about the foundations of our entire Western 'civilization.'
> talentless 'developers'
It's not talentless developers, it's the foundations of civilization. Capitalism, chasing the $ not the writing of beautiful software. If an Electron app makes me more money (e.g. by getting to market quicker) than a well crafted native app, then the Electron app will be built. See also building construction and why we end up with 60m^2 poorly constructed but funky looking apartments - who to blame. The concrete pourers? The architect? I don't think it is that simple.
It is quite simple. Either you are helping the world, or you are hurting it. Either you do what's right, or you make excuses and continue in the course of wrongdoing. The choice is yours. And that is the existential problem we face--getting people to admit their own wrongdoing and culpability and to repent. I don't want to hear any fucking excuses from anyone about how they were just trying to make a buck or whatever. You don't have to do a damn thing, and sure as hell shouldn't, if it's fucking up the world! "Keeping up with the Joneses" is NOT a valid excuse; nor is "just following orders."
Yes, it is true, many programming they don't do such a good job. I do try to make better software, but will not always succeed. Also computer hardware is becoming too complicated too actually, I think. I generally don't add so much dependencies to a program though. Some people (including myself) do still write DOS programs sometimes. The web browser is too complicated. I use IRC and NNTP, I think they are much better anyways than Slack and so on (and even this HN, too, I think). I do program in C (I use other programming languages as well, but mainly C). Many programs I think have too many animations. And a lot of programs, they just write it stupid!! TeX is good and it still works more than thirty years later, and is fast, too. But anyways I don't like WYSIWYG, so that is why I don't use LibreOffice and Microsoft Word and so on.
Even quite underpowered phones can boot in ~1-2 seconds if optimized for that. Not everything in the phone will startup in that amount of time (modem, wifi), but it's possible to start to Linux userspace and display fully interactive UI in that amount of time.
Even my e-book reader Linux port boots to UI in ~2s.
To be honest, I have never seen any phone or computer that could boot in about 2 seconds. 6-7 seconds is the absolute minimum I've ever seen, regardless of hardware or OS.
You know? Once upon a time i've done some freelance sysadmin for small businesses. So i installed a new fileserver and a networking gateway with least-cost router functionality for the phones, somewhere. When ready i asked what that big tower did, that stood aside unplugged. 'Nothing, that is just trash we haven't discarded yet.' Since the case had a nice design it seemed wasteful to me, and i took it back home. At home i plugged it into to some spare screen, and since it had a networking card into that also. When ready i rocked the big red switch it had with a satisfying clack into the upwards posistion and wanted to do something else at my other systems while it booted up (i thought). So i rolled, sitting on my chair, maybe 3 feet, and then heard a loud 'Ta-Daa!'. I looked sideways and barely saw how Word 6 and Excel 5 popped up, out of the corner of my eyes. I was dumbstruck!
I shut it down and repeated that, looking at my wristwratch.
FOUR SECONDS!
I couldn't believe that and tested maybe five times again. It took never more than 4 seconds from rocking the switch to WfW 3.11 Desktop with Word 6 and Excel 5 in autostart ready to use! It even got a DHCP address while being at it.
The thing is, at that time i had some Sun SparcStations, some HP PA-Risc Workstations, and assorted X86 PCs, one of them an AMD Athlon XP PR1800 overclcocked to PR2100 with 1.5
GB virtual channel memory SD-Ram, running NetBSD (because it just worked, don't ask). I felt very ahead of 'the curve'.
And then this trash came along and burst my bubble...
I sat there and only thought: 'what for?!'
Now some old Windows running atop of DOS isn't something to really envy, but in this combination of components, BIOS, drivers it got the job done without any hassle, FAST!
Edit: WRT BIOS. It had a MR-Bios from Microid Research which allowd to use it over the onboard serial ports, you could choose which, how, and so on. Another functionality which has been gone, and came back later only as expensive add on.
I agree it's all slower and sucks. But I don't think it's solely a technical problem.
1/ What didn't seem to get mentioned was the speed to market. It's far worse to build the right thing no one wants, than to build the crappy thing that some people want a lot. As a result, it makes sense for people to leverage electron--but it has consequences for users down the line.
2/ Because we deal with orders of magnitude with software, it's not actually a good ROI to deal with things that are under 1x improvement on a human scale. So what made sense to optimize when computers were 300MHz doesn't make sense at all when computers are 1GHz, given a limited time and budget.
3/ Anecdotally (and others can nix or verify), what I hear from ex-Googlers is that no one gets credit for maintaining the existing software or trying to make it faster. The only way you get promoted is if you created a new project. So that's what people end up doing, and you get 4 or 5 versions of the same project that do the same thing, all not very well.
I agree that the suckage is a problem. But I think it's the structure of incentives in the environment that software is written that also needs to be addressed, not just the technical deficiencies of how we practice writing software, like how to maintain state.
It's interesting Chris Granger submitted this. I can see that the gears have been turning for him on this topic again.
Most developers I have known want to work on the new great new thing. They don't want to spend a great deal of time on the project either. Forget about them wanting to dedicate time to software maintenance. Not sexy enough.
Ok but why ? And what can we do to improve things? Promote maintenance, but I think one of the issues is that you can show something new, it's much more difficult to show that something could have changed (failure, difficulty to grow), but didn't.
To the extent it's in your power as a developer and a team member, don't tolerate low-performance code from yourself or your co-workers.
In my experience, a lot of performance problems boil down to really stupid problems, like simple code using the wrong data structure out of convenience (e.g. linked lists instead of arrays for lots of randomly-accessed data), or structured in a bad way (e.g. allocating a lot of small pieces of memory all the time). Often times, there are cheap performance wins to be had if you occasionally run the product through a profiler and spend couple of hours fixing the most pressing issue that shows up. Couple of hours isn't much; there's enough slack in the development process to find those hours every month or two, without slowing down your regular work.
I agree with your point of developers being responsible for the performance.
But I have a different experience (probably because we work in different areas):
Most the performance problems of the products I ever worked were purely systemic.
They boiled down to technologies and architectures having been chosen for "organizational" rather than technological reasons.
And "organizational" is in quotes because sometimes it was just blackmail: I worked with two developers who quit in protest after the prototype they wrote in Scala was deemed not good enough and dropped for... being too slow, ironically.
This has been a major frustration for me as a UI developer on the current application I work on. The UI is often hamstrung by how the backend API was implemented. There are frequently cases where we stitch together pre-existing API functionality to make something work in a far-from-ideal manner just because it would take longer to do it right and no one is interested.
I've seen a similar thing happen. It all started with good intentions, like only having simple endpoints that do "one and only one thing".
In the end the backend was pure and beautiful, but the the frontend devs had to perform joins in the the client and make 21 API calls in a 20-item list and then everything goes to hell.
I might strengthen your argument even more and say it's largely a non-technical problem. We have had the tools necessary to build good software for a long time. As others have pointed out, I think a lot of this comes down to incentives and the fact that no one has demonstrated the tradeoff in a compelling way so far.
I find it really interesting that no one in the future of programming/coding community has been able to really articulate or demonstrate what an "ideal" version of software engineering would be like. What would the perfect project look like both socially and technically? What would I gain and what would I give up to have that? Can you demonstrate it beyond the handpicked examples you'll start with? We definitely didn't get there.
It's much harder to create a clear narrative around the social aspects of engineering, but it's not impossible - we weren't talking about agile 20 years ago. The question is can we come up with a complete system that resonates enough with people to actually push behavior change through? Solving that is very different than building the next great language or framework. It requires starting a movement and capturing a belief that the community has in some actionable form.
I've been thinking a lot about all of this since we closed down Eve. I've also been working on a few things. :)
I'll take this opportunity to appreciate C# in VS as a counterexample to the article. Fast as hell (sub-second compile times for a moderately large project on my 2011 vintage 2500k), extremely stable, productive, and aesthetically pleasing. So, thanks.
It's very hard for me to get away from C# because it's just so crazy productive. The tooling is fanstastic and the runtime performance is more than good enough.
One thing I found was that surprisingly the C# code I write outperforms the C++ code I used to write at equal development times.
I was good at C++, but the language has so many footguns and in general is so slow to develop in that I would stick to "simple" and straightforward solutions. I avoided multi-threading like the plague because it was just so hard to get right.
Meanwhile in C# it's just so easy to sprinkle a little bit of multithreading into almost any application (even command-line tools) that I do it "just because". Even if the single-threaded performance is not-so-great, the end result is often much better.
Similarly, it's easy to apply complex algorithms or switch between a few variants until something works well. In C++ or even Rust, the strict ownership semantics makes some algorithm changes require wholesale changes to the rest of the program, making this kind of experimentation a no-go.
The thing that blows my mind is the "modern" approach to programming that seems to be mostly young people pretending that Java or C# just don't exist.
Have you seen what JavaScript and Python people call "easy?"
I saw a page describing a REST API based on JSON where they basically had thousands of functions with no documentation, no schema, and no typed return values. It was all "Just look at what the website JS does and reverse engineer it! It's so easy!"
I was flabbergasted. In Visual Studio I can literally just paste a WSDL URL into a form and it'll auto-generate a 100K-line client with async methods and strongly-typed parameters and return values in like... a second. Ditto for Linq-2-SQL or similar frameworks.
I think you will see great change if you were to look at the personalities around one opportunity.
Because it's never problems really, it's perceived that way though.
A certain challange needs a specific set of personalities to solve it. That's the real puzzle.
Great engineers will never be able to solve things properly unlessed given the chance by those who control the surroundings.
We seek how we should develop, what method should be used, is it agile or is it lean? But maybe the problem starts earlier and focusing on exactly what methods and tools to use we miss out on the most simplest solution even beginners can see.
For example I am an architect, I tend to not touch the economics in a project. It's better fitted for other persons.
While not having read much about team based development I do want to be directed to well read literature about it. Maybe it's better called social programming, just another label of what we really do.
The one I miss the most at work is my wife. She clearly is the best reverse of me and makes me perform 1000x better. I find that very funny since she does not care about IT at all.
I've also been lurking on the FoC community, and hadn't seen much on an articulation on the social and incentive structures that produce software. Do you think they'd be receptive to it?
And by "social and inventive structures", I'm assuming you're talking about change on the order of how open source software or agile development changed how we develop software?
While agile did address how to do software in an environment for changing requirements and limited time, we don't currently have anything that addresses an attention to speed of software, building solid foundations, and incentives to maintain software.
What would a complete system encompass that's currently missing in your mind?
The stuff I write I don't think is that bloated, but like most things these days the stuff I write pulls in a bunch of dependencies which in turn pulls in their own dependencies. The result, pretty bloated software.
Writing performant, clean, pure software is super appealing as a developer, so why don't I do something about the bloated software I write? I think a big part of it is it's hard to see the direct benefit from the very large amount of effort I'll have to put in.
Sure I can write that one thing from that one library that I use myself instead of pulling in the whole library. I might be faster, I might end up with a smaller binary, it might be more deterministic because I know exactly what it's doing. But it'll take a long time, might have a lot of bugs and forget about maintaining it. Then end of the day, do the people that use my software care that I put in the effort to do this? They probably won't even notice.
I think part of it is knowing how to use libraries. It's actually a good thing to make use of well-tested implementations a lot of time rather than re-inventing the wheel: for instance it would be crazy to implement your own cryptography functions, or your own networking stack in most cases. Libraries are good when they can encapsulate a very well-defined set of functionality behind a well-defined interface. Even better if that interface is arrived at through a standards process.
To me, where libraries get a bit more questionable is when they exist in the realm of pure abstraction, or when they try to own the flow of control or provide the structure around which your program should hang. For instance, with something like Ruby on Rails, it sometimes feels like you are trying to undo what the framework has assumed you need so that you can get the functionality you want. A good library should be something you build on top of, not something you carve your implementation out of.
I totally agree with him that it's awful, but I think the problem is that making things efficient is expensive in terms of time and having to hire expertise. Evidently efficiency just doesn't have enough return on investment for companies to care. The thing about looking back at the windows 95 era and comparing it to now, is that windows 95 needed to be efficient to be usable, windows 10 doesn't.
The exception would be games and embedded software, but even there, there's certain degrees of lazyness. For instance, games are very cpu/memory/gpu efficient, but they're almost always ridiculous when it comes to disk space usage. There's no reason that your average AAA game needs to take up 50GB other than that things which could address that aren't worth the fuss. (I'm thinking common demo-scene tricks like procedural textures/data/everything, aggressive compression schemes, reusing assets, etc.)
But what real enhancements does Windows 10 provide, which couldn't have simply been done as patches to Windows 7, keeping the same main structure and layout and overall design, but just continually improving it, the way Henry Ford did with the Model T for 20+ years?
For that matter, one could for example take the Windows NT 4.0 source code, add in drivers for the necessary hardware, fix boot code, linking, etc to be compatible with late model computers, spruce up the UI with better font rendering, antialiasing, 24-bit color wallpaper, OpenGL rendering even--and in the end, you'd have something just as functional as Win 7/10 but at 1/4 of the bloat.
This sort of thing would be technically very easy to do. It's much easier than the status quo of continually reinventing the wheel. So why, oh why, is there this overpowering desire to continually throw out good code and replace it with heavier, more bloated junk, which doesn't really offer any real increase in functionality?
> For that matter, one could for example take the Windows NT 4.0 source code, add in drivers for the necessary hardware, fix boot code, linking, etc to be compatible with late model computers, spruce up the UI with better font rendering, antialiasing, 24-bit color wallpaper, OpenGL rendering even--and in the end, you'd have something just as functional as Win 7/10 but at 1/4 of the bloat.
Except that it also happens with internal software, where there's no separation between developers and end-users. It's second-system effect writ large. "We'll rewrite this old, outdated software and add plenty of bells and whistles! This time will be different, we'll finally be doing it right."
See, to me, internal software may be the one place where "developer time" really is more valuable (depending on the size of the organization), because you're developing for far fewer uses.
One second times five million users is almost two months. One second times five thousand users is less than 90 minutes.
And what is the difference in productivity between using your custom built tools in your business workflow, vs just sitting there with your dick in your hand instead, unable to work, or having to use Notepad and Excel?
There's way more to life than just dollars and cents. Indeed, this really seems to be the larger point here. The bean counters and ladder climbers are now in control of everything, with disastrous consequences.
My point isn’t that they couldn’t or shouldn’t be more efficient, it’s that there’s no market pressure. No incentives means no change in behaviour. The only way to fix it is to change the incentives, otherwise we’re just in “old man yells at clouds” territory.
Besides, you could essentially do what you’re describing by piecing together a Linux desktop from a lean distro. It’s not hard to find efficient software, it’s just not really mainstream
I'll relax when I can get through the day without fighting some idiotic organizations excuse for a web form or attention hogging pop ups or, god help me, what else?!
We shouldn't have to put up with all this shit. That's the point.
Excellent. What, then, are you doing to make sure this shit doesn't happen?
Are you willing to pay $20 for a phone app, instead of fishing for the free version? Are you willing to pay for websites to get rid of ads? Are you willing to pay $300 for a new OS?
Alternatively, are you willing to ditch capitalism for a system that prioritizes the commons?
It's us. We are creating the incentives for a world that produces this. Unless we change that world, this is what we'll get.
> Isn't this the the world of ubiquitous code and unlimited resources we wanted?
I don't really know what this is supposed to mean, it sounds rather poetic and abstract, but I'm quite certain few if any people ever asked for a world where it takes 10+ seconds to open a simple email.
Probably because a browser like FF has the goal to load and display arbitrary dynamic content in realtime like a reddit infinite scroll with various 4k videos and ad bullshit, whereas the game has the goal to render a known, tested number of pre-downloaded assets in realtime.
Also on shitty pages the goal is different-- load a bunch of arbitrary adware programs and content that the user doesn't want, and only after that display the thing they want to read.
Also, you can click a link somewhere in your scrolling that opens a new, shitty page where you repeat the same crazy number of net connections, parsing ad bullshit, and incidentally rendering the text that the user wants to read.
If you want to compare fairly, imaging a game character entering a cave and immediately switching to a different character like Spiderman and inheriting all the physics and stuff from that newly loaded game. At that point the bulk of your gameplay is going to be loading new assets and you're back to the same responsiveness problems of the shitty web.
Browsers are fine. It's the websites that are slow.
It's not the fault of Firefox that Reddit's new UI is pathetically slow. It's the Reddit's implementation of their UI itself which is total garbage.
And given that people do write fast, complex, real-time games in JavaScript for the browser, gamedev absolutely becomes a valid reference point for the possible performance of any individual page.
Hmm, that leads be to an interesting counter idea.
Why should Firefox or any other dynamic software have the ability to be slow for what it archives? If compilers should be fast, Web engines should be equally as fast. The Web should have never been designed such that a slow website (relative to the task) could be achieved. In the same way that you can only express memory safe code in rust and type safe code in haskell why not being only able to express "fast for what is interactive"?
> The Web should have never been designed such that a slow website (relative to the task) could be achieved
That's already the case, your orders of magnitude are just off. Long-running AJAX or page loads are timed out at a pretty consistent point across browsers. Half-open/closed TCP connections are timed out at a pretty consistent point across operating systems. Busy-looping JS gets you a "page is not responding" block in a similar amount of time; same for nonresponsive native applications on many operating systems.
Their definition of "slow" or "stuck" just tends to be "tens of seconds or minutes", not the threshold of perceived responsiveness you want in a website.
Also, your parenthetical is a pretty tall-née-impossible order:
> a slow website (relative to the task)
How could the "task" be classified? Do you mean "task" as in "clicking a button and having a DOM update"? Or as in "this is a TODO application so it should have responsiveness threshold X"?
I'm both a web developer and a game developer, and this comparison doesn't ring true at all. Games usually have tons of arbitrary dynamic content to display in realtime. Minecraft will load about 9 million blocks around your character plus handle mobs, pathfinding, lighting, etc. Reddit infinite scroll loads a sequence of text, images, and videos. Multiplayer games have such tight latency and bandwidth targets that game developers routinely do optimizations web developers wouldn't even consider.
As a web developer, sending an 8 KB JSON response is no problem. That's nice and light. In a networked action game, that's absurd. First, (hypothetical network programmer talking here) we're going to use UDP and write our own network layer on top of it to provide reliability and ordering for packets when we need it. We're going to define a compact binary format. Your character's position takes 96 bits in memory (float x, y, z); we'll start by reducing that to 18 bits per component, and we'll drop the z if you haven't jumped. Then we'll delta compress them vs the previous frame. Etc.
Really, what's happening is things are getting optimized as much as they need to be. If your game is running at 10 fps, it's going to get optimized. When it's hitting 60+ fps on all target platforms, developers stop optimizing, even if it could potentially be faster. Same for Reddit; it's fast enough for most users.
This doesn't really relate to my point. The blog post asked why is it that we can handle games (fancy 3D simulations, sometimes with hundreds of players and millions of objects) at a smooth 60 fps but not scrolling a web page. The parent comment suggested that it's easier to render games smoothly because you know the content in advance. I'm suggesting that software gets optimized (by necessity) until it works well enough. If some website had to display a million elements, the devs would either optimize it until it could do so, or the project would get scrapped.
When I talk about sending 8 KB in a "networked action game", I'm referring to the update packets sent to and from clients in something like Fortnite or Counter-Strike, not a game patch. I'm not trying to make a competition for who uses the least bandwidth (which a 60 GB game would lose just on the initial download). I'm trying to illustrate that games don't run faster than some website because it's inherently easier to make games run fast, but rather that developers, by necessity, optimize games until they run fast (or in this example, until they reduce network lag enough).
I'm not sure why a chat app would tack on something like 7.5 KB of overhead on a chat message, but I wouldn't be surprised if there's a chat app out there that does so. Users won't notice the extra couple milliseconds (especially so because they don't know exactly when the other person hit send). A 3 character Discord message is close to 1 KB including the headers. The same message over UDP in a game might be under 20 bytes, including the UDP header (games could also use TCP for chat - text chat isn't going to strain anything). So I'd say the overhead of a Discord message is still an order of magnitude or two bigger than it could be. Which is perfectly fine; we can afford 1 KB of overhead on a modern connection. It's optimized as much as it needs to be.
The irony is, Web 2.0 was enabled by a performance improvement! It's only once JavaScript engines became fast enough to be useful that all these troubles on the Web started.
> Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters?
I think the analogy here is backwards. The better question is "how much would you prioritize a car that used only 0.05 liters per 100km over one that used 0.5? What about one that used only 0.005L?". I'd say that at that point, other factors like comfort, performance, base price, etc. become (relatively) much more important.
If basic computer operations like loading a webpage took minutes rather than seconds, I think there would be more general interest in improving performance. For now though, most users are happy-enough with the performance of most software, and other factors like aesthetics, ease-of-use, etc. are the main differentiators (admittedly feature bloat, ads, tracking, etc. are also a problem, but I think they're mostly orthogonal to under-the-hood performance).
These days, I think most users will lose more time and be more frustrated by poor UI design, accidental inputs, etc. than any performance characteristics of the software they use. Hence the complexity/performance overhead of using technologies that allow software to be easily iterated and expanded are justified, to my mind (though we should be mindful of technology that claims to improve our agility but really only adds complexity).
I have to be careful about what I describe, but I don't think people care about speed or performance at all when it comes to tech, and it makes me sad. In fact, there are so many occasions where the optimisation is so good that the end user doesn't believe that anything happened. So you have to deliberately introduce delay because a computer has to feel like it thinks the same way you do.
At my current place of employment we have plenty of average requests hitting 5-10 seconds and longer, you've got N+1 queries against the network, rather than the DB. As long as it's within 15 or 30 seconds nobody cares, they probably blame their 4G signal for it (especially in the UK where our mobile infrastructure is notoriously spotty, and entirely absent even within the middle of London). But since I work on those systems I'm upset and disappointed that I'm working on APIs that can take tens of seconds to respond.
The analogy is also not great because MPG is an established metric for fuel efficiency in cars. The higher the MPG the better.
> In fact, there are so many occasions where the optimisation is so good that the end user doesn't believe that anything happened. So you have to deliberately introduce delay because a computer has to feel like it thinks the same way you do.
I never liked this view. I can't think of a single legitimate use case that couldn't be solved better than by hiding your true capabilities, and thus wasting people's time.
> they probably blame their 4G signal for it
Sad thing is, enough companies thinking like this and the incentive to improve on 4G itself evaporates, because "almost nothing can work fast enough to make use of these optimizations anyway".
"I can't think of a single legitimate use case that couldn't be solved better than by hiding your true capabilities, and thus wasting people's time."
Consider a loading spinner with a line of copy that explains what's happening. Say it's for an action that can take anywhere from 20 milliseconds to several seconds, based on a combination of factors that are hard to predict beforehand. At the low end, showing the spinner will result in it flashing on the screen jarringly for just a frame. To the user it will appear as some kind of visual glitch since they won't have time to even make out what it is, much less read the copy.
In situations like this, it's often a good idea to introduce an artificial delay up to a floor that gives the user time to register what's happening and read the copy.
Flash is good. If the state transition is "no indicator -> spinner -> checkmark", then if the user notices the spinner flashing for one frame, that only ensures them the task was actually performed.
It's a real case, actually. I don't remember a name, but I've encountered this situation in the past, and that brief flash of a "in progress" marker was what I used to determine whether me clicking a "retry" button actually did something, or whether the input was just ignored. It's one of those unexpected benefits of predictability of UI coding; the less special cases there are, the better.
This doesn't work well in apps but games do incredible things to hide that state; and it's partially a consequence of avoiding a patent on minigames inside loading screens.
e.g. back in the 90s with Resi 1, the loading screen was hidden by a slow and tense animation of a door opening. It totally fit the atmosphere.
Plenty of games add an elevator or a scripted vehicle ride, or some ridiculous door locking mechanism that serves the same purpose without breaking immersion, especially as those faux-loading screens can be dynamic.
It's pretty much the exact same technique used in cinema when a director wants to stitch multiple takes into a single shot (e.g. that episode in True Detective; that other one in Mr Robot; all of Birdman).
> In fact, there are so many occasions where the optimisation is so good that the end user doesn't believe that anything happened. So you have to deliberately introduce delay because a computer has to feel like it thinks the same way you do.
I see this argument coming up a lot, but this can be solved by better UX. Making things slow on purpose is just designers/developers being lazy.
Btw users feeling uneasy when something is "too fast" is an indictment of everything else being too damn slow. :D
> "how much would you prioritize a car that used only 0.05 liters per 100km over one that used 0.5? What about one that used only 0.005L?". I'd say that at that point, other factors like comfort, performance, base price, etc. become (relatively) much more important.
I'll prioritize the 0.005L per 100km car for sure. That means the car can be driven for all its expected lifetime (500k km) in a single tank of gas, filled up at the time of purchase! That means there is a huge opportunity to further optimize for many things in the system:
- The car no longer needs to have a hole on the side for filling up. A lot of pipes can be removed. Gas tank can be moved to a safer/closer location where it is used.
- The dashboard doesn't need a dedicated slot for showing the fuel gauge, more wirings and mechanical parts removed.
- No needs for huge exhaust and cooling systems, since the wasted energy is significantly reduce. No more pump, less vehicle weights...
Of course, that 0.005L car won't come earlier than a good electric car. However, if it's there, I'd totally prioritize it higher than other things you listed. I think people tend to underestimate how small efficiency improvements add up and enable exponential values to the system as a whole.
The big problem is this, if we related this back to software it would mean the software being delivered in 10-15 years, rather than in 6 months. Kind of a big downside...
Not necessarily. For one, relating this doesn't remove the ability for incremental development. Another thing, there's very little actual innovation in software being done. Almost anything we use existed in some version in the past two or three decades, and it was much more faster, even if rougher at the corners. Just think how many of the startups and SaaS projects we see featured on HN week after week are just reimplementing a feature or a small piece of workflow from Excel or Photoshop as a standalone web app?
That's the old Ruby on Rails argument. In that specific case it only made sense when there were no similar frameworks for faster languages, but that's hardly the case today.
Ironically though, I'd be willing to bet that end-user performance on most traditional server-side-rendered apps using the "heavyweight" RoR framework is far better than the latest and greatest SPA approach.
In a previous life I did back office development for ecommerce. We had two applications, one RoR monolith and a "modern" JavaScript Meteor SPA. The SPA was actually developed to replace the equivalent functionality in the RoR application but we ended up killing it and sticking with what we had. Depending on what you're trying to accomplish server side rendering is just as good, if not better than the latest and greatest in client side rendering.
This is definitely an interesting take on the car analogy so thanks for posting it! I don't know that I agree 100% (I think I could 'settle' for a car that needed be be fueled once or twice a year if it came with some other noticeable benefits), but it is definitely worth remembering that sometimes an apparently small nudge in performance can enable big improvements. Miniaturization of electronics (including batteries and storage media) and continuing improvements to wireless broadband come to mind as the most obvious of these in the past decades.
I'm struggling to think of recent (or not-so-recent) software improvements that have had a similar impact though. It seems like many of the "big" algorithms and optimization techniques that underpin modern applications have been around for a long time, and there aren't a lot of solutions that are "just about" ready to make the jump from supercomputers to servers, servers to desktops, or desktops to mobile. I guess machine learning is a probably contender in this space, but I imagine that's still an active area of optimization and probably not what the author of the article had in mind. I'd love if someone could provide an example of recent consumer software that is only possible due to careful software optimization.
V8 would be one example. Some time ago, JavaScript crossed a performance threshold, which enabled people to start reimplementing a lot of desktop software as web applications. In the following years, algorithms for collaborative work were developed[0], which shifted the way we work with some of those applications, now always on-line.
That would be the meaningful software improvements I can think of. Curiously, the key enabler here seems to be performance - we had the capability to write web apps for a while, but JS was too slow to be useful.
--
[0] - They may or may not have been developed earlier, but I haven't seen them used in practice before the modern web.
> I'll prioritize the 0.005L per 100km car for sure. That means the car can be driven for all its expected lifetime (500k km) in a single tank of gas, filled up at the time of purchase!
It's a nice idea but it wouldn't work. The gasoline would go bad before you could use it all.
Plug-in hybrids already have this problem. Their fuel management systems try to keep the average age of the fuel in the tank under 1 year. The Chevy Volt has a fuel maintenance mode that runs every 6 weeks:
Instead of having a "lifetime tank", a car that uses 0.005L per 100km would be better off with a tiny tank. And then instead of buying fuel at a fuel station you'd buy it in a bottle at the supermarket along with your orange juice.
There is [1] https://duckduckgo.com/?q=alkylate+petrol which is said to last anything between two years and up to ten years, depending on the mixture while burning rather clean.
You are thinking too small, with a car generating power that cheaply you could use it to power a turbine and provide cheap electricity to the entire world. It would fix our energy needs for a very long time and it would usher a new age!
Or the car could just be very efficient. Gasoline has a lot of energy. Transporting a person 100km on 34MJ/l * 0.05l =1.71MJ doesn't sound as impossible as you make it seem.
Trains transports at 0.41 MJ/t·km. If the person weights 0.1t it would take a train packed full of people 41MJ per person to transport them 100km, or a bit more than one litre of gasoline. I don't think it is possible to go significantly below that without transporting them on mag rails or vacuum pipes.
Secondly we talked about 0.005l cars, not 0.05l, so it would be a few hundreds times more efficient than train transportation.
> These days, I think most users will lose more time and be more frustrated by poor UI design, accidental inputs, etc. than any performance characteristics of the software they use.
I’m willing to bet that a significant percentage of my accidental inputs are due to UI latency.
Virtually all of my accidental inputs are caused by application slowness or repaints that occur several hundred milliseconds after they should have.
I want all interactions with all of my computing devices to occur in as close to 0ms as possible. 0ms is great; 20ms is good; 200ms is bad; 500ms is absolutely inexcusable unless you're doing significant computation. I find it astonishing how many things will run in the 200-500ms range for utterly trivial operations such as just navigating between UI elements. And no, animation is not an acceptable illusion to hide slowness.
I am with the OP. "Good enough" is a bane on our discipline.
How about the i-am-about-to-press-this-button-but-wait-we-need-to-rerender-the-whole-page. At which point you misclick or not at all. Especially some recent shops and ad heavy pages use this great functionality ;)
The rule for games is that you have 16ms (for a 60Hz monitor) to process all input and draw the next frame. That's a decent rule for everything related to user input. And since there are high refresh-rate monitors, and it's a web app and not a game using 100% CPU & GPU, just assume 4-5ms for a nicer number. If you take longer than that to respond to user input on your lowest-capability supported configuration, you've got a bug.
0ms is great, 4ms is very good, 16ms is minimally acceptable, 20ms needs improvement (you're skipping frames), 200ms is bad (it's visible!), 500ms is ridiculous and should have been showing a progress bar or something.
Responding to input doesn't necessarily mean being done with processing, it just means showing a response.
Don’t get me started with all the impressive rotating zooming in Google Maps every time you accidentally brush the screen.
The usage story requires you to switch to turn-by-turn, and there’s no way to have bird eye map following your location along route (unless you just choose some zoom level and manually recenter every so often.)
It’s awful, distracting and frankly a waste of time... just to show a bit of animation every time I accidentally fail to register a drag...
Well, Google Maps is its own story - it's like the app is being actively designed to be as useless as possible as a map - a means to navigate. The only supported workflow is search + turn-by-turn navigation, and everything else seems to be disincentivized on purpose.
I dont know, that just feels wrong. If anything, the rise of mobile means there should be more emphasis on speed. All the bloat is because of misguided aesthetics (which all look the same, as if designers move between companies every year, which they do) and fanciness. Can you point to a newish app that is clearly better that its predecessor?
> All the bloat is because of misguided aesthetics (which all look the same, as if designers move between companies every year, which they do) and fanciness
That's not really true. Slack could be just as pretty and a fraction of the weight, if they hadn't used Electron.
I think there are two factors preventing mobile from being a force to drive performance optimizations.
One, phone OSes are being designed for single-tasked use. Outside of alarms and notifications in the background (which tend to be routed through a common service), the user can see just one app at a time, and mobile OSes actively restrict background activity of other apps. So every application can get away with the assumption that it's the sole owner of the phone's resources.
Two, given the above, the most noticeable problem is now power usage. As Moore's law has all but evaporated for single-threaded performance, hardware is now being upgraded for multicore and (important here) power performance. So apps can get away with poor engineering, because every new generation of smartphones has a more power-efficient CPU, so the lifetime on single charge doesn't degrade.
A UI where each interaction takes several seconds is poor UI design. I do lose most of my time and patience to poor UI design, including needless "improvements" every few iterations that break my workflow and have me relearn the UI.
I find the general state of interaction with the software I use on a daily basis to be piss poor, and over the last 20 or so years I have at best seen zero improvement on average, though if I was less charitable I'd say it has only gone downhill. Applications around the turn of the century were generally responsive, as far as I can remember.
I respectfully disagree -- something that is 10 times more efficient costs 10 times less energy (theoretically). When the end user suffers a server outage due to load, when they run out of battery ten times quicker, all of these things matter. When you have to pay for ten servers to run your product instead of one, this cost gets passed on to the end user.
I was forced to use a monitor at 30 fps for a few days due to a bad display setup. It made me realize how important 60 fps is. Even worse, try using an OS running in a VM for an extended period of time...
There are plenty of things that are 'good enough', but once users get used to something better they will never go back (if they have the choice, at least).
Yes, but it's not just relative quantities that matter, absolute values matter too, just as the post you replied to was saying.
Optimizing for microseconds when bad UI steals seconds is being penny-wise and pound foolish. Business might not understand tech but they do generally understand how it ends up on the balance sheet.
But the balance sheets encompass more than delivering value to end-users; business can and do trade off that value for some money elsewhere (see e.g. pretty much everything that has anything to do with ads).
Note also the potential deadlock here. Optimizing core calculations at μs level is bad because UI is slow, but optimizing UI to have μs responsiveness is bad, because core calculations are slow. Or the database is slow. This way, every part of the program can use every other part of the program as a justification to not do the necessary work. Reverse tragedy of the commons perhaps?
Another problem is that the inefficiency of multiple products tends to compound.
- Opening multiple tabs in a browser will kill your battery, and it's not the fault of a single page, but of all of them. Developers tend to blame the end user for opening too many tabs.
- Running a single Electron app is fast enough in a newer machine but if you need multiple instances or multiple apps you're fucked.
- Some of my teammates can't use their laptops without the charger because they have to run 20+ docker containers just to have our main website load. The machines are also noisy because the fan is always on.
- Having complex build pipelines that take minutes or hours to run is something that slows dow developers, which are expensive. It's not the fault of a single software (except maybe of the chosen programming language), but of multiple inefficient libraries and packages.
> Even worse, try using an OS running in a VM for an extended period of time...
I do that for most of my hobbyist Linux dev work. It's fine. It can do 4k and everything. It's surely not optimal but it's better than managing dual boot.
Host is Windows, guest is Ubuntu. Hypervisor is VMWare Workstation 12 Player. There is a very straightforward process to get graphics acceleration in the VM. The shell has a "mount install CD" option that causes a CD containing drivers to be loaded in the guest (Player > Manage > Reinstall VMWare Tools). You install those, and also enable acceleration in the VMWare settings (https://imgur.com/a/PUaE38u). Again, it's not perfect, but I can e.g. play fullscreen 1080p YouTube videos. Not sure how it would like playing 4k videos, but my desktop doesn't like that so much even in the host OS.
I do this the other way around, Ubuntu host and a KVM virtual machine controlled by virt-manager with PCIe passthrough for its own GPU and NVMe boot drive. I enjoy Linux too much for daily use (and rely on it for bulk storage with internal drives mergerfs fused together and backed up with snapraid), but I do a lot of photography and media work so I also rely on Windows. This way, I can use a KVM frame relay like looking-glass to get a latency free almost native performance windows VM inside a Ubuntu host, without the need to dual boot (but since the NVMe drive is just windows, I can always boot into windows if I please)
> "Even worse, try using an OS running in a VM for an extended period of time..."
I actually do this for development and it works really well.
Ubuntu Linux VM in VMware Fusion on a Macbook Pro with MacOS.
Power consumption was found to be better than running Linux natively. (I'm guessing something about switching between the two GPUs, but who knows.)
GPU acceleration works fine; the Linux desktop animations, window fading and movement animations etc are just as I'd expect.
Performance seems to be fine generally, and I do care about performance.
(But I don't measure graphics performance, perhaps that's not as good as native. And when doing I/O intensive work, that's on servers.)
Being able to do a four-finger swipe on the trackpad to switch between MacOS desktops and Linux desktops (full screen) is really nice. It feels as if the two OSes are running side by side, rather than one inside another.
I've been doing Linux-in-a-VM for about 6 years, and wouldn't switch back to native on my laptop if I had a choice. The side-by-side illusion is too good.
Before that I ran various Linux desktops (or Linux consoles :-) for about 20 years natively on all my development machines and all my personal laptops, so it's not like don't know what that's like. In general, I notice more graphics driver bugs in the native version...
(The one thing that stands out as buggy is VMware's host-to-guest file sharing is extremely buggy, to the point of corrupting files, even crashing Git. MacOS's own SMB client is also atrocious in numerous ways, to the point of even deleting random files, but does it less often so you don't notice until later what's gone. I've had to work hard to find good workarounds to have reliable files! I mention this as a warning to anyone thinking of trying the same setup.)
What year MBP is this? I tried running Ubuntu on Virtual Box on my mid 2014 MBP with 16GB ram, but that was anything but smooth. I ended up dual booting my T460s instead.
But perhaps the answer is VMware Fusion instead then.
Having your VM stored inside a file on a slow filesystem is bad. Having a separate lvm volume (on linux)/zvols (with zfs)/partition/disk is much more performant.
I store my Linux VM disk inside a file on a Mac filesystem (HFS+, the old one), and I haven't noticed any significant human-noticable I/O latency issues when using it. The Linux VM disk is formatted as ext4.
That's about human-scale experience, rather than measured latency. It won't be as fast as native, but it seems adequate for my use, even when grepping thousands of files, unpacking archives, etc, and I haven't noticed any significant stalling or pauses. It's encrypted too (by MacOS).
(That's in contrast to host-guest file access over the virtual network, which definitely has performance issues. But ext4 on the VM disk seems to work well.)
The VM is my main daily work "machine", and I'm a heavy user, so I'd notice if I/O latency was affecting use.
I'm sure it helps that the Mac has a fast SSD though.
(In contrast, on servers I use LVM a lot, in conjunction with MD-RAID and LUKS encryption.)
I've only given Linux 6GB RAM at the moment, and it's working out fine. Currently running Ubuntu 19.10.
I picked VMware Fusion originally because it was reported to have good-ish support for GPU emulation that was compatible with Linux desktops at the time. Without it, graphics can be a bit clunky. With it, it feels smooth enough for me, as a desktop.
My browser is Firefox on the Mac side, but dev web servers all on the Linux side.
The VM networking is fine, but I use a separate "private" network (for dev networking) from the "NAT" network (outgoing connections from Linux to internet), so Wifi IP address changes in the latter don't disrupt active connections of the former.
My editor is Emacs GUI on the Mac side (so it integrates with the native Mac GUI - Cmd-CV cut and paste etc, better scrolling), although I can call up Emacs sessions from Linux easily, and for TypeScript, dev language servers etc., Emacs is able to run them remotely as appropriate.
Smoothness over SSH from iTerm is a different thing from graphical desktop smoothness.
When doing graphics work (e.g. Inkscape/GIMP/ImageMagick), or remote access to Windows servers using Remmina for VNC/RDP, I use the Linux desktop.
But mostly I do dev work in Linux over SSH from iTerm. I don't think I've ever noticed any smoothness issues with that, except when VMware networking crashes due to SMB/NFS loops that I shouldn't let happen :-)
>And build times? Nobody thinks compiler that works minutes or even hours is a problem. What happened to “programmer’s time is more important”? Almost all compilers, pre- and post-processors add significant, sometimes disastrous time tax to your build without providing proportionally substantial benefits.
FWIW, I did RTFA (top to bottom) before commenting. I chose to reply to some parts of the article and not others, especially the parts I felt were particularly hyperbolic.
Anecdotally, in my career I've never had to compile something myself that took longer than a few minutes (but maybe if you work on the Linux kernel or some other big project, you have; or maybe I've just been lucky to mainly use toolchains that avoid the pitfalls here). I would definitely consider it a problem if my compiler runs regularly took O(10mins), and would probably consider looking for optimizations or alternatives at that point. I've also benefited immensely from a lot of the analysis tools that are built into the toolchains that I use, and I have no doubt that most or all of them have saved me more pain than they've caused me.
Then you're being disingenuous in picking a quarter of the quote.
>You’ve probably heard this mantra: “Programmer time is more expensive than computer time.” What it means basically is that we’re wasting computers at an unprecedented scale. Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time.
The point is that we are wasting all the resources at every scale. We are supposedly burning computer cycles because developer time is more important. Yet we are also burning developer time with compiling, or testing for interpreted languages, at a rate that is starting to approach the batch processing days.
I use webpages for most of the social networking platforms such as Facebook. I am left handed and scroll with my left thumb (left half of the screen). I have accidentally ‘liked’ people’s posts, sent accidental friend requests only because of this reason.
Guessing along with language selection, it might be helpful to have a selection of hand preference for mobile browsing.
I think objections like this may be put in terms of measurable cost-benefits but they often come down to the feeling of wasted time and effort involved in writing, reading and understanding garbage software.
Moreover, the same cost-equation that produces software that is much less efficient than it could be produces software that might be usable for it's purpose (barely) but is much more ugly, confusing, and buggy than it needs to be.
That equation is add the needed features, sell the software first, get lock in, milk it 'till it dies and move on. That's equation is locally cost-efficient. Locally, that wins and that produces the world we see every day.
Maybe, the lack of craftsmanship, the lack of doing one's activity well, is simply inevitable. Or maybe the race to the bottom is going to kill us - see the Boeing 737 Max as perhaps food for thought (not that software as such was to blame there but the quality issue was there).
> admittedly feature bloat, ads, tracking, etc. are also a problem, but I think they're mostly orthogonal to under-the-hood performance
I think for webpages it is the opposite: non-orthogonal in most cases.
If you disable your JS/Ad/...-blocker, and go to pages like Reddit, it is definitely slower and the CPU spikes. Even with a blocker, the page still does a thousand things in the first-party scripts (like tracking mouse movements and such) that slow everything down a lot.
The analogy is wrong as well because a car engine is used for a single purpose, moving the car itself. Imagine if you had an engine that powered a hundred cars instead, but a lot of those cars were unoptimized so you can only run two cars at a time instead of the theoretical 100.
or... something.
The car analogy does remind me of one I read a while ago, comparing cars and their cost and performance with CPUs.
I wonder how this trend will be affected by the slowing of Moore’s law. There will always be demand for more compute, and until now that’s largely been met with improvements in hardware. When that becomes less true, software optimization may become more valuable.
What are you up to these days Chris? And the obligatory question, what is your take on the article/why post?
I piled standout quotes below.
I think a big takeaway from the intersection of Bret Victor, Alan Kay, Jim Hollan and the ink&switch folks and your work is that the right dynamic interface can be the "place we live in" on the computer.
Victor shows a history of interactive direct manipulation interfaces, live environments where explorations of models or the creation of art go hand in hand with everything else related to that task: data input, explicit (programmatic) requirements and the visual output.
Hollan and ink&switch show the environment (ZUIs, canvas) can contain everything for doing work, the code alongside any manipulation of the viewport that can be conceived. Tools infinitely more advanced than Microsoft OneNote and designed 40 years ago.
From what I know about your work, I see another take on the environment I want to live in on the computer. I dont understand why I would want to lose power by stepping away from my language/interpreter/compiler/repl into a GUI or some portal when I can bring whatever it is which is nice about GUIs or portals into my dynamic computing environment. I very much want a personal DSL or set of DSLs for what I do on the computer, and I want to be able to hook into anything ala middle mouse button in plan9.
The superior alternative to walled gardens and this absurd world of bloat and 'feature loss' (for lack of a better term for software engineering's enthusiastic rejection of history) seems to be known, and facets of it advocated by you and these others. It seems clear that "using the computer" needs to return to "programming the computer" and that to achieve that we need to fundamentally change "programming the computer" to be a more communicative activity, to foster a better relationship between the computer and the user.
Where is this work being done now? VPRI shut down 2 years ago, Dynamicland seems to be on hiatus? I am inspired most these days by indie developers who write their own tools and build wild looking knowledge engines or what they sometimes call "trackers."[1] And of course the histories and papers put forward by the above and their predecessors. And I play with my own, building an environment where I can write, draw, code, execute and interact with it all. I see no existing product which approaches what I want.
> Everyone is busy building stuff for right now, today, rarely for tomorrow.
> Even when efficient solutions have been known for ages, we still struggle with the same problems: package management, build systems, compilers, language design, IDEs.
> You need to occasionally throw stuff away and replace it with better stuff.
> Business won’t care Neither will users. They are only learned to expect what we can provide.
> There’s no competition either. Everybody is building the same slow, bloated, unreliable products.
> The only thing required is not building on top of a huge pile of crap that modern toolchain is.
> I want something to believe in, a worthy end goal, a future better than what we have today, and I want a community of engineers who share that vision.
I'm working on tools/interfaces at Relational AI, which is doing really cool work in the declarative languages space. It was started by several of the folks whose papers were foundational to Eve. :)
I agree with the post, though as others have pointed out, it doesn't really dive into the fact that this problem is systemic and would require a shift in incentive structure.
I think the last quote you have is one of the most important missing pieces for making a meaningful change in this space. A lot of people want something better, but right now, as a community, I don't think we really know what that is. What is the complete story for an ideal version of software development? And by that I don't mean idealized examples, I mean the ideal version of the real process we have to go through. What does perfect look like in the world of changing requirements, shifting teams, legacy systems, crappy APIs, and insufficient budgets? If we could show that - not the simple examples we had for Eve, but something that addresses the raw reality of engineering - I think it would just be a matter of beating the drum.
Companies and management are preferring devs with strong interpersonal skills that have weak programming skills, this leads to software that just works, slowly slowly slowly.
Hire
More
Autists!
I'm so weary of this moral panic we've been having. There are so many other factors to be weighed against efficiency when it comes to making software. There are completely legitimate tradeoffs to be made that sacrifice performance. There are also programmers who write bad code on all dimensions - performance included - out of sheer laziness. But those aren't the primary cause of this hardware "waste". Demanding efficiency for efficiency's sake, ignoring all other constraints, is shortsighted and narrow-minded.
Yeah, I really dislike the black and white thinking. The python script example the author gives is a perfect example of what doesn’t need to be made any faster. If you are interested only in execution time, you might as well never write anything in native python.
But on the other hand, a lot of web content does need to be faster. Gmail has somehow gotten so much slower to load over time. And every time I visit a newspaper/magazine website I am aghast at how bloated they are. Does that mean nodejs is inherently bad, no, but it does mean people should try to optimize noticeably terrible performance that actually degrades UX.
Sure. But even that probably has more to do with businesses prioritizing features over quality, not programmers lacking character.
Much of it on the web also has to do with how much browsers can do. The number of CSS properties that can be applied, the ways different elements' sizes can automatically influence the layout of other elements, etc. These traits are what make the web such a powerful and attractive platform for user interfaces, but the complexity of the platform is definitely becoming a real issue that deserves attention.
A couple of points:
- NodeJS is server-only and usually has nothing to do with perceived performance of web apps
- The biggest offender of web performance is ads. They dump piles and piles of crappy JavaScript from dozens of different sources that all include their own copies of common libraries and have no incentive not to slow down the page.
- Beyond ads, the bottleneck is usually not even JavaScript, but layout (as in the paragraph above). Web layout is incredibly flexible and incredibly complex. Computing and rendering it all is slow, but it does serve a purpose. Not that it couldn't be improved.
Bad ads are a tragedy of the commons and I don't know what can be done about them unless Google or Facebook decided to throw their weight around to force them to be better.
I do wonder if a new web standard could be developed for using some constrained subset of the layout vocabulary, that would be cheaper and more straightforward to compute. The current version has to remain for backwards compatibility reasons, but it's trying to serve a bunch of different types of cases at once, and therefore doesn't do a great job at any one of them.
The thing is people are not looking at the less visible stuff hidden in scope.
A mere fraction of a second increase in the time you don't have to wait after klicking something? Barely noticeable. Perfectly usable. Hardly worth the time investment to improve... after all that dev's expensive and then that button or whatever gets used by countless people. That fraction of a second ends up actually being numerous lifetimes.
And as it's surrounded by numerous such "slightly inefficient but efficient from some other perspective" interactions overall efficiency dies a death of a thousand cuts.
When you actively start paying attention to it and comparing it to little examples of what could be you start noticing how utterly garbage everything is.
Even at the base level in things that have a massive userbase and that are used not just once in a while but constantly it's disgusting trash.
I look around at the company i work at.
They're all using it and probably not noticing at all but the fucking file explorer in Windows is slow as fuck as is countless other elements and interactions of it.
The companies website so simple in it's content and functionality and is made by a webdev agency but is a bloated mess that takes a while to get to....a logo that shows whilst it continues to load.
The software my coworker wrote is small scale and he said i was wasting a lot of my time making some small action faster not recognising that it's been used many thousand times a day every day for more than 10 years now.
He seems to make a contradictory point... he complains:
> iOS 11 dropped support for 32-bit apps. That means if the developer isn’t around at the time of the iOS 11 release or isn’t willing to go back and update a once-perfectly-fine app, chances are you won’t be seeing their app ever again.
but then he also says:
> To have a healthy ecosystem you need to go back and revisit. You need to occasionally throw stuff away and replace it with better stuff.
So which is it? If you want to replace stuff with something better, that means the old stuff won't work anymore... or, it will work by placing a translation/emulation layer around it, which he describes as:
> We put virtual machines inside Linux, and then we put Docker inside virtual machines, simply because nobody was able to clean up the mess that most programs, languages and their environment produce. We cover shit with blankets just not to deal with it.
I'd argue that of any software project on the planet, Windows is the closest to having unlimited resources; especially when you consider the number of Windows customers for whom backwards compatibility is the #1 feature on the box.
And speed isn't the only metric that matters; having both the 32-bit and 64-bit versions of DLLs uses a non-trivial (to some people) amount of disk space, bandwidth, complexity, etc.
Surely, Apple and Google have just about as many resources as Microsoft does.
If Android, Mac OS, etc were super slimmed down systems in comparison to Windows, I would understand the argument much better. Instead, it feels like we're in the worst of both worlds.
Yeah, didn't say it's impossible. I said it's a tradeoff.
Windows does it and pays for it with slower releases, more engineers, bugs, strange interaction between old and new, several layers of UI and API code for devs to decode and for users to be confused with, less ability to move to new paradigms (why would devs bother if the old work), 2 versions of libs loaded (32/64 bit), and several other ways besides...
E.g. I've stopped using Windows for a decade or so, but I read of the 3 (4?) settings panels it has, the modern, the Vista style, the XP style, and so on, with some options in one, the others in the other (if you click some "advanced" menu, etc).
And yet at the time of its release, iOS 11 was the most buggy version in recent memory. (This record has since been beaten by iOS 13.)
I don't quite know what's going on inside Apple, but it doesn't feel like they're choosing which features to remove in a particularly thoughtful way.
---
Twenty years ago, Apple's flagship platform was called Mac OS (Mac OS ≠ macOS), and it sucked beyond repair. So Apple shifted to a completely different platform, which they dubbed Mac OS X. A slow and clunky virtualization layer was added for running "classic" Mac OS software, but it was built to be temporary, not a normal means of operation.
For anyone invested in the Mac OS platform at the time, this must have really sucked. But what's important is that Apple made the transition once! They realized that a clean break was essential, and they did it, and we've been on OS X ever since. There's a 16-year-old OS X app called Audio Slicer which I still use regularly in High Sierra. It would break if I updated to Catalina, but, therein lies my problem with today's Apple.
If you really need to make a clean break, fine, go ahead! It will be painful, but we'd best get it over with.
But that shouldn't happen more than once every couple decades, and even less as we get collectively more experienced at writing software.
The goal is that you throw out things that aren't useful (cost > benefit, or better replacement available and easily usable), not that you have a periodic "throw out everything written before X".
I think that's not quite the point in the article. The idea is, in my reading, that we've built lazily on castles of sand for so long that sometimes we think it makes sense to throw away things we shouldn't, and other times we obsessively wrap/rewrap/paper over things we should throw away. What falls into each category is obviously debatable, but the author seems to be critiquing the methodology we use to make those decision--debatable or not, people aren't debating it so much as they're taking the shortest and often laziest path without prioritizing the right things (efficiency, consistency).
Even with our priorities in order, there will still be contentious, hard choices (to deprecate so-and-so or not; to sacrifice a capability for consistency of interface or not), but the author's point is that our priorities are not in order in the first place, so the decisions we make end up being arbitrary at best, and harmful/driven by bad motivations at worst.
See also: in Good times create weak men [0], the author explains his interpretation as to why. I can't summarize it well. It's centered around a Jonathan Blow talk [1] Preventing the collapse of civilization.
I watched that talk a while ago. It is great, and it did change my opinion on a few things. Whether you agree with the premise or not, you can still learn something. For me, the importance of sharing knowledge within a team to prevent "knowledge rot". "Generations" in a team are much more rapid than the general population/civilisation, so that effect is magnified IMO.
I think people who recognise this have generally been developing from the days where computing resources were scarce (or on mediums now where they need to be efficient). It was a necessecity to implement efficient techniques instead of a nice to have. Nowadays those restrictions have been lifted for the most part.
In this day of "Agile" development, as long as something's working during UAT, that's all that's needed for sales and consumers.
Webdev, IME, is an example where the ecosystem has facilitated bloated websites. I've worked with developers who throw any library they can just for basic things because they don't have a need to try to optimise. The meme of using jQuery for everything when it came out has just been replaced by other frameworks. I find it often depends on developers who really want to work on something and take pride in it vs those who just need something on their CV or got hired by following a few tutorials on the web but not understanding what they wrote (which, to me, signifies a hiring problem in the company). During code reviews, I encourage leads to keep calling up hacky code to a point where the developer will just start writing it properly the first time round. As developers, I feel we should be aware of not creating selfish software which hogs memory from other software or requires huge data downloads for mobile users (whenever doable). Possibly a naive ideal but if it's a byproduct of developing fast software for my end users, I think that's a win-win.
Indeed, and pragmatism should be applied, but I mean in the context of not being rushed. I don't mind my team watching YouTube/browsing the web during work if things are going well but I wouldn't accept it if it's done after submission of suboptimal code. If there's time to watch YT, there's time to improve your code (unless it's clearly too much of a refactor).
> Jonathan Blow has a language he alone develops for his game that can compile 500k lines per second on his laptop. That’s cold compile, no intermediate caching, no incremental builds. You don’t have to be a genius to write fast programs.
Depends on your definition of genius, but I definitely agree that these folks don't quite hold up the sentiment that "anyone can do it." I would put Martin Thompson, Raph Levien, and Jonathan Blow at least in the top 0.1% of programmers.
They are great examples for his overall point though. It probably would've been better just to leave out the genius bit and talk about them as folks proving it can be done.
Yeah, there are many others I'd add to a more general list - Carmack, Bellard, Wirth, the folks from Our Machinery, etc. I was just referencing the people specifically mentioned in the original post.
One thing nobody seems to mention is the environmental cost of inefficient software. All those wasted CPU cycles consume electricity. A single laptop or phone on it's own is insignificant, but there are billions of them. Combine that with the energy wasted shovelling unnecessary crap around the internet, and it adds up to a big CO2 problem that nobody talks about.
I hear that argument very frequently and I don’t buy it.
Think about all the gas that is saved because people don’t have to drive to the library, all the plane trips saved by video conferencing, all the photo film, all the sheets of paper in file cabinets, all the letters being sent as emails, all the mail order catalogues, ... you get the idea.
Does anybody know of a comprehensive study on this?
> Think about all the gas that is saved because people don’t have to drive to the library....
You're right that computers have saved huge amounts of energy compared with the things you mention. My point here was that even more could be saved with a bit of thought about efficiency in programming.
If websites and business software were as lean as they could be, most computers could have amazingly weak, low-powered processors.
I'm quite disenchanted with software myself. It takes way too long to open any program, for this JIRA ticket to properly display.
One thing that has improved was boot times, I seem to remember that Windows 7 was quite a bit faster than XP. Maybe someone in upper management wanted it to be as fast as MacOS? So speed IS possible, if it is prioritized.
> One thing that has improved was boot times, I seem to remember that Windows 7 was quite a bit faster than XP. Maybe someone in upper management wanted it to be as fast as MacOS? So speed IS possible, if it is prioritized.
I seem to remember boot times being a frequent topic of discussion the early 2000s, because people turned off their computers.
In a way, this is a great little microcosm of the problem. Just fix habits instead of fixing the software.
It's not either-or. I don't buy the argument that if we didn't shovel garbage that we call "software" today, we wouldn't have equivalent but better software at all. It's a multi-agent problem, and a lot of it is driven by business dysfunction, not even actual complexity or programmer laziness.
In my - perhaps limited - work experience, there's enough slack in the process of software development that I don't buy the "time to market" argument all that much.
I agree but apparently the number is not very big - computing is max 8% perhaps of electricity usage. But it still feels so wasteful, and also wasteful of people’s time.
I think it's actually the time that matters the most. I think the tens of thousands human lifetimes killed each day on slow software we're forced to use really adds up, how exactly I don't know, it's hard to imagine what humanity would do with the millions of work hours saved.
I'm not sure in the accounting of the environmental costs of modern life that inefficient software counts for much. Doing totally pointless crap with highly efficient software might be worse.
Sure: If you can optimize the software that runs on many millions of computers then you can have a huge impact. If you run those computers yourself you can even save money.
But the vast majority of software is one-off stuff. It makes no sense to optimize it for performance instead of features, development time, correctness, ease of use, etc.
> But the vast majority of software is one-off stuff.
Is it now? I can't think of any example, except a few tools we use internally in a project. Everything else I use, or I see anyone else using, has an userbase of many thousands to millions, and a lot of that is used in context of work - which means a good chunk of the userbase is sitting in front of that software day in, day out.
It'd be interesting to know, say, what percentage of software developers work on programs that have less than a million users.
I wouldn't be at all surprised if it's the majority. For every big-ticket software offering there's going to at least be things the user never interacts with, like a build system and a test suite and a bug tracking system and whatever else. And there's just so much software everywhere, most of which we never see. Every small business has its little web site. Who knows how much software there is powering this or that device or process at random factories, laboratories, and offices.
The Debian popularity contest looks like it has a big long tail of relatively unpopular packages [1]. It looks like the app store has 2 million apps, only 2857 (.14%) have more than a million dollars of annual revenue. These are of course incomplete and flawed and do not really directly address the question. I don't really know how to research this in a thorough way.
I've heard this argument often, but I don't buy it.
1. The environmental cost of ineffective software is negligible, when compared to Bitcoin mining or other forms of hardware planned obsolescence.
2. By using more efficient software, surely, you can save a lot of CPU cycles, and it can improve the energy efficiency of some specific workloads under some particular scenarios. However, on a general-purpose PC, the desire for performance is unlimited, the CPU cycles saved in one way will only be consumed in other ways, and in the end, the total CPU cycles used remain a constant.
Running programs on a PC is like buying things, when you have a fixed budget but everything is cheaper, often people will just buy more. For example, I only start closing webpages when my browser becomes unacceptably slow, but if you make every webpage use 50% less system resource, I'll simply open 2x more webpages simultaneously. LED lightning is another example, while I think the overall effect is a reduction of energy uses, but in some cases, I think it actually makes people to install more lightnings, such as those outdoor billboards.
This is called the Jevons paradox [0].
For PCs, certainly, as I previously stated, in specific workloads under some particular scenarios, I totally agree that there are cases that energy use can be reduced (e.g. faster system update), but I don't think it helps much in the grand scheme of things.
Perhaps it's more that most (not all) of the many programmers today are not as good as the (relatively fortunate) few who had access to computers in the past.
In fact, I'd go so far as to suggest there are more good programmers about today then there were in the past. Though having said that, percentage-wise I'd say the profession is definitely being deskilled (good thing if you're a manager, bad thing if you're a good programmer).
I agree with the overal sentiment, yet the examples could be better.
> Modern text editors have higher latency than 42-year-old Emacs. Text editors! What can be simpler? On each keystroke, all you have to do is update a tiny rectangular region and modern text editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D game can fill the whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come?
Text is very complicated. Does your 42-year-old Emacs support Unicode? And not just accents, but whole different scripts?
I'm not sure if the author meant current versions of emacs (with their 42 year lineage) or not. But if that's the case, then I think emacs was one of the very first to support Unicode and whole different scripts. If my memory does not fail me, emacs had extensive wide character support long before Unicode. With an extraordinarily broad treasure trope of different input methods on top.
emacs has also always been (in)famous for its text update render algorithms, that probably still work well over the old slow terminal lines it was originally used for.
I was in Rome recently, and google maps were basically unusable on EDGE (dsepite pre-downloading the area before the trip). We'd wait a minute (or more) for a timetable of a bus stop and a route of the bus to be show on the map.
Try planning a route in an unfamiliar area with this slow an UI when you are standing outside and there's no place to sit and rest, and you need to click around on a bunch of stops just to see what busses are going through a stop and where they are heading.
So yes, optimization is still important.
We replaced the glorious and easily iterated and expanded google maps app with a photograph of a public transport map, and we could get an answer of how to get from any A to any B within seconds of looking at the map without typing or searching or waiting for anything.
Which also shows that, sometimes, slow software is less than useless.
Replace Rome with the German countryside and you'll still just have EDGE.
Sites like Hacker News or https://i.reddit.com are still perfectly usable. In contrast to the 'modern' Reddit UI that that takes a couple of seconds to load even on my home WiFi.
It would be nice to have a maps/routing service that could just produce a basic HTML page with text directions from a given point A to another given point B.
Bonus points if it also lists the before and after streets at direction changes so people know if they missed a turn.
This could be implemented as a proxy to either of the maps apps (Google, Apple, OpenStreetMap)
> In contrast to the 'modern' Reddit UI that that takes a couple of seconds to load even on my home WiFi.
On my phone it takes about 4-5 seconds to load even on Gbit WiFi. It's so bad I have a conspiracy theory that the pulsing Reddit orb is just on a timer and it's not actually loading anything. More likely it's just horrifically unoptimized I suppose.
(not gp) I have T-Mobile service in the USA, and my plan includes free international phone and data, but only at EDGE speeds. I could get a local SIM, but it's rarely worth it if I'm traveling to multiple countries.
Google Maps is just horrible on mobile when you aren't on Wifi. I'd like to wireshark it some day to figure out just how many different web requests it is making.
I was traveling in Europe for a few months and I didn't care to change my phone plan so I was stuck with 2G speeds the entire time. It gave me a profound appreciation for web applications that load the minimal amount of code/resources/whatnot to display. HN really shines here. :)
I agree with your general point, but in this particular case bandwidth is muddying the waters. 2g tech has less bandwidth dedicated to it now than it did years ago, so your kbps is lower now than it would have been when using 2g in the past.
iOS has evolved into such a buggy mess I switched to Android. I used to be a huge Apple fan, and now I don't use any Apple products. I feel they are one of the worst offenders in degrading software quality, unfortunately.
I think we need a guild. We need licensed software engineers.
Not every programmer needs to be one, just like not every engineer needs to be licensed, but there needs to be a licensed engineer on every team. And, of course, sometimes there doesn't need to be. But I sure wish there was the option.
And hell, bring back apprenticeships and mentoring with the guild. There is so much we could learn from the physical science engineering disciplines
It's true. We do. People don't want CS degrees to mean a "programming" degree, and this is one way to do. Create guilds, apprenticeships, and mentoring so that when companies push for faster code, more programmers, etc, we have a solution.
While the apprenticeship model is a good one and can work in software, I would be wary of such an effort considering how rapidly software evolves. Guilds work well for technologies that are fairly stable and require years of practice and study to get right (metal working, carpentry, plumbing, surgery etc.), while with software you could specialize in languages and frameworks that become obsolete in the order of decades.
However, the idea of working closely with Senior Engineers and learning from them is certainly something that I vehemently agree with. I've been fortunate to have had that opportunity.
While software engineering is growing and evolving, there's also a lot of cyclical fads that we should look past. Sure, there is the framework du jour, but the fundamentals of computer science and software engineering grow far slower.
Mastery of frameworks is NOT mastery of our craft. They are useful tools that can come and go. But the underlying principles are what should concern is.
To that point, none of your examples of stable disciplines are static. New surgical tools, techniques, and technology are constantly produced and surgeons must learn to lay down their old ones and adapt. Metalworking, carpentry, plumbing all need to learn about new materials developed and code changes.
FTA:
> As a general trend, we’re not getting faster software with more features. We’re getting faster hardware that runs slower software with the same features.
Or fewer features. Talk to an actual professional who uses spreadsheets all day long about switching from Excel to Google sheets. It's infuriating the infantilization of UIs and the "oh they'll never miss it" attitude.
504 comments
[ 2.7 ms ] story [ 322 ms ] threadAll of this bloated 'shitware' today is the result of it having been written by people who a) have no deeper understanding of what the computer is actually doing; also known as typical Python/Java/etc/etc/etc programmers, and/or b) simply not giving a damn about conservation of resources--as further evidenced by all of the other extremely wasteful and destructive habits they hold in their personal lives, and in their societies in general.
After all, this is the same civilization that's burning through increasingly vast quantities of oil at an astounding rate, despite the fact that previously existing abundant and cheap oil is nearly depleted, with no possibility of replenishment or replacement. So is it any surprise that foolish developers also burn through CPU and memory with reckless abandon?
Really, the problems we face aren't just in software; they're more about the foundations of our entire Western 'civilization.' Such problems generally tend to be rather intractable, in the historical view.
I'm working to construct, in my own computing life, something of a 'personal oasis', which is increasingly removed and estranged from all of the horrible things I see Other People out there having to suffer in their personal computing lives, thanks to talentless 'developers' who Just Don't Fucking Care. Some of these pricks actually have the audacity to call themselves 'engineers', even.
My "oasis" is just text files in git, or photos in directories for the most part. The challenge is integrating with others and my god damn iPhone.
Much of the blame lies with bloated frameworks, such as Electron, rather than with the developer who uses the framework. You've covered this under Option B.
This topic cropped up recently on a Show HN. Someone built a Slack client for Windows 3.1. [0] It uses a tiny fraction of the memory used by the official Electron-based client, of course.
[0] https://news.ycombinator.com/item?id=21832815
Not just software, see houses, furniture, consumer goods, etc.
I see no point in getting angry, you are in control of what you use.
That statement contradicts itself. Is this satire?
> All of this bloated 'shitware' today is the result of it having been written by people who a) have no deeper understanding of what the computer is actually doing; also known as typical Python/Java/etc/etc/etc programmers, and/or b) simply not giving a damn about conservation of resources--as further evidenced by all of the other extremely wasteful and destructive habits they hold in their personal lives, and in their societies in general.
Everything has a cost / benefit associated with it. Pretending otherwise show how little you know about engineering.
> I'm working to construct, in my own computing life, something of a 'personal oasis', which is increasingly removed and estranged from all of the horrible things I see Other People out there having to suffer in their personal computing lives, thanks to talentless 'developers' who Just Don't Fucking Care. Some of these pricks actually have the audacity to call themselves 'engineers', even.
I dunno how far your head has to be up your own behind to actually believe this? It not like most developers these days work on large software projects that are normally poorly costed, estimated, planned usually with stifling restrictions because certain "enterprise" technology is mandated by some architect who hasn't written a line of code in a couple of decades. Most people like a consistent regular income which allows them to support themselves and their family. I suspect these concerns are more important then incoherent ramblings of some guy on HN.
My comment was the first one on this article. 473 comments later, it looks like over 90% of the other commenters agree with my observations, considering 90% of the other posts could have been written by me also. So the real question is, what do we know that you don't?
I started programming in 1991. Since the 90s I've watched as software has been steadily circling the drain. Many others have noticed it too, as shown by this discussion. The growing discontent is palpable. Where have you been during this time?
> 'Everything has a cost / benefit associated with it. Pretending otherwise show how little you know about engineering.'
Typical excuse from a lazy, talentless hack, excusing his shit code. What you are doing is not engineering.
> 'It not like most developers these days work on large software projects that are normally poorly costed, estimated, planned usually with stifling restrictions because certain "enterprise" technology is mandated by some architect who hasn't written a line of code in a couple of decades. Most people like a consistent regular income which allows them to support themselves and their family. I suspect these concerns are more important then incoherent ramblings of some guy on HN.'
Typical, common, ordinary, everyday selfishness. "Well I thought about taking a stand for what's right, but decided against it Because Reasons." Pathetic.
What are you doing that is actively making the world of computers, or even the world in general, a better place? Is that something you care about at all? Or do you prefer to just cash that check while making bullshit excuses?
I live in a cabin in the woods just because I'm not going to swim in the shark pit that you call home. Nor am I going to actively fuck up the world just to make a paycheck. It's called having principles.
> they're more about the foundations of our entire Western 'civilization.'
> talentless 'developers'
It's not talentless developers, it's the foundations of civilization. Capitalism, chasing the $ not the writing of beautiful software. If an Electron app makes me more money (e.g. by getting to market quicker) than a well crafted native app, then the Electron app will be built. See also building construction and why we end up with 60m^2 poorly constructed but funky looking apartments - who to blame. The concrete pourers? The architect? I don't think it is that simple.
Even my e-book reader Linux port boots to UI in ~2s.
It really is just bloat and lack of care.
I shut it down and repeated that, looking at my wristwratch.
FOUR SECONDS!
I couldn't believe that and tested maybe five times again. It took never more than 4 seconds from rocking the switch to WfW 3.11 Desktop with Word 6 and Excel 5 in autostart ready to use! It even got a DHCP address while being at it.
The thing is, at that time i had some Sun SparcStations, some HP PA-Risc Workstations, and assorted X86 PCs, one of them an AMD Athlon XP PR1800 overclcocked to PR2100 with 1.5 GB virtual channel memory SD-Ram, running NetBSD (because it just worked, don't ask). I felt very ahead of 'the curve'.
And then this trash came along and burst my bubble...
I sat there and only thought: 'what for?!'
Now some old Windows running atop of DOS isn't something to really envy, but in this combination of components, BIOS, drivers it got the job done without any hassle, FAST!
That was eye opening for me.
1/ What didn't seem to get mentioned was the speed to market. It's far worse to build the right thing no one wants, than to build the crappy thing that some people want a lot. As a result, it makes sense for people to leverage electron--but it has consequences for users down the line.
2/ Because we deal with orders of magnitude with software, it's not actually a good ROI to deal with things that are under 1x improvement on a human scale. So what made sense to optimize when computers were 300MHz doesn't make sense at all when computers are 1GHz, given a limited time and budget.
3/ Anecdotally (and others can nix or verify), what I hear from ex-Googlers is that no one gets credit for maintaining the existing software or trying to make it faster. The only way you get promoted is if you created a new project. So that's what people end up doing, and you get 4 or 5 versions of the same project that do the same thing, all not very well.
I agree that the suckage is a problem. But I think it's the structure of incentives in the environment that software is written that also needs to be addressed, not just the technical deficiencies of how we practice writing software, like how to maintain state.
It's interesting Chris Granger submitted this. I can see that the gears have been turning for him on this topic again.
There's ways to develop working software, but not if it's all locked behind closed OSes and other bullshit.
In my experience, a lot of performance problems boil down to really stupid problems, like simple code using the wrong data structure out of convenience (e.g. linked lists instead of arrays for lots of randomly-accessed data), or structured in a bad way (e.g. allocating a lot of small pieces of memory all the time). Often times, there are cheap performance wins to be had if you occasionally run the product through a profiler and spend couple of hours fixing the most pressing issue that shows up. Couple of hours isn't much; there's enough slack in the development process to find those hours every month or two, without slowing down your regular work.
But I have a different experience (probably because we work in different areas):
Most the performance problems of the products I ever worked were purely systemic.
They boiled down to technologies and architectures having been chosen for "organizational" rather than technological reasons.
And "organizational" is in quotes because sometimes it was just blackmail: I worked with two developers who quit in protest after the prototype they wrote in Scala was deemed not good enough and dropped for... being too slow, ironically.
In the end the backend was pure and beautiful, but the the frontend devs had to perform joins in the the client and make 21 API calls in a 20-item list and then everything goes to hell.
I find it really interesting that no one in the future of programming/coding community has been able to really articulate or demonstrate what an "ideal" version of software engineering would be like. What would the perfect project look like both socially and technically? What would I gain and what would I give up to have that? Can you demonstrate it beyond the handpicked examples you'll start with? We definitely didn't get there.
It's much harder to create a clear narrative around the social aspects of engineering, but it's not impossible - we weren't talking about agile 20 years ago. The question is can we come up with a complete system that resonates enough with people to actually push behavior change through? Solving that is very different than building the next great language or framework. It requires starting a movement and capturing a belief that the community has in some actionable form.
I've been thinking a lot about all of this since we closed down Eve. I've also been working on a few things. :)
One thing I found was that surprisingly the C# code I write outperforms the C++ code I used to write at equal development times.
I was good at C++, but the language has so many footguns and in general is so slow to develop in that I would stick to "simple" and straightforward solutions. I avoided multi-threading like the plague because it was just so hard to get right.
Meanwhile in C# it's just so easy to sprinkle a little bit of multithreading into almost any application (even command-line tools) that I do it "just because". Even if the single-threaded performance is not-so-great, the end result is often much better.
Similarly, it's easy to apply complex algorithms or switch between a few variants until something works well. In C++ or even Rust, the strict ownership semantics makes some algorithm changes require wholesale changes to the rest of the program, making this kind of experimentation a no-go.
The thing that blows my mind is the "modern" approach to programming that seems to be mostly young people pretending that Java or C# just don't exist.
Have you seen what JavaScript and Python people call "easy?" I saw a page describing a REST API based on JSON where they basically had thousands of functions with no documentation, no schema, and no typed return values. It was all "Just look at what the website JS does and reverse engineer it! It's so easy!"
I was flabbergasted. In Visual Studio I can literally just paste a WSDL URL into a form and it'll auto-generate a 100K-line client with async methods and strongly-typed parameters and return values in like... a second. Ditto for Linq-2-SQL or similar frameworks.
Because it's never problems really, it's perceived that way though.
A certain challange needs a specific set of personalities to solve it. That's the real puzzle.
Great engineers will never be able to solve things properly unlessed given the chance by those who control the surroundings.
We seek how we should develop, what method should be used, is it agile or is it lean? But maybe the problem starts earlier and focusing on exactly what methods and tools to use we miss out on the most simplest solution even beginners can see.
For example I am an architect, I tend to not touch the economics in a project. It's better fitted for other persons.
While not having read much about team based development I do want to be directed to well read literature about it. Maybe it's better called social programming, just another label of what we really do.
The one I miss the most at work is my wife. She clearly is the best reverse of me and makes me perform 1000x better. I find that very funny since she does not care about IT at all.
And by "social and inventive structures", I'm assuming you're talking about change on the order of how open source software or agile development changed how we develop software?
While agile did address how to do software in an environment for changing requirements and limited time, we don't currently have anything that addresses an attention to speed of software, building solid foundations, and incentives to maintain software.
What would a complete system encompass that's currently missing in your mind?
Writing performant, clean, pure software is super appealing as a developer, so why don't I do something about the bloated software I write? I think a big part of it is it's hard to see the direct benefit from the very large amount of effort I'll have to put in.
Sure I can write that one thing from that one library that I use myself instead of pulling in the whole library. I might be faster, I might end up with a smaller binary, it might be more deterministic because I know exactly what it's doing. But it'll take a long time, might have a lot of bugs and forget about maintaining it. Then end of the day, do the people that use my software care that I put in the effort to do this? They probably won't even notice.
To me, where libraries get a bit more questionable is when they exist in the realm of pure abstraction, or when they try to own the flow of control or provide the structure around which your program should hang. For instance, with something like Ruby on Rails, it sometimes feels like you are trying to undo what the framework has assumed you need so that you can get the functionality you want. A good library should be something you build on top of, not something you carve your implementation out of.
The exception would be games and embedded software, but even there, there's certain degrees of lazyness. For instance, games are very cpu/memory/gpu efficient, but they're almost always ridiculous when it comes to disk space usage. There's no reason that your average AAA game needs to take up 50GB other than that things which could address that aren't worth the fuss. (I'm thinking common demo-scene tricks like procedural textures/data/everything, aggressive compression schemes, reusing assets, etc.)
For that matter, one could for example take the Windows NT 4.0 source code, add in drivers for the necessary hardware, fix boot code, linking, etc to be compatible with late model computers, spruce up the UI with better font rendering, antialiasing, 24-bit color wallpaper, OpenGL rendering even--and in the end, you'd have something just as functional as Win 7/10 but at 1/4 of the bloat.
This sort of thing would be technically very easy to do. It's much easier than the status quo of continually reinventing the wheel. So why, oh why, is there this overpowering desire to continually throw out good code and replace it with heavier, more bloated junk, which doesn't really offer any real increase in functionality?
You've almost described ReactOS!
One second times five million users is almost two months. One second times five thousand users is less than 90 minutes.
There's way more to life than just dollars and cents. Indeed, this really seems to be the larger point here. The bean counters and ladder climbers are now in control of everything, with disastrous consequences.
Besides, you could essentially do what you’re describing by piecing together a Linux desktop from a lean distro. It’s not hard to find efficient software, it’s just not really mainstream
Not everything needs to be super efficient. Most things are tuned for production cost and time. Efficient code isn't going anywhere. Relax guys.
We shouldn't have to put up with all this shit. That's the point.
Are you willing to pay $20 for a phone app, instead of fishing for the free version? Are you willing to pay for websites to get rid of ads? Are you willing to pay $300 for a new OS?
Alternatively, are you willing to ditch capitalism for a system that prioritizes the commons?
It's us. We are creating the incentives for a world that produces this. Unless we change that world, this is what we'll get.
I don't really know what this is supposed to mean, it sounds rather poetic and abstract, but I'm quite certain few if any people ever asked for a world where it takes 10+ seconds to open a simple email.
Probably because a browser like FF has the goal to load and display arbitrary dynamic content in realtime like a reddit infinite scroll with various 4k videos and ad bullshit, whereas the game has the goal to render a known, tested number of pre-downloaded assets in realtime.
Also on shitty pages the goal is different-- load a bunch of arbitrary adware programs and content that the user doesn't want, and only after that display the thing they want to read.
Also, you can click a link somewhere in your scrolling that opens a new, shitty page where you repeat the same crazy number of net connections, parsing ad bullshit, and incidentally rendering the text that the user wants to read.
If you want to compare fairly, imaging a game character entering a cave and immediately switching to a different character like Spiderman and inheriting all the physics and stuff from that newly loaded game. At that point the bulk of your gameplay is going to be loading new assets and you're back to the same responsiveness problems of the shitty web.
Edit: clarification
Say hello to shaders.
It's not the fault of Firefox that Reddit's new UI is pathetically slow. It's the Reddit's implementation of their UI itself which is total garbage.
And given that people do write fast, complex, real-time games in JavaScript for the browser, gamedev absolutely becomes a valid reference point for the possible performance of any individual page.
Why should Firefox or any other dynamic software have the ability to be slow for what it archives? If compilers should be fast, Web engines should be equally as fast. The Web should have never been designed such that a slow website (relative to the task) could be achieved. In the same way that you can only express memory safe code in rust and type safe code in haskell why not being only able to express "fast for what is interactive"?
That's already the case, your orders of magnitude are just off. Long-running AJAX or page loads are timed out at a pretty consistent point across browsers. Half-open/closed TCP connections are timed out at a pretty consistent point across operating systems. Busy-looping JS gets you a "page is not responding" block in a similar amount of time; same for nonresponsive native applications on many operating systems.
Their definition of "slow" or "stuck" just tends to be "tens of seconds or minutes", not the threshold of perceived responsiveness you want in a website.
Also, your parenthetical is a pretty tall-née-impossible order:
> a slow website (relative to the task)
How could the "task" be classified? Do you mean "task" as in "clicking a button and having a DOM update"? Or as in "this is a TODO application so it should have responsiveness threshold X"?
As a web developer, sending an 8 KB JSON response is no problem. That's nice and light. In a networked action game, that's absurd. First, (hypothetical network programmer talking here) we're going to use UDP and write our own network layer on top of it to provide reliability and ordering for packets when we need it. We're going to define a compact binary format. Your character's position takes 96 bits in memory (float x, y, z); we'll start by reducing that to 18 bits per component, and we'll drop the z if you haven't jumped. Then we'll delta compress them vs the previous frame. Etc.
Really, what's happening is things are getting optimized as much as they need to be. If your game is running at 10 fps, it's going to get optimized. When it's hitting 60+ fps on all target platforms, developers stop optimizing, even if it could potentially be faster. Same for Reddit; it's fast enough for most users.
It depends on what that 8 KB is doing. If that 8 KB is a chat message, that's way too big. On the other hand, I've never seen an 8 KB game patch.
When I talk about sending 8 KB in a "networked action game", I'm referring to the update packets sent to and from clients in something like Fortnite or Counter-Strike, not a game patch. I'm not trying to make a competition for who uses the least bandwidth (which a 60 GB game would lose just on the initial download). I'm trying to illustrate that games don't run faster than some website because it's inherently easier to make games run fast, but rather that developers, by necessity, optimize games until they run fast (or in this example, until they reduce network lag enough).
I'm not sure why a chat app would tack on something like 7.5 KB of overhead on a chat message, but I wouldn't be surprised if there's a chat app out there that does so. Users won't notice the extra couple milliseconds (especially so because they don't know exactly when the other person hit send). A 3 character Discord message is close to 1 KB including the headers. The same message over UDP in a game might be under 20 bytes, including the UDP header (games could also use TCP for chat - text chat isn't going to strain anything). So I'd say the overhead of a Discord message is still an order of magnitude or two bigger than it could be. Which is perfectly fine; we can afford 1 KB of overhead on a modern connection. It's optimized as much as it needs to be.
I think the analogy here is backwards. The better question is "how much would you prioritize a car that used only 0.05 liters per 100km over one that used 0.5? What about one that used only 0.005L?". I'd say that at that point, other factors like comfort, performance, base price, etc. become (relatively) much more important.
If basic computer operations like loading a webpage took minutes rather than seconds, I think there would be more general interest in improving performance. For now though, most users are happy-enough with the performance of most software, and other factors like aesthetics, ease-of-use, etc. are the main differentiators (admittedly feature bloat, ads, tracking, etc. are also a problem, but I think they're mostly orthogonal to under-the-hood performance).
These days, I think most users will lose more time and be more frustrated by poor UI design, accidental inputs, etc. than any performance characteristics of the software they use. Hence the complexity/performance overhead of using technologies that allow software to be easily iterated and expanded are justified, to my mind (though we should be mindful of technology that claims to improve our agility but really only adds complexity).
At my current place of employment we have plenty of average requests hitting 5-10 seconds and longer, you've got N+1 queries against the network, rather than the DB. As long as it's within 15 or 30 seconds nobody cares, they probably blame their 4G signal for it (especially in the UK where our mobile infrastructure is notoriously spotty, and entirely absent even within the middle of London). But since I work on those systems I'm upset and disappointed that I'm working on APIs that can take tens of seconds to respond.
The analogy is also not great because MPG is an established metric for fuel efficiency in cars. The higher the MPG the better.
I never liked this view. I can't think of a single legitimate use case that couldn't be solved better than by hiding your true capabilities, and thus wasting people's time.
> they probably blame their 4G signal for it
Sad thing is, enough companies thinking like this and the incentive to improve on 4G itself evaporates, because "almost nothing can work fast enough to make use of these optimizations anyway".
Consider a loading spinner with a line of copy that explains what's happening. Say it's for an action that can take anywhere from 20 milliseconds to several seconds, based on a combination of factors that are hard to predict beforehand. At the low end, showing the spinner will result in it flashing on the screen jarringly for just a frame. To the user it will appear as some kind of visual glitch since they won't have time to even make out what it is, much less read the copy.
In situations like this, it's often a good idea to introduce an artificial delay up to a floor that gives the user time to register what's happening and read the copy.
It's a real case, actually. I don't remember a name, but I've encountered this situation in the past, and that brief flash of a "in progress" marker was what I used to determine whether me clicking a "retry" button actually did something, or whether the input was just ignored. It's one of those unexpected benefits of predictability of UI coding; the less special cases there are, the better.
e.g. back in the 90s with Resi 1, the loading screen was hidden by a slow and tense animation of a door opening. It totally fit the atmosphere.
Plenty of games add an elevator or a scripted vehicle ride, or some ridiculous door locking mechanism that serves the same purpose without breaking immersion, especially as those faux-loading screens can be dynamic.
It's pretty much the exact same technique used in cinema when a director wants to stitch multiple takes into a single shot (e.g. that episode in True Detective; that other one in Mr Robot; all of Birdman).
I see this argument coming up a lot, but this can be solved by better UX. Making things slow on purpose is just designers/developers being lazy.
Btw users feeling uneasy when something is "too fast" is an indictment of everything else being too damn slow. :D
IMO it can be attributed more to bad UI than optimizations.
I'll prioritize the 0.005L per 100km car for sure. That means the car can be driven for all its expected lifetime (500k km) in a single tank of gas, filled up at the time of purchase! That means there is a huge opportunity to further optimize for many things in the system:
- The car no longer needs to have a hole on the side for filling up. A lot of pipes can be removed. Gas tank can be moved to a safer/closer location where it is used.
- The dashboard doesn't need a dedicated slot for showing the fuel gauge, more wirings and mechanical parts removed.
- No needs for huge exhaust and cooling systems, since the wasted energy is significantly reduce. No more pump, less vehicle weights...
Of course, that 0.005L car won't come earlier than a good electric car. However, if it's there, I'd totally prioritize it higher than other things you listed. I think people tend to underestimate how small efficiency improvements add up and enable exponential values to the system as a whole.
I've worked with a Preact SPA where the time to initial render was faster than the HAML templates it replaced.
But then, again, that was an outlier. If your target is speed, traditional SSR or static pages are the best bet, anyway.
I'm struggling to think of recent (or not-so-recent) software improvements that have had a similar impact though. It seems like many of the "big" algorithms and optimization techniques that underpin modern applications have been around for a long time, and there aren't a lot of solutions that are "just about" ready to make the jump from supercomputers to servers, servers to desktops, or desktops to mobile. I guess machine learning is a probably contender in this space, but I imagine that's still an active area of optimization and probably not what the author of the article had in mind. I'd love if someone could provide an example of recent consumer software that is only possible due to careful software optimization.
That would be the meaningful software improvements I can think of. Curiously, the key enabler here seems to be performance - we had the capability to write web apps for a while, but JS was too slow to be useful.
--
[0] - They may or may not have been developed earlier, but I haven't seen them used in practice before the modern web.
In this thought experiment we are talking about a 2 orders magnitude improvement - hardly a small nudge!
It's a nice idea but it wouldn't work. The gasoline would go bad before you could use it all.
Plug-in hybrids already have this problem. Their fuel management systems try to keep the average age of the fuel in the tank under 1 year. The Chevy Volt has a fuel maintenance mode that runs every 6 weeks:
https://www.nytimes.com/2014/05/11/automobiles/owners-who-ar...
https://www.autoblog.com/2011/03/18/chevy-volts-sealed-gas-t...
Instead of having a "lifetime tank", a car that uses 0.005L per 100km would be better off with a tiny tank. And then instead of buying fuel at a fuel station you'd buy it in a bottle at the supermarket along with your orange juice.
If you had pure gasoline, you could store it for years (and in past, countries and armies did exactly that for their reserves).
Secondly we talked about 0.005l cars, not 0.05l, so it would be a few hundreds times more efficient than train transportation.
I’m willing to bet that a significant percentage of my accidental inputs are due to UI latency.
I want all interactions with all of my computing devices to occur in as close to 0ms as possible. 0ms is great; 20ms is good; 200ms is bad; 500ms is absolutely inexcusable unless you're doing significant computation. I find it astonishing how many things will run in the 200-500ms range for utterly trivial operations such as just navigating between UI elements. And no, animation is not an acceptable illusion to hide slowness.
I am with the OP. "Good enough" is a bane on our discipline.
0ms is great, 4ms is very good, 16ms is minimally acceptable, 20ms needs improvement (you're skipping frames), 200ms is bad (it's visible!), 500ms is ridiculous and should have been showing a progress bar or something.
Responding to input doesn't necessarily mean being done with processing, it just means showing a response.
The usage story requires you to switch to turn-by-turn, and there’s no way to have bird eye map following your location along route (unless you just choose some zoom level and manually recenter every so often.)
It’s awful, distracting and frankly a waste of time... just to show a bit of animation every time I accidentally fail to register a drag...
Damn Ui
It does fill some other requirements that a regular car doesn't.
That's not really true. Slack could be just as pretty and a fraction of the weight, if they hadn't used Electron.
One, phone OSes are being designed for single-tasked use. Outside of alarms and notifications in the background (which tend to be routed through a common service), the user can see just one app at a time, and mobile OSes actively restrict background activity of other apps. So every application can get away with the assumption that it's the sole owner of the phone's resources.
Two, given the above, the most noticeable problem is now power usage. As Moore's law has all but evaporated for single-threaded performance, hardware is now being upgraded for multicore and (important here) power performance. So apps can get away with poor engineering, because every new generation of smartphones has a more power-efficient CPU, so the lifetime on single charge doesn't degrade.
A UI where each interaction takes several seconds is poor UI design. I do lose most of my time and patience to poor UI design, including needless "improvements" every few iterations that break my workflow and have me relearn the UI.
I find the general state of interaction with the software I use on a daily basis to be piss poor, and over the last 20 or so years I have at best seen zero improvement on average, though if I was less charitable I'd say it has only gone downhill. Applications around the turn of the century were generally responsive, as far as I can remember.
I was forced to use a monitor at 30 fps for a few days due to a bad display setup. It made me realize how important 60 fps is. Even worse, try using an OS running in a VM for an extended period of time...
There are plenty of things that are 'good enough', but once users get used to something better they will never go back (if they have the choice, at least).
Optimizing for microseconds when bad UI steals seconds is being penny-wise and pound foolish. Business might not understand tech but they do generally understand how it ends up on the balance sheet.
Note also the potential deadlock here. Optimizing core calculations at μs level is bad because UI is slow, but optimizing UI to have μs responsiveness is bad, because core calculations are slow. Or the database is slow. This way, every part of the program can use every other part of the program as a justification to not do the necessary work. Reverse tragedy of the commons perhaps?
- Opening multiple tabs in a browser will kill your battery, and it's not the fault of a single page, but of all of them. Developers tend to blame the end user for opening too many tabs.
- Running a single Electron app is fast enough in a newer machine but if you need multiple instances or multiple apps you're fucked.
- Some of my teammates can't use their laptops without the charger because they have to run 20+ docker containers just to have our main website load. The machines are also noisy because the fan is always on.
- Having complex build pipelines that take minutes or hours to run is something that slows dow developers, which are expensive. It's not the fault of a single software (except maybe of the chosen programming language), but of multiple inefficient libraries and packages.
I do that for most of my hobbyist Linux dev work. It's fine. It can do 4k and everything. It's surely not optimal but it's better than managing dual boot.
I actually do this for development and it works really well.
Ubuntu Linux VM in VMware Fusion on a Macbook Pro with MacOS.
Power consumption was found to be better than running Linux natively. (I'm guessing something about switching between the two GPUs, but who knows.)
GPU acceleration works fine; the Linux desktop animations, window fading and movement animations etc are just as I'd expect.
Performance seems to be fine generally, and I do care about performance.
(But I don't measure graphics performance, perhaps that's not as good as native. And when doing I/O intensive work, that's on servers.)
Being able to do a four-finger swipe on the trackpad to switch between MacOS desktops and Linux desktops (full screen) is really nice. It feels as if the two OSes are running side by side, rather than one inside another.
I've been doing Linux-in-a-VM for about 6 years, and wouldn't switch back to native on my laptop if I had a choice. The side-by-side illusion is too good.
Before that I ran various Linux desktops (or Linux consoles :-) for about 20 years natively on all my development machines and all my personal laptops, so it's not like don't know what that's like. In general, I notice more graphics driver bugs in the native version...
(The one thing that stands out as buggy is VMware's host-to-guest file sharing is extremely buggy, to the point of corrupting files, even crashing Git. MacOS's own SMB client is also atrocious in numerous ways, to the point of even deleting random files, but does it less often so you don't notice until later what's gone. I've had to work hard to find good workarounds to have reliable files! I mention this as a warning to anyone thinking of trying the same setup.)
But perhaps the answer is VMware Fusion instead then.
Having your VM stored inside a file on a slow filesystem is bad. Having a separate lvm volume (on linux)/zvols (with zfs)/partition/disk is much more performant.
That's about human-scale experience, rather than measured latency. It won't be as fast as native, but it seems adequate for my use, even when grepping thousands of files, unpacking archives, etc, and I haven't noticed any significant stalling or pauses. It's encrypted too (by MacOS).
(That's in contrast to host-guest file access over the virtual network, which definitely has performance issues. But ext4 on the VM disk seems to work well.)
The VM is my main daily work "machine", and I'm a heavy user, so I'd notice if I/O latency was affecting use.
I'm sure it helps that the Mac has a fast SSD though.
(In contrast, on servers I use LVM a lot, in conjunction with MD-RAID and LUKS encryption.)
I've only given Linux 6GB RAM at the moment, and it's working out fine. Currently running Ubuntu 19.10.
I picked VMware Fusion originally because it was reported to have good-ish support for GPU emulation that was compatible with Linux desktops at the time. Without it, graphics can be a bit clunky. With it, it feels smooth enough for me, as a desktop.
My browser is Firefox on the Mac side, but dev web servers all on the Linux side.
The VM networking is fine, but I use a separate "private" network (for dev networking) from the "NAT" network (outgoing connections from Linux to internet), so Wifi IP address changes in the latter don't disrupt active connections of the former.
My editor is Emacs GUI on the Mac side (so it integrates with the native Mac GUI - Cmd-CV cut and paste etc, better scrolling), although I can call up Emacs sessions from Linux easily, and for TypeScript, dev language servers etc., Emacs is able to run them remotely as appropriate.
Smoothness over SSH from iTerm is a different thing from graphical desktop smoothness.
When doing graphics work (e.g. Inkscape/GIMP/ImageMagick), or remote access to Windows servers using Remmina for VNC/RDP, I use the Linux desktop.
But mostly I do dev work in Linux over SSH from iTerm. I don't think I've ever noticed any smoothness issues with that, except when VMware networking crashes due to SMB/NFS loops that I shouldn't let happen :-)
>And build times? Nobody thinks compiler that works minutes or even hours is a problem. What happened to “programmer’s time is more important”? Almost all compilers, pre- and post-processors add significant, sometimes disastrous time tax to your build without providing proportionally substantial benefits.
Anecdotally, in my career I've never had to compile something myself that took longer than a few minutes (but maybe if you work on the Linux kernel or some other big project, you have; or maybe I've just been lucky to mainly use toolchains that avoid the pitfalls here). I would definitely consider it a problem if my compiler runs regularly took O(10mins), and would probably consider looking for optimizations or alternatives at that point. I've also benefited immensely from a lot of the analysis tools that are built into the toolchains that I use, and I have no doubt that most or all of them have saved me more pain than they've caused me.
>You’ve probably heard this mantra: “Programmer time is more expensive than computer time.” What it means basically is that we’re wasting computers at an unprecedented scale. Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time.
The point is that we are wasting all the resources at every scale. We are supposedly burning computer cycles because developer time is more important. Yet we are also burning developer time with compiling, or testing for interpreted languages, at a rate that is starting to approach the batch processing days.
I use webpages for most of the social networking platforms such as Facebook. I am left handed and scroll with my left thumb (left half of the screen). I have accidentally ‘liked’ people’s posts, sent accidental friend requests only because of this reason.
Guessing along with language selection, it might be helpful to have a selection of hand preference for mobile browsing.
Moreover, the same cost-equation that produces software that is much less efficient than it could be produces software that might be usable for it's purpose (barely) but is much more ugly, confusing, and buggy than it needs to be.
That equation is add the needed features, sell the software first, get lock in, milk it 'till it dies and move on. That's equation is locally cost-efficient. Locally, that wins and that produces the world we see every day.
Maybe, the lack of craftsmanship, the lack of doing one's activity well, is simply inevitable. Or maybe the race to the bottom is going to kill us - see the Boeing 737 Max as perhaps food for thought (not that software as such was to blame there but the quality issue was there).
I think for webpages it is the opposite: non-orthogonal in most cases.
If you disable your JS/Ad/...-blocker, and go to pages like Reddit, it is definitely slower and the CPU spikes. Even with a blocker, the page still does a thousand things in the first-party scripts (like tracking mouse movements and such) that slow everything down a lot.
Wait, are you implying they don't ? What world do you live in, and how do I join?
or... something.
The car analogy does remind me of one I read a while ago, comparing cars and their cost and performance with CPUs.
I piled standout quotes below.
I think a big takeaway from the intersection of Bret Victor, Alan Kay, Jim Hollan and the ink&switch folks and your work is that the right dynamic interface can be the "place we live in" on the computer.
Victor shows a history of interactive direct manipulation interfaces, live environments where explorations of models or the creation of art go hand in hand with everything else related to that task: data input, explicit (programmatic) requirements and the visual output.
Hollan and ink&switch show the environment (ZUIs, canvas) can contain everything for doing work, the code alongside any manipulation of the viewport that can be conceived. Tools infinitely more advanced than Microsoft OneNote and designed 40 years ago.
From what I know about your work, I see another take on the environment I want to live in on the computer. I dont understand why I would want to lose power by stepping away from my language/interpreter/compiler/repl into a GUI or some portal when I can bring whatever it is which is nice about GUIs or portals into my dynamic computing environment. I very much want a personal DSL or set of DSLs for what I do on the computer, and I want to be able to hook into anything ala middle mouse button in plan9.
The superior alternative to walled gardens and this absurd world of bloat and 'feature loss' (for lack of a better term for software engineering's enthusiastic rejection of history) seems to be known, and facets of it advocated by you and these others. It seems clear that "using the computer" needs to return to "programming the computer" and that to achieve that we need to fundamentally change "programming the computer" to be a more communicative activity, to foster a better relationship between the computer and the user.
Where is this work being done now? VPRI shut down 2 years ago, Dynamicland seems to be on hiatus? I am inspired most these days by indie developers who write their own tools and build wild looking knowledge engines or what they sometimes call "trackers."[1] And of course the histories and papers put forward by the above and their predecessors. And I play with my own, building an environment where I can write, draw, code, execute and interact with it all. I see no existing product which approaches what I want.
> Everyone is busy building stuff for right now, today, rarely for tomorrow.
> Even when efficient solutions have been known for ages, we still struggle with the same problems: package management, build systems, compilers, language design, IDEs.
> You need to occasionally throw stuff away and replace it with better stuff.
> Business won’t care Neither will users. They are only learned to expect what we can provide.
> There’s no competition either. Everybody is building the same slow, bloated, unreliable products.
> The only thing required is not building on top of a huge pile of crap that modern toolchain is.
> I want something to believe in, a worthy end goal, a future better than what we have today, and I want a community of engineers who share that vision.
[1]: https://webring.xxiivv.com
I agree with the post, though as others have pointed out, it doesn't really dive into the fact that this problem is systemic and would require a shift in incentive structure.
I think the last quote you have is one of the most important missing pieces for making a meaningful change in this space. A lot of people want something better, but right now, as a community, I don't think we really know what that is. What is the complete story for an ideal version of software development? And by that I don't mean idealized examples, I mean the ideal version of the real process we have to go through. What does perfect look like in the world of changing requirements, shifting teams, legacy systems, crappy APIs, and insufficient budgets? If we could show that - not the simple examples we had for Eve, but something that addresses the raw reality of engineering - I think it would just be a matter of beating the drum.
But on the other hand, a lot of web content does need to be faster. Gmail has somehow gotten so much slower to load over time. And every time I visit a newspaper/magazine website I am aghast at how bloated they are. Does that mean nodejs is inherently bad, no, but it does mean people should try to optimize noticeably terrible performance that actually degrades UX.
Much of it on the web also has to do with how much browsers can do. The number of CSS properties that can be applied, the ways different elements' sizes can automatically influence the layout of other elements, etc. These traits are what make the web such a powerful and attractive platform for user interfaces, but the complexity of the platform is definitely becoming a real issue that deserves attention.
A couple of points:
- NodeJS is server-only and usually has nothing to do with perceived performance of web apps
- The biggest offender of web performance is ads. They dump piles and piles of crappy JavaScript from dozens of different sources that all include their own copies of common libraries and have no incentive not to slow down the page.
- Beyond ads, the bottleneck is usually not even JavaScript, but layout (as in the paragraph above). Web layout is incredibly flexible and incredibly complex. Computing and rendering it all is slow, but it does serve a purpose. Not that it couldn't be improved.
Bad ads are a tragedy of the commons and I don't know what can be done about them unless Google or Facebook decided to throw their weight around to force them to be better.
I do wonder if a new web standard could be developed for using some constrained subset of the layout vocabulary, that would be cheaper and more straightforward to compute. The current version has to remain for backwards compatibility reasons, but it's trying to serve a bunch of different types of cases at once, and therefore doesn't do a great job at any one of them.
And as it's surrounded by numerous such "slightly inefficient but efficient from some other perspective" interactions overall efficiency dies a death of a thousand cuts.
When you actively start paying attention to it and comparing it to little examples of what could be you start noticing how utterly garbage everything is. Even at the base level in things that have a massive userbase and that are used not just once in a while but constantly it's disgusting trash. I look around at the company i work at. They're all using it and probably not noticing at all but the fucking file explorer in Windows is slow as fuck as is countless other elements and interactions of it. The companies website so simple in it's content and functionality and is made by a webdev agency but is a bloated mess that takes a while to get to....a logo that shows whilst it continues to load. The software my coworker wrote is small scale and he said i was wasting a lot of my time making some small action faster not recognising that it's been used many thousand times a day every day for more than 10 years now.
There's way too little moral panic
> iOS 11 dropped support for 32-bit apps. That means if the developer isn’t around at the time of the iOS 11 release or isn’t willing to go back and update a once-perfectly-fine app, chances are you won’t be seeing their app ever again.
but then he also says:
> To have a healthy ecosystem you need to go back and revisit. You need to occasionally throw stuff away and replace it with better stuff.
So which is it? If you want to replace stuff with something better, that means the old stuff won't work anymore... or, it will work by placing a translation/emulation layer around it, which he describes as:
> We put virtual machines inside Linux, and then we put Docker inside virtual machines, simply because nobody was able to clean up the mess that most programs, languages and their environment produce. We cover shit with blankets just not to deal with it.
Seems like he wants it both ways.
Otherwise it's a tradeoff if you add constraints like cost, effort, time to market, and so on...
And speed isn't the only metric that matters; having both the 32-bit and 64-bit versions of DLLs uses a non-trivial (to some people) amount of disk space, bandwidth, complexity, etc.
If Android, Mac OS, etc were super slimmed down systems in comparison to Windows, I would understand the argument much better. Instead, it feels like we're in the worst of both worlds.
Yeah, didn't say it's impossible. I said it's a tradeoff.
Windows does it and pays for it with slower releases, more engineers, bugs, strange interaction between old and new, several layers of UI and API code for devs to decode and for users to be confused with, less ability to move to new paradigms (why would devs bother if the old work), 2 versions of libs loaded (32/64 bit), and several other ways besides...
E.g. I've stopped using Windows for a decade or so, but I read of the 3 (4?) settings panels it has, the modern, the Vista style, the XP style, and so on, with some options in one, the others in the other (if you click some "advanced" menu, etc).
I don't quite know what's going on inside Apple, but it doesn't feel like they're choosing which features to remove in a particularly thoughtful way.
---
Twenty years ago, Apple's flagship platform was called Mac OS (Mac OS ≠ macOS), and it sucked beyond repair. So Apple shifted to a completely different platform, which they dubbed Mac OS X. A slow and clunky virtualization layer was added for running "classic" Mac OS software, but it was built to be temporary, not a normal means of operation.
For anyone invested in the Mac OS platform at the time, this must have really sucked. But what's important is that Apple made the transition once! They realized that a clean break was essential, and they did it, and we've been on OS X ever since. There's a 16-year-old OS X app called Audio Slicer which I still use regularly in High Sierra. It would break if I updated to Catalina, but, therein lies my problem with today's Apple.
If you really need to make a clean break, fine, go ahead! It will be painful, but we'd best get it over with.
But that shouldn't happen more than once every couple decades, and even less as we get collectively more experienced at writing software.
Even with our priorities in order, there will still be contentious, hard choices (to deprecate so-and-so or not; to sacrifice a capability for consistency of interface or not), but the author's point is that our priorities are not in order in the first place, so the decisions we make end up being arbitrary at best, and harmful/driven by bad motivations at worst.
[0] https://tonsky.me/blog/good-times-weak-men/
[1] https://www.youtube.com/watch?v=pW-SOdj4Kkk
In this day of "Agile" development, as long as something's working during UAT, that's all that's needed for sales and consumers.
Webdev, IME, is an example where the ecosystem has facilitated bloated websites. I've worked with developers who throw any library they can just for basic things because they don't have a need to try to optimise. The meme of using jQuery for everything when it came out has just been replaced by other frameworks. I find it often depends on developers who really want to work on something and take pride in it vs those who just need something on their CV or got hired by following a few tutorials on the web but not understanding what they wrote (which, to me, signifies a hiring problem in the company). During code reviews, I encourage leads to keep calling up hacky code to a point where the developer will just start writing it properly the first time round. As developers, I feel we should be aware of not creating selfish software which hogs memory from other software or requires huge data downloads for mobile users (whenever doable). Possibly a naive ideal but if it's a byproduct of developing fast software for my end users, I think that's a win-win.
The guy is most definitely at least a genius.
They are great examples for his overall point though. It probably would've been better just to leave out the genius bit and talk about them as folks proving it can be done.
Think about all the gas that is saved because people don’t have to drive to the library, all the plane trips saved by video conferencing, all the photo film, all the sheets of paper in file cabinets, all the letters being sent as emails, all the mail order catalogues, ... you get the idea.
Does anybody know of a comprehensive study on this?
You're right that computers have saved huge amounts of energy compared with the things you mention. My point here was that even more could be saved with a bit of thought about efficiency in programming.
I'm quite disenchanted with software myself. It takes way too long to open any program, for this JIRA ticket to properly display.
One thing that has improved was boot times, I seem to remember that Windows 7 was quite a bit faster than XP. Maybe someone in upper management wanted it to be as fast as MacOS? So speed IS possible, if it is prioritized.
I seem to remember boot times being a frequent topic of discussion the early 2000s, because people turned off their computers.
In a way, this is a great little microcosm of the problem. Just fix habits instead of fixing the software.
In my - perhaps limited - work experience, there's enough slack in the process of software development that I don't buy the "time to market" argument all that much.
(Do individuals really bother mining bitcoin these days anyway?)
if you haven’t seen it already, you’d probably be interested in the below talk by chuck moore, inventor of forth.
https://www.infoq.com/presentations/power-144-chip/
Fascinating talk. Thanks for the link.
But the vast majority of software is one-off stuff. It makes no sense to optimize it for performance instead of features, development time, correctness, ease of use, etc.
Is it now? I can't think of any example, except a few tools we use internally in a project. Everything else I use, or I see anyone else using, has an userbase of many thousands to millions, and a lot of that is used in context of work - which means a good chunk of the userbase is sitting in front of that software day in, day out.
I wouldn't be at all surprised if it's the majority. For every big-ticket software offering there's going to at least be things the user never interacts with, like a build system and a test suite and a bug tracking system and whatever else. And there's just so much software everywhere, most of which we never see. Every small business has its little web site. Who knows how much software there is powering this or that device or process at random factories, laboratories, and offices.
The Debian popularity contest looks like it has a big long tail of relatively unpopular packages [1]. It looks like the app store has 2 million apps, only 2857 (.14%) have more than a million dollars of annual revenue. These are of course incomplete and flawed and do not really directly address the question. I don't really know how to research this in a thorough way.
[1] https://popcon.debian.org/by_inst [2] https://expandedramblings.com/index.php/itunes-app-store-sta...
The things I'm doing on my phone today are not fundamentally different than what I was doing ten years ago. And yet, I had to buy a new phone.
1. The environmental cost of ineffective software is negligible, when compared to Bitcoin mining or other forms of hardware planned obsolescence.
2. By using more efficient software, surely, you can save a lot of CPU cycles, and it can improve the energy efficiency of some specific workloads under some particular scenarios. However, on a general-purpose PC, the desire for performance is unlimited, the CPU cycles saved in one way will only be consumed in other ways, and in the end, the total CPU cycles used remain a constant.
Running programs on a PC is like buying things, when you have a fixed budget but everything is cheaper, often people will just buy more. For example, I only start closing webpages when my browser becomes unacceptably slow, but if you make every webpage use 50% less system resource, I'll simply open 2x more webpages simultaneously. LED lightning is another example, while I think the overall effect is a reduction of energy uses, but in some cases, I think it actually makes people to install more lightnings, such as those outdoor billboards.
This is called the Jevons paradox [0].
For PCs, certainly, as I previously stated, in specific workloads under some particular scenarios, I totally agree that there are cases that energy use can be reduced (e.g. faster system update), but I don't think it helps much in the grand scheme of things.
[0] https://en.wikipedia.org/wiki/Jevons_paradox
In fact, I'd go so far as to suggest there are more good programmers about today then there were in the past. Though having said that, percentage-wise I'd say the profession is definitely being deskilled (good thing if you're a manager, bad thing if you're a good programmer).
FTFY
> Modern text editors have higher latency than 42-year-old Emacs. Text editors! What can be simpler? On each keystroke, all you have to do is update a tiny rectangular region and modern text editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D game can fill the whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come?
Text is very complicated. Does your 42-year-old Emacs support Unicode? And not just accents, but whole different scripts?
See https://news.ycombinator.com/item?id=21105625 for some discussion and a good link about the complexities of rendering text.
emacs has also always been (in)famous for its text update render algorithms, that probably still work well over the old slow terminal lines it was originally used for.
Emacs had MULE in late 90's. It was THE best Unicode editor because it supported lot of encodings. You could switch from iso8859 to UTF8 on the fly.
Try planning a route in an unfamiliar area with this slow an UI when you are standing outside and there's no place to sit and rest, and you need to click around on a bunch of stops just to see what busses are going through a stop and where they are heading.
So yes, optimization is still important.
We replaced the glorious and easily iterated and expanded google maps app with a photograph of a public transport map, and we could get an answer of how to get from any A to any B within seconds of looking at the map without typing or searching or waiting for anything.
Which also shows that, sometimes, slow software is less than useless.
Sites like Hacker News or https://i.reddit.com are still perfectly usable. In contrast to the 'modern' Reddit UI that that takes a couple of seconds to load even on my home WiFi.
Bonus points if it also lists the before and after streets at direction changes so people know if they missed a turn.
This could be implemented as a proxy to either of the maps apps (Google, Apple, OpenStreetMap)
On my phone it takes about 4-5 seconds to load even on Gbit WiFi. It's so bad I have a conspiracy theory that the pulsing Reddit orb is just on a timer and it's not actually loading anything. More likely it's just horrifically unoptimized I suppose.
You can contrast that with mapbox which seems to have been designed upfront with offline mode in mind.
If you mean UX, then fair enough.
I think we need a guild. We need licensed software engineers.
Not every programmer needs to be one, just like not every engineer needs to be licensed, but there needs to be a licensed engineer on every team. And, of course, sometimes there doesn't need to be. But I sure wish there was the option.
And hell, bring back apprenticeships and mentoring with the guild. There is so much we could learn from the physical science engineering disciplines
However, the idea of working closely with Senior Engineers and learning from them is certainly something that I vehemently agree with. I've been fortunate to have had that opportunity.
Mastery of frameworks is NOT mastery of our craft. They are useful tools that can come and go. But the underlying principles are what should concern is.
To that point, none of your examples of stable disciplines are static. New surgical tools, techniques, and technology are constantly produced and surgeons must learn to lay down their old ones and adapt. Metalworking, carpentry, plumbing all need to learn about new materials developed and code changes.
All of those things are like frameworks.
Or fewer features. Talk to an actual professional who uses spreadsheets all day long about switching from Excel to Google sheets. It's infuriating the infantilization of UIs and the "oh they'll never miss it" attitude.