Because it's not built with client side optimization in mind any more. It's built with circular investment in mind.
Big tech funds startups (through obfuscated legal and financial means) and that funding goes straight back into big tech cloud hosting and SaaS services.
The private equity and venture capitalist playbook has always been enshittification: you either take an existing high quality product or innovate one subsidized with investment capital, then once it’s time to flip the business you 10x the price and cut costs (which usually cuts quality).
AI is the perfect tool for enshittification, because it can get you 80% of a finished product with a fraction of the costs it used to take, but the last 20% takes 2-5x times longer. So you just ignore the last 20%, make it up in volume by producing 100 mediocre products instead of 1 great one.
And that’s basically the state of the tech industry right now. I don’t think it’ll be like this forever, I think there will be better ways to collaborate with AI rather than wholesale delegation and prompting. But for now we’re stuck with mass enshittifation. People who have no sense of quality, taste, or craftsmanship don’t even understand, they skim over some generic blog post from Claude that takes five paragraphs to say one thing or watch the Coca Cola AI ad and don’t event notice it’s a different train in every shot and they think it’s so amazing. I hope this isn’t just the way things are now.
AI SDLCs simply compress the same incentives that were always there: you could invest in optimizing your architecture and improving your UX, neither of which make for big bullet points that close sales, or you could invest in new features that do.
The investment that was previously measured in months of developer time may now be measured in token spend across days, but the underlying dynamics shaping where it’s directed haven’t changed.
I don’t know why you got downvoted but you have a point. It looks as if OP doesn’t write for readership and his posts contents have similar characteristics of reckless abandon for conventional norms for a tech post. Too many cross references and link-backs to his own other articles that makes it really hard to follow the main thread of the essay.
The downvotes may be related to community enforcing what's in the HN guidelines:
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
Ironically a perfect thing to iterate on with your favorite coding agent. "make this page look better, present me 2 versions, I'll pick my favorite, and we'll iterate until I say stop"
Counterpoint - I like Dan's site. I think it fits his vibe and priorities, and I don't find it unreadable. I can always pick my browser width and font size with a couple keyboard touches, and I never have to fight his idea of how I should want to read it.
What exactly do you mean by "make it approachable"? Is the content-to-fluff ratio too high for you? This page is exactly what I want the majority of websites to look like: no fluff, no idiotic "modern design" trendchasing bullshit, just pure and simple content.
Also, perhaps you should learn how to make use of the user stylesheet feature of your browser, or ask an AI to tell you how...
I disagree. It does have some minimal CSS (it sets max width of pictures to 100%, which is reasonable and my opinion is that it ought to be the default), and in my experience the CSS usually makes it worse than not having CSS at all. This way, I do not have to disable CSS (nor use reader mode) to make it readable.
There is no reason for it to be non-extensible either and depending on the use cases people might want to trade speed/optimization for more runtime customizations which will inherently be slower than pre-baked SIMD regex and whatnot.
And now the other aspect of the trade off is token budget allocation.
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
So long as the way to improve software is "just stack another VM on it", software will continue to get slower.
Cutting edge development practice in 2025 was using a big JavaScript framework (not quite VMs, but they inherently strain the environments they run in).
That code is run by the JavaScript interpreter VM, which is running in the browser (another VM), which was (in whole or part) written in a managed language (another VM).
That program runs on top of the OS (another VM), which runs on another VM (the default hypervisor for Windows 10/11, or the desktop environment for Linux/macOS).
Each VM lets you have an entirely new managed programming environment- you have a new program that lets you write software even quicker than before- but that comes at the expense of memory usage and (most importantly) latency. Which is why it takes modern systems a few seconds to figure out you pressed a button in your React application that's running in an isolated browser instance consuming 1GB of RAM, where systems in the 90s were instant by comparison even though they had a fraction of the processing power.
It's conceivable that in 10-20 years, LLMs could be used as VMs in and of themselves, which will still likely be using 2020s development languages. So your software is going to require at least 128GB of RAM to compile, use another 32GB to run, and it'll take twice as long to respond to clicking a button.
For the same reasons, too. Software is buggy and slow because you impact the bottom line when you make it better. Same thing with LLMs and token expenses.
I agree, but it won’t be because we can’t do it. It will be easier than ever to achieve, but of course money for the tokens to do it will go toward some other aspect of the business. That the products get worse seems to be the rule.
you are probably right as far as desktop stuff is concerned.
But there were some sea changes from time to time.
When the iphone came out, phones became very responsive to fingers. hate the control apple gave to customers, but their guidance made apps respond in milliseconds. Before that phones were clumsy. Wasn't a desktop, but was different.
Will we have voice/chat interfaces that will be bloated... but responsive like a conversation?
For Desktop use, 100% of electron and python slowness will be gone in a couple of years, at least for new projects. If you're using Rust (or Mojo or C/C++), you'll have to try hard (almost a deliberate effort) to make software slow.
Well, at least with the RAM crunch that the AI boom has caused, I hope there will be some sort of pressure to write efficient software, otherwise device manufacturers are quite fucked.
I've been hoping for this for my entire software engineering career but it never came.
It's messed up. Sometimes it feels like everything in the universe is aligned to ensure that skilled software engineers lead horrible lives full of anguish, frustration and powerlessness.
It feels like the economic system creates incentives which go against efficiency. So then it makes sense that software engineers would be most impacted.
There's a weird duality to our job; it's supposed to be about automation and efficiency but in reality, engineers who make the most money are those who create unnecessary complexity because it creates better lock-in and thus gives them more leverage in negotiations.
Worst part is that nobody listens to us. Business people with money only listen to engineers who have money; who are mostly merchants of complexity. Engineers who want to improve things have no voice in this market.
Yeah, I've been thinking this as well. I have some optimism, but not with high confidence. The exponentially increasing power of computing hardware up until this point is often cited as the reason performance optimization has been sidelined in the software industry. Now that there's a definite hiccup in that trend, I'm hoping programmers will remember that software actually can be fast and memory-efficient, and that poor design choices that lead to bad performance are exactly that, a choice.
The author of the article definitely seems to think LLMs are what enables this to happen, but I personally am much more skeptical of that. I think what it really needs is bringing engineering back into software, not just throwing LLMs at it and calling it a day.
I cannot imagine how nitpicky people are. AI went from 'good autocomplete' to 'I just tell it to do an entire thing and it does it and it works the first try'.
Maybe pick languages that are more boring. That's what works for me. It worked for hiring crappy outsourced programmers and it works for AI. It seems to do well with PHP - no long running processes / side effects. it works great with golang, simple language spec, and a very common style of writing.1
Regarding speed - now it takes much less learning to use a fast web framework(like Meteor), this means web based software will be more performant.
Regarding security - i wonder: can we develop a framework that is responsible to 100% of the security, with zero responsibility for security on the app developer?
Color me sceptical but I’d say that’s a bold claim.
Can you give a concrete example of that being the case? I’d love to see the prompt and harness. And understand the scale of problem you’re talking about solving with a one-shot approach.
How long will it take to write the right spec? How will you know ahead of time that the spec is correct?
Which agentic guardrails do you mean? How can they be set up?
-
I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
> I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
Indeed. But how you came by that code has changed completely. If you still want to be able to produce at scale, at the speeds customers have come to expect now, you've got to use LLMs.
This makes sense for a verifiable task like a regex engine - but those are clearly defined and I doubt an LLM will have a meaningful speed increase over current human regex engines.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software
I recently built a piece of code which downloads a bulk set of data, indexes it for search, and then serves a pretty web UI on top of this with the help of some AIs. Normally I would have reached for sstables, sqlite, etc. This time, because the lookup patterns actually would not have been too efficient on sstables and SQLite would have been overkill, I had an agent take the data structures, pack the text effectively, and build a prefix tree for fast auto completion from the search bar. It was great. I could have done this all before but I wouldn't have. I would have felt sqlite was fast enough. The resulting web server is significantly faster feeling (because the optimized lookup speeds) than an sqlite implementation would feel like.
I think engineers building very complex systems now have a lot of performance knobs to twiddle that would have just been too costly for human effort. Since we constrain the responsibilities of the agent slop is less of a problem. We relegate it to defined tasks with clear API boundaries and test harnesses.
>LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly
LLMs have caused such an increase in ram prices that pine64 is no longer making linux machines. When you can't afford RAM, as a direct result of LLMs, it's going to make your assembly app slower too. Sorry, this is the future you chose.
Eh, not sure. Very few humans do software optimization productively, so I'm not optimistic about machines built by humans, either. Every encounter I've had with agentic optimization involved trying to apply a bunch of myths to an existing code base, in a way that seemed cool (unrolling loops, eliminating apparent branches, SIMD) but which was all pointless because the only credible optimizations were going to come from doing fewer loads and taking up fewer itlb slots and stuff like that.
Data point of one, but after a few months of uselessness, I have managed to get some pretty serious, measurable performance improvements with AI optimizations -- order-of-magnitude speedups of business critical processes which took days as well as significant latency reductions.
But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.
Giving an LLM a program that does a well defined thing correctly and asking it to make a faster version and a crapload of tests to show that it performs exactly the same task is a great way to burn through a bunch of tokens and getting a faster program in return.
I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths crossed, corners taken (plus a few other metrics, like proximity of opposite direction corners of unconnected paths.)
I would be genuinely interested to see how it evolves the same program over a large number of generations. Given a whole lot of software is more a collection of programs rather than a singular "program", I have to imagine that the outcomes would be vastly different in shape when dealing with smaller targeted, articulable-in-a-few-paragraphs programs vs many-years-of-business-domain layered programs.
Jevon's paradox implies we're getting more fast software, and more slow software. Sturgeon’s law implies the ratio will remain the same. 90% of what ships will be slow, buggy, slop. Just as it was before LLMs.
I wonder if AI has changed the 90% rate. From my perspective it’s already increased the ratio of what I’d consider bad.
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
I think this is the most accurate take on this thread. I've written super optimised code in assembly and SIMD. But only for contests never at work, because I've never had an employer that cared to optimise things that far, they just wanted cheap software that worked just good enough to get the job done. Even in places where they actually cared about quality would usually prefer adding a couple features over optimising things that much.
> performance work that used to require a person or team that had a rare set of skills can be done by anyone who can type a few sentences
This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not only do people believe this, even if they didn't, they would still have the problem that most people (including many engineers) are bad at telling who is a good software engineer.
So this means that code will get worse and worse; less performant, less reliable, less secure.
It will be impossible to fix because nobody would know what the problems are let alone identify who is able to solve those problems.
I don't know how bad it has to get before people understand we have to rebuild everything from scratch on much more rigid guardrails.
> This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
I think for well defined problems, it does a really good job, but if you're starting from scratch and don't know what to ask, it can take you on a wild goose chase in an extremely sub-optimal direction and it never backtracks from that bad design.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.
One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it.
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
I know nothing about web dev, but can't they use ML to predict where I'm going to click, and pre-cache the predicted page whenever the predicted button doesn't mutate important state? Or skip the difficult ML and have some basic rule of thumb that pre-caches frequent button clicks like a markov chain or something, conditional on those pages being low bandwidth to load.
If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and just pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web and somehow it feels like we've landed on the worst possible one.
> If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
Right, but my browser doesn't interpret that. The site tells me to run some code which interprets it.
I should be able to get the lay of the land without trusting the site enough to blindly execute whatever code it points me at. It's needless attack surface.
Also it's not really pointing me at data, its pointing me a certain kinds of requests which I have to trust will be responded to consistently. I'd much rather have a hash so if I have that data lying around I can just forgo the request entirely and use what's present locally.
>but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
And the first thing webdevs did once this became widely available, is change their apps to cache-bust their code; between that, and the short release periods in webshit ecosystem in general, and security and privacy considerations messing up things as usual, the promise of users mostly hitting just local cache with any marginal request, never materialized.
Interesting. Is this common? Like, is there some way to enable it on my LAN so that when I become disconnected from the internet I can still browse pages which were cached by other devices on that LAN?
1. You need write access to the server if you want to add one
2. The server could change its behavior at any time and there's no way to know that caches now need to be invalidated
3. If something goes wrong with connectivity or name resolution, there's no fallback since the authoritative thing was not something durable like a trusted human via a public key but rather an ephemeral thing: a named server which has pinkey promised to stay online.
It asks the user to treat a server like a trustworthy source of perisisant data.
But there's no reason to couple these kinds of trust. The skills necessary to persist and trafficking data are orthogonal to being trustworthy about the content of that data. Coupling them creates needless load on single sources of failure which are simultaneously single points for corruption to target.
Trust people, not servers. Use digital signatures to validate that what you're seeing came from those people. <a> tags prevent this.
> pre-caches frequent button clicks, using a markov chain
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
The problem isn't in preloading, it's in how much data needs to be sent while quite probably most of the data could be either fetched on startup in an efficient format and rendered natively, or is completely unnecessary in the first place (telemetry, ads).
So Next.JS actually pre-fetches links when they move into the viewport or you hover over it. It's interesting, but then you get wasted battery on mobile while on bad networks. The world is full of tradeoffs. Tech workers tend to want to consume more battery and data to be faster. Other people want to do less work.
This exist but the downside is that it uses much more of your bandwidth and client resources (probably not matter in many cases but it does if on a phone in a country with bad connection) and your server resources (if not mostly static content)
Oh god, don't give them ideas. All ML is in-cloud AI now. I dread the day everything around my mouse movements needs to get tokenized and vibed into the ClosedAI cloud before my buttons start working again.
segor.de goes one better and just downloads the entire catalogue when you first open it. About 2MB decompressed.
Clicking and even searching is instant.
Funny thing is it apparently predates JSON. It's a bunch of data[foo][bar] = baz; - go look.
Website's in German obviously, and a surprising number of German electronic terms are very different from English. They use two different words for stranded and non-stranded wire.
I have a new laptop with a rtx 5090. Opening any GL context takes more than half a second. There's tons of things that can be optimized and are pretty far from web.
I assume you're running proprietary drivers? Because I've never experienced anything like that on mesa. Launching an app that opens a window with a gl or vk context is so fast on my almost 10 year old hardware that it's nearly imperceptible.
There must be some quirk of whatever combination of software packages are installed there, but the proprietary drivers are not the culprit, at least not alone (i.e. there could be some interaction with other software packages with which I have little experience, like Gnome).
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
So the usual self inflicted misconfiguration then.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
I’ve been playing with this with software that needs to work with agents and also without internet at all (we’re serving construction projects that have limited access)
Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
I think a lot of the time when people say the network is slow. They really mean their backend is slow.
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
I don't know how hard it is. But I can certainly say there is no business inscentive for it.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
> With a fast backend ~1-5ms response times (not even that fast).
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
Project into sqlite on your app server is the main trick I use. Denormalize if you have to.
Hell, for a lot of projects you don't even need to get that fancy. Run a single server with an embedde database, running Go or Java and you're good to go.
More to the point it's that the server has to retrieve and massage data from several docker services to retrieve the full context needed to process the request
There are CDNs everywhere, true. It is not true that everyone deploys to them all, though. Also it dodges that the networks available to everyone are still not equal.
I think that "almost everyone" has a multi region cdn, but fewer have multi region application deployment, or cdn workers handling a significant portion of the application logic. My experience here may be incorrect or not generalizable, but I've rarely seen web apps that are slow due to latency loading static resources, but I've often seen slowness from high latency of the API calls and due to large static payloads.
That only helps the assets, does nothing when they run their actual backend servers in one of the US aws regions and your traffic has to traverse the planet anyway.
This is kind of a shallow assessment of "slowness". Slowness is a feeling, not a fact. Network is slow as a rule relative to other parts of the stack, but it is not usually what contributes to the feeling that your software is slow. It takes a good amount of incompetence and arrogance to cultivate that particular experience.
Fair, it's shallow because I was succinct however, I do understand the problem space more in depth than this.
But if I were to pick one single thing that would speed up the most UIs across the board, it would be poor handling of the UI in networked systems. As you noted, that doesn't mean eliminating them, it means handling the inevitable in a way that doesn't tank the UI feel.
True, dev environments are fast. One dev implements a wrapper with roundtrips, another integrates it into a UI and no-one stops to think if it'll have terrible lag in practise. They don't notice so even if there's a ticket it'll starve and end up WONTFIX.
Somehow I don't think I'm the only one who presses a button and when nothing happens presses it repeatedly until something happens, or I kill the app, or even power off whatever piece of shit computer I'm using.
UI blocking can make sense in some situations; otherwise things "happen" suddenly that are unexpected.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
This is why my most recent website does its server-side rendering on the client. I'm not even being that sarcastic. We stream a subset of the user's data (1 mb at most) in the background and have a WASM client that has the exact same server views. On SPA events, the WASM blob intercepts a lot of requests and can instantly render. Makes a laggy connection feel pretty quick.
This is an incomplete and quite superficial view of what is going on out there, in my opinion. I've worked on plenty of projects where the assumption was that since the round-trip to the server is going to take almost 100ms that'll dwarf anything that's going to happen on the server itself, justifying poor choices that lead to potentially adding a whopping 100ms onto that number. These numbers only get larger with a larger perceived "Nothing we can do about it" budget as well, programmers often feel justified in doing just about anything once round-trip time grows, not understanding that they're just adding to an already existing problem.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
I don't think it's superficial, but two problems adding up. Previous poster is talking about general latency issues because everything is networked and potentially quite far away.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
A friend of mine was once tasked with writing a kind of simulation that simulates millions of scenarios per session/run, and searches for a best-so-far solution while doing so. He proposed writing it in Rust (justifying it as: fast, low level, fewer memory bugs, fewer parallelism bugs (so potentially faster than "fast")), and management over-ruled them, and insisted on using raw/plain Python (without even an underlying C library), "because that is the industry standard", and "premature optimization is the root of all evil", and "nobody else knows Rust"[0].
Another friend, worked at a company, that got a new manager (I think as a result of a merger), and that manager halted all work on "yak shaving" projects. These "yak shaving" projects were things like logging, and debugging, and some kind of integrity-verification. When asked why they were being halted, the new manager said: "none of our customers asked for any of these things". When told that these things enable the team to produce a better product for the customers, the manager (I am told) looked at them with confusion and suspicion. Those projects were never improved since, and the product stopped improving as well. I am not sure if it affected their business (the pandemic was much more distortive).
What you call "business logic", is not even logic, and it has little to do with business. It is what Feynman called a "cargo cult". The obvious name for it is "cargo cult business management/logic".
It truly is embarrassing and shameful that after decades of idiotic decisions, it took a _trillion_[1] dollars of investment into a chat-bot technology, to finally crack open _one_[2] door to slightly less idiotic decisions, while opening dozens of new doors to decisions of an unknowable character.
Most companies (and, consequently, their engineering organizations) are simply _cosplaying_ as the things they are supposed to be.
I do not see how an AI assistant (or any kind of assistant or consultant) can save these fools from themselves. The only logical explanation is that most software companies are cursed -- you would have much better luck engaging a witch-doctor.
[0]: Nobody else knew C or C++ either. The fact is, that nobody cared. In fact, even Go would have been a better choice than raw Python, but nobody cared. Even Common Lisp (which is at least as abstract as Python, and has native execution speeds (GC and runtime type-checking can be turned off for compute-heavy workloads that mutate data in-place)), is a better choice, and yet, an _abundance_ of obviously superior options (all implemented and maintained by obviously superior engineers) was not enough to prevent the organization from choosing an inferior one, and using it stupidly (without a fast native-code component).
[1]: I see estimates from hundreds of billions to a trillion, depending on how you count it.
This is closely related to the "There's no point in addressing that problem, because if/when it does become a problem, you'll have 10,000 other bigger problems to worry about" mentality -- which quite ironically, inevitably results in 10,001 things going wrong at an inopportune moment.
I agree and see this as a side effect of a subtler thing. As people shall be replacable, software is designed and subtly mimics the organization's communication patterns. Features are cut up into the tiniest pieces with clear separation from the start (at least its claimed), and over time whatever change or feature seems overly complicated, won't be done or won't be done in a sane manner because its uneconomical.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
I could go on about this all day. Best meme is when you estimate stories for storypoints and then they haggle with you without changing the content of the story. Meanwhile points mean actual time. Then on another side if you haggled down points, you then have more slots for more points. So you end up getting assigned 3x work for the same time. The blame game starts when the sprints elapse lmao
The only ways out of these situations are: a manager that understands what you’re doing; knowing that these “story points” are just indicative and there’s no broken incentive towards gaming them.
Storypoints do not mean "time". Time would be too concrete and measurable and would be bad for selling more agile coachings.
Instead storypoints represent "effort". What does it mean and how can it be used to estimate a shipping date? I was told that I just don't understand.
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don’t anthropomorphize your lawnmower, the lawnmower just mows the lawn - you stick your hand in there and it’ll chop it off, the end. You don’t think "oh, the lawnmower hates me" – lawnmower doesn’t give a shit about you, lawnmower can’t hate you. Don’t anthropomorphize the lawnmower. Don’t fall into that trap about Oracle."
The UI threading model is usually not the problem.
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
Networks are much faster than you think, it's networked software that tends to be slow. 10 GbE is now table stakes, you should fire any vendor who can't offer it. I certainly don't require your internet connection to be 10Gbps, or all your desktop machines, but your internal server network should be if you're building a new one in 2026, because there's no excuse not to any more.
> Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful.
Yes we can - we can fix why the information isn't available. If someone said to you "sorry, we don't have the info because the other thread is doing Sleep(5000);" you'd call them an idiot right? You'd go and delete the sleep call to make it faster. Most real problems are harder than that, but there's no fundamental rule saying your database has to be slow. Ping time across your LAN is probably under a millisecond, so where are the other 299 milliseconds going? Is your database doing a full table scan? Is it using spinning rust for frequently accessed data?
The majority of companies are building their stuff entirely in the cloud, where network speed scales with the instance size. I have had to explain this to multiple engineers at multiple companies, who are surprised to learn that network bandwidth isn’t unlimited.
As to your database comment, IME most of the time the bottleneck is the ORM and/or language. The amount of work an ORM does to generate a representation of a row is frankly shocking. Not understanding the cost of context-switching is the language half of it: Python, of course, is single-threaded, but you can use greenlets to cheat, because they’re I/O bound — except for all of them serializing behind a single process handling serdes for the queries.
> If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
there is different scales at which software is slow. this os one and definitely a pain in the ass. everything being online for no good reason other than to harvest user data. which it really turns out to be every time. (for good or bad purpose).
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
This is true, but it is an entirely different problem in a different place to what the article is talking about.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
Yes. Apple Music is the most egregious example of this. It could be ridiculously fast on your pocket supercomputer but the moment a web request gets fired off from stumbling blindly across the field-of-dung user interface, bam, you’re done. Especially if your network connection isn’t great at that time.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
Sonos’ app is another example of this. As a very brief tl;dr if anyone isn’t aware of it, Sonos is a wireless speaker company that can group speakers in different rooms into zones, so you can have different music playing in different rooms, or all the same, or at different volumes, etc. The quality isn’t going to blow away audiophiles, but IMO they’re legitimately great.
The original design had the speakers setting up a private mesh network, and the app would send commands directly to the speakers via your LAN. Then, they got the brilliant idea to route commands via their cloud service. The app would send commands to an endpoint, which would send them back to your speakers. Imagine trying to smoothly fade volume with a WAN hop. This went over as well as you’d expect, and they’ve since promised to work on performance. Thus far they seem to have been doing so; it isn’t as snappy as the original, but it’s quite a bit better.
So many developers do this, and it's infuriating. I have a device sitting there on my perfectly good LAN, yet if I want to remote control it, the brilliant software decides to send the commands to the Internet, then back to my device, then the response gets routed to the Internet, and back to my phone.
Device developers, stop doing this! You people realize that LANs exist, don't you?
And look what happened to Sonos soon after that! I’m sorry, but I’ve gotten to the point where I would’ve just yelled back at the guy “Prove it, chudmuffin! You’re advocating for something you know nothing about and sending the company in a disastrous direction which will lead to its demise. If your idea is so great, then you prove it’s better than decades of established precedent at the largest companies in the world.” Then watch their head asplode and challenge me to a fistfight in the parking lot. Yes, I’ve had that happen with a sales “guy.”
Not saying that’s what you could or would have done, I’ve just gotten to the point in life where I’m alreaover it, ready to throw it back at them. Maybe it’s from living in this part of the world, but our plumber said “always be ready for people to be mean to you“ and while it doesn’t make for a very peaceful life, it certainly makes for fun ripping heads off.
Developers know better, but they are overridden by management and suits. Which pays their salary, showing that doing better usually gets sidelined by doing what’s good for you.
I'm not letting the developers entirely off the hook. At many companies, they are decision makers too, and partially share the blame with their product leadership and other decision makers.
Unpopular Opinion, but if you have absolutely zero say in the content of what you're developing, and just take orders from JIRA, you shouldn't call yourself an engineer. You should also keep your eyes open for a better job.
Agreed. Plus, if you can get an entire team to tell the PM “this is fucking stupid, we aren’t doing it,” what are they gonna do - axe an entire team? I doubt it. Maybe at big companies, but small ones? Nah.
I have a large and varied experience of “developers”. The majority aren’t any more morally virtuous than a brick. They wake up, get down to the sausage factory and make sausages.
Again this is mostly our biases playing us. Within HN and similar communities there are a lot of above average and caring developers. Those who don’t care aren’t going to be here toook like they do.
Yes, but my point is that it's an awfully shitty fallback if you need to wait 30 seconds for something to time out first.
It's one thing not to e.g. spend the extra time to ensure everything is cached and mutations are queued up. It's another thing not to do the bare minimum to ensure what is already available and working locally is gated on the network being up.
Case in point: The other day I was checking our train tickets in an app, and the network was awful, and the train tickets which the app has local copies of took 30+ seconds to appear when the network went down. Everything I needed worked once the timeouts had been hit, it was just ridiculously slow waiting for timeouts for functionality I wasn't trying to use to be hit first.
Yeah, those apps were all coded in perfect network conditions and the designers refuse to change the UX to inform user about origin of data (offline, last cached X mins ago etc.)
Shotout to PowerSync for enabling companies to go in the opposite direction and build offline-first apps. My company is a (production) customer, and we recommend it.
To dogfood this, rent a VPS in Australia and put a test environment there. Should be about 200ms ping or a bit higher. How it runs for you is how Australian users are seeing your site, even if their last mile connection is fiber.
Surely the biggest cause of slowness is doing more stuff.
We don't turn faster hardware into faster programs, we turn it into more program. AI isn't going to change that. We'll just get even more program because the optimisation has freed up space for that.
Unfortunately most of the time, the more program isn't for our benefit. I note that by far the heaviest program I use is my web browser. The one thing I don't get to choose what code gets thrust upon me.
A browser isn't really a program any more but a platform for running other programs. Like how javaw.exe is really Minecraft, firefox.exe is really YouTube. Go to about:processes to find more detail.
The point is, if I want to edit a text file, I don't need to use eclipse. I can use something that hasn't added loads of features. I don't need to use whatever Adobe product, I can use some paint app.
If I want to watch streaming videos, I don't have a choice about how I do that.
Fine Firefox is basically a bloated YouTube app. That doesn't change the fact that it is inefficient (from the pov of my CPU) for doing that.
This has been my bugbear for years. Even in places with good cell service on average there's a hundred individual places that have terrible service. Also a good signal to the handset doesn't necessarily mean good actual service. It doesn't even require traveling, just normal daily movements to get wildly variable network performance.
It's infuriating when it's obvious that the developer of an app only ever tested it in a simulator on their dev machine on their super fast WiFi. It never seems to connect with those people developing a mobile app (or web app) that the "mobile" part has a meaning more than just on a handheld device.
Such as? There are alternatives to almost anything you can think of already. All popular sites and services have clones. What more do you want?
The issue I think you’ll run into is that they’re lacking users and kind of just worse versions. Usually because people who say things like this, don’t actually want to be personally inconvenienced to switch.
Unfortunately, you either put all the commercially interesting bits on your own server and let your customers eat the latency, or you ship it to the edge and let piracy decimate your profits. I don't think there is any technical way out of this, and probably not any reasonable legal ways.
I feel like you're hitting at the real reason why so much of this happens. Not necessarily the piracy, but centralized control of the data and data flow. Even if it's slow as hell to send off that data to proprietary company servers (or rented cloud servers) to verify it, you're still verifying it. You, as in you the company, can't do that if it's local compute only. You can't control whether the user installed your paid plugin or some free alternative. You can't control someone stripping out libraries or code to remove intentional friction points designed to annoy them into a higher tier of the software. You can't control whether or not they update, or whether or not you can force the software into end-of-life with an update despite it still functioning.
If there's a connection to your services outside of the user's machine you can control all of that.
"In the Hypertext Transfer Protocol (HTTP) for the World Wide Web, servers insert a MIME header field at the beginning of any Web transmission. Clients use the content type header to select an appropriate viewer application for the type of data indicated."
A fan of "let's enable the program to do everything" philosophy I am not. This idea is embodied in the so-called "modern" web browser and a countless number of other "apps". Alas, this design, perhaps justified on "convenience" grounds (or so-called "user experience"), has been abused, e.g., for commercial purposes. One casualty of the abuse might be speed. Other sacrifices might be reliability, resource usage, "privacy", "security", etc. The most important sacrifice for me in using "do everything" software is _control_
Instead I use a number of small command line clients for making HTTP requests ("web requests").^1 I can edit the source code and compile these applications quickly with low resources
The clients are request makers, not response viewers. The historical "select an appropriate viewer application" step remains, as I prefer it
This software is not slow. I seem to avoid the dissatisfaction that I see from commenters who use software that can "do everything"
1. Generally this is one application that accepts URLs on stdin and generates HTTP on stdout and another that accepts HTTP on stdin, makes connections and sends it, typically a TCP client. But since I use a local forward proxy that has a built-in httpclient I don't necessarily need those programs to make requests, e.g.,
x=https://danluu.com/perf-opt/
echo "@1;expert-mode on;httpclient GET $x"|socat stdio unix:/path/socket
The proxy lets me control all the possible details of the requests (not through the built-in httpclient of course), including some details that can't be controlled using a gigantic, complicated, so-called "modern" browser
Feel like this is the equivalent of a traffic engineer standing at the grand opening of the 5th lane for the highway saying there’s no reason for traffic to be slow anymore.
That is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes).
you dont mesaure the quality of a plane by how much it weighs. but yea i think everything is rot to the core. app to os if just wackness that nobody cared about.
Sorry I can't resist the pedantry but a traffic engineer knows better than anyone that adding a 5th lane (often even a 4th) won't improve traffic except in fairly specific circumstances. :D
That cannot be an emperical statement. Traffic engineers have absolutely wrecked the american city, largely be ause they didnt know the impact of adding lanes.
It's not the only reason. I think that traffic engineers lack/lacked a feel, a certain gut understanding , an inability to feel through the skin. Just look around you. See and hear them at municipal meetings. They are unable to sensually experience the effects of their work, their praxis.
Who's to say they aren't the same as the developers implementing all the bad stuff at <pick your favorite hated tech company>? Listen to what they say, and plenty know it's bad, but they shrug their shoulders because that's what the bosses want and the pay is good.
I mean that was kind of my point, but judging by the responses I may have been too snarky.
Both the traffic engineer and the author of this blogpost are well-qualified enough in their fields of expertise to be careful about the claims that they make with regards to speed and efficiency.
A big issue is how those claims are interpreted by non-technical people, especially in leadership. Would hate for my CEO to stumble upon this blog post and believe, based on the title and a brief skimming, that his engineers just aren’t using AI correctly and that’s why the website still takes a few seconds to load its content.
The title is a disservice to the author’s point of view, then, because the article also reads as a “look what agents can do to optimize a toy problem” blog post.
Maybe the author should be a bit more careful about what titles they give their blog posts, lest they be misinterpreted by mere mortals such as myself.
The HN crowd understands that, depending on bottleneck, adding a fifth network connection (lane) between 2 nodes may not necessary decrease the latency (travel time).
I agree with traffic example, however, I don’t think it quite applies to software in the same way. Imagine a world where for some reason our current quality of hardware is locked in place; they can never get better or faster in any way shape or form. Then the big tech race will be optimizing software. I personally think that there would be a 10x to even 100x increase in software speed that is possible through many optimizations. The current software incentive is geared more towards adding more features rather than optimizing performance since software engineers know they can always know hardware will keep getting better which will keep their
slower, less optimize software at similar speeds. This isn’t a critique of software engineers, simply a statement of the incentives under which they operate.
Incentive has shifted. It’s now more important to worry about “developer productivity” and churning out as much code as quickly as possible than it is to spend time optimizing that code.
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
The refutation of your takeaway is autoresearch and similar. They can brute force novel optimizations (and generally achieve superhuman performance) when provided with an appropriate environment.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
I don't know if it's the first and it certainly isn't state of the art at this point but I think karpathy/autoresearch is a quintessential starting point.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
yep, i think the real LLM superpower is knowing that something has been done before and having access to the code that did it. so much of even novel software includes bits and pieces that have well-optimised existing solutions, and the bot knows those solutions a lot better than i do, and can even pattern match them from the general shape of the problem.
Yes. It’s also excellent at reading large codebases and putting together a picture of what’s going on. I’ve been using it a lot lately to brief me on projects and design decisions. “Look at these two projects. They both solve task X. Write a report about their similarities and differences, and the tradeoffs as a result.” And then I ask followup questions. Saves a ton of time.
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
> It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why.
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
I think you nailed it. And that would make sense and should be expected. 99.9% of the world's software (the training data) is several orders of magnitude away from max performance.
Another possible confusing thing for an LLM is that getting close to max doesn't necessarily require any "tricks." A big part of getting in the ballpark is just not doing anything you don't have to. If program A is faster than program B, most of the time is not some magic algorithm. It's that program A just did less stuff.
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
Reasons this doesn't follow:
(1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising)
(2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
419 comments
[ 0.93 ms ] story [ 9.8 ms ] threadBig tech funds startups (through obfuscated legal and financial means) and that funding goes straight back into big tech cloud hosting and SaaS services.
AI is the perfect tool for enshittification, because it can get you 80% of a finished product with a fraction of the costs it used to take, but the last 20% takes 2-5x times longer. So you just ignore the last 20%, make it up in volume by producing 100 mediocre products instead of 1 great one.
And that’s basically the state of the tech industry right now. I don’t think it’ll be like this forever, I think there will be better ways to collaborate with AI rather than wholesale delegation and prompting. But for now we’re stuck with mass enshittifation. People who have no sense of quality, taste, or craftsmanship don’t even understand, they skim over some generic blog post from Claude that takes five paragraphs to say one thing or watch the Coca Cola AI ad and don’t event notice it’s a different train in every shot and they think it’s so amazing. I hope this isn’t just the way things are now.
AI SDLCs simply compress the same incentives that were always there: you could invest in optimizing your architecture and improving your UX, neither of which make for big bullet points that close sales, or you could invest in new features that do.
The investment that was previously measured in months of developer time may now be measured in token spend across days, but the underlying dynamics shaping where it’s directed haven’t changed.
Otherwise, I would have left immediately.
I'm all for speedy, simple, plaintext websites, but it is a negligible amount of work in 2026 to throw some barebones CSS in and make it approachable.
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
https://news.ycombinator.com/newsguidelines.html
Also, perhaps you should learn how to make use of the user stylesheet feature of your browser, or ask an AI to tell you how...
But reading small white text on black background that's 100% width is nearly impossible to read.
And Munger's Law never cares if you hate its consequences.
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
Cutting edge development practice in 2025 was using a big JavaScript framework (not quite VMs, but they inherently strain the environments they run in).
That code is run by the JavaScript interpreter VM, which is running in the browser (another VM), which was (in whole or part) written in a managed language (another VM).
That program runs on top of the OS (another VM), which runs on another VM (the default hypervisor for Windows 10/11, or the desktop environment for Linux/macOS).
Each VM lets you have an entirely new managed programming environment- you have a new program that lets you write software even quicker than before- but that comes at the expense of memory usage and (most importantly) latency. Which is why it takes modern systems a few seconds to figure out you pressed a button in your React application that's running in an isolated browser instance consuming 1GB of RAM, where systems in the 90s were instant by comparison even though they had a fraction of the processing power.
It's conceivable that in 10-20 years, LLMs could be used as VMs in and of themselves, which will still likely be using 2020s development languages. So your software is going to require at least 128GB of RAM to compile, use another 32GB to run, and it'll take twice as long to respond to clicking a button.
But there were some sea changes from time to time.
When the iphone came out, phones became very responsive to fingers. hate the control apple gave to customers, but their guidance made apps respond in milliseconds. Before that phones were clumsy. Wasn't a desktop, but was different.
Will we have voice/chat interfaces that will be bloated... but responsive like a conversation?
It's messed up. Sometimes it feels like everything in the universe is aligned to ensure that skilled software engineers lead horrible lives full of anguish, frustration and powerlessness.
It feels like the economic system creates incentives which go against efficiency. So then it makes sense that software engineers would be most impacted.
There's a weird duality to our job; it's supposed to be about automation and efficiency but in reality, engineers who make the most money are those who create unnecessary complexity because it creates better lock-in and thus gives them more leverage in negotiations.
Worst part is that nobody listens to us. Business people with money only listen to engineers who have money; who are mostly merchants of complexity. Engineers who want to improve things have no voice in this market.
The author of the article definitely seems to think LLMs are what enables this to happen, but I personally am much more skeptical of that. I think what it really needs is bringing engineering back into software, not just throwing LLMs at it and calling it a day.
Yet when I ask for code it writes, by default, both slow and insecure code that mostly works. Kinda.
As I try to get AI to rewrite it into more secure, less bloated and optimized code is when it starts to randomly crash.
Then I read articles about how AI is "moving too fast" and cry.
Most of these things are just true as of today.
> You protect your job by learning how to use new tools as the technology advances.
Sure and the more you learn the more you see how hyped they were and reality untangle
> Engineers don't ask a tool to write code.
And what they ask? They create loops or graphs? These tools ultimately vomit code anyways
Maybe pick languages that are more boring. That's what works for me. It worked for hiring crappy outsourced programmers and it works for AI. It seems to do well with PHP - no long running processes / side effects. it works great with golang, simple language spec, and a very common style of writing.1
Regarding security - i wonder: can we develop a framework that is responsible to 100% of the security, with zero responsibility for security on the app developer?
The fact is that with the right spec, agentic guardrails, and evals, frontier models can now one-shot secure, performant code.
This is the job of software engineering now. Learn to properly use the tools for the job.
Can you give a concrete example of that being the case? I’d love to see the prompt and harness. And understand the scale of problem you’re talking about solving with a one-shot approach.
How long will it take to write the right spec? How will you know ahead of time that the spec is correct?
Which agentic guardrails do you mean? How can they be set up?
-
I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
Indeed. But how you came by that code has changed completely. If you still want to be able to produce at scale, at the speeds customers have come to expect now, you've got to use LLMs.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software
I think engineers building very complex systems now have a lot of performance knobs to twiddle that would have just been too costly for human effort. Since we constrain the responsibilities of the agent slop is less of a problem. We relegate it to defined tasks with clear API boundaries and test harnesses.
LLMs have caused such an increase in ram prices that pine64 is no longer making linux machines. When you can't afford RAM, as a direct result of LLMs, it's going to make your assembly app slower too. Sorry, this is the future you chose.
The Nobel prize for being smart is traveling to your mailbox as we speak. Do you also want it served with a jazz orchestra playing on the background?
Get real. Such bitter ranting isn't helpful for anyone. So: who is "you"?
But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.
problem solved
I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths crossed, corners taken (plus a few other metrics, like proximity of opposite direction corners of unconnected paths.)
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not only do people believe this, even if they didn't, they would still have the problem that most people (including many engineers) are bad at telling who is a good software engineer.
So this means that code will get worse and worse; less performant, less reliable, less secure.
It will be impossible to fix because nobody would know what the problems are let alone identify who is able to solve those problems.
I don't know how bad it has to get before people understand we have to rebuild everything from scratch on much more rigid guardrails.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web and somehow it feels like we've landed on the worst possible one.
Welcome to Next.js
The .js part makes me think not.
I should be able to get the lay of the land without trusting the site enough to blindly execute whatever code it points me at. It's needless attack surface.
Also it's not really pointing me at data, its pointing me a certain kinds of requests which I have to trust will be responded to consistently. I'd much rather have a hash so if I have that data lying around I can just forgo the request entirely and use what's present locally.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
1. You need write access to the server if you want to add one
2. The server could change its behavior at any time and there's no way to know that caches now need to be invalidated
3. If something goes wrong with connectivity or name resolution, there's no fallback since the authoritative thing was not something durable like a trusted human via a public key but rather an ephemeral thing: a named server which has pinkey promised to stay online.
It asks the user to treat a server like a trustworthy source of perisisant data.
But there's no reason to couple these kinds of trust. The skills necessary to persist and trafficking data are orthogonal to being trustworthy about the content of that data. Coupling them creates needless load on single sources of failure which are simultaneously single points for corruption to target.
Trust people, not servers. Use digital signatures to validate that what you're seeing came from those people. <a> tags prevent this.
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Speculation...
https://nextjs.org/docs/app/guides/prefetching#hover-trigger...
In my view, websites should not take seconds to load with gigabit fiber. Whatever happened to "mobile first"?
Clicking and even searching is instant.
Funny thing is it apparently predates JSON. It's a bunch of data[foo][bar] = baz; - go look.
Website's in German obviously, and a surprising number of German electronic terms are very different from English. They use two different words for stranded and non-stranded wire.
Try turning off wifi and see if it improves.
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
Hell, for a lot of projects you don't even need to get that fancy. Run a single server with an embedde database, running Go or Java and you're good to go.
Don’t know about that, pretty much everything is hosted on Cloudflare, Azure or S3 these days and all of them have at least one CDN on each continent.
But if I were to pick one single thing that would speed up the most UIs across the board, it would be poor handling of the UI in networked systems. As you noted, that doesn't mean eliminating them, it means handling the inevitable in a way that doesn't tank the UI feel.
Somehow I don't think I'm the only one who presses a button and when nothing happens presses it repeatedly until something happens, or I kill the app, or even power off whatever piece of shit computer I'm using.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
We've come full circle.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
Another friend, worked at a company, that got a new manager (I think as a result of a merger), and that manager halted all work on "yak shaving" projects. These "yak shaving" projects were things like logging, and debugging, and some kind of integrity-verification. When asked why they were being halted, the new manager said: "none of our customers asked for any of these things". When told that these things enable the team to produce a better product for the customers, the manager (I am told) looked at them with confusion and suspicion. Those projects were never improved since, and the product stopped improving as well. I am not sure if it affected their business (the pandemic was much more distortive).
What you call "business logic", is not even logic, and it has little to do with business. It is what Feynman called a "cargo cult". The obvious name for it is "cargo cult business management/logic".
It truly is embarrassing and shameful that after decades of idiotic decisions, it took a _trillion_[1] dollars of investment into a chat-bot technology, to finally crack open _one_[2] door to slightly less idiotic decisions, while opening dozens of new doors to decisions of an unknowable character.
Most companies (and, consequently, their engineering organizations) are simply _cosplaying_ as the things they are supposed to be.
I do not see how an AI assistant (or any kind of assistant or consultant) can save these fools from themselves. The only logical explanation is that most software companies are cursed -- you would have much better luck engaging a witch-doctor.
[0]: Nobody else knew C or C++ either. The fact is, that nobody cared. In fact, even Go would have been a better choice than raw Python, but nobody cared. Even Common Lisp (which is at least as abstract as Python, and has native execution speeds (GC and runtime type-checking can be turned off for compute-heavy workloads that mutate data in-place)), is a better choice, and yet, an _abundance_ of obviously superior options (all implemented and maintained by obviously superior engineers) was not enough to prevent the organization from choosing an inferior one, and using it stupidly (without a fast native-code component).
[1]: I see estimates from hundreds of billions to a trillion, depending on how you count it.
[2]: The performance door, if Luu is correct.
"This other unrelated thing in the code is slower, so writing this correctly is a poor use of time!"
Non-tautological rephrasing: When optimizing code, first actually just de-slowify it, and that should most often be plenty enough.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
Best example in a monolith case is oracle...
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
you are the fucking lawn mower
“…the structure of a designed system mirrors the communication structure of the organization that built it”
https://en.wikipedia.org/wiki/Conway%27s_law
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
> Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful.
Yes we can - we can fix why the information isn't available. If someone said to you "sorry, we don't have the info because the other thread is doing Sleep(5000);" you'd call them an idiot right? You'd go and delete the sleep call to make it faster. Most real problems are harder than that, but there's no fundamental rule saying your database has to be slow. Ping time across your LAN is probably under a millisecond, so where are the other 299 milliseconds going? Is your database doing a full table scan? Is it using spinning rust for frequently accessed data?
As to your database comment, IME most of the time the bottleneck is the ORM and/or language. The amount of work an ORM does to generate a representation of a row is frankly shocking. Not understanding the cost of context-switching is the language half of it: Python, of course, is single-threaded, but you can use greenlets to cheat, because they’re I/O bound — except for all of them serializing behind a single process handling serdes for the queries.
I wish Atlassian listened to you.
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
The original design had the speakers setting up a private mesh network, and the app would send commands directly to the speakers via your LAN. Then, they got the brilliant idea to route commands via their cloud service. The app would send commands to an endpoint, which would send them back to your speakers. Imagine trying to smoothly fade volume with a WAN hop. This went over as well as you’d expect, and they’ve since promised to work on performance. Thus far they seem to have been doing so; it isn’t as snappy as the original, but it’s quite a bit better.
Device developers, stop doing this! You people realize that LANs exist, don't you?
Not saying that’s what you could or would have done, I’ve just gotten to the point in life where I’m alreaover it, ready to throw it back at them. Maybe it’s from living in this part of the world, but our plumber said “always be ready for people to be mean to you“ and while it doesn’t make for a very peaceful life, it certainly makes for fun ripping heads off.
[1] https://news.ycombinator.com/item?id=49376040
Unpopular Opinion, but if you have absolutely zero say in the content of what you're developing, and just take orders from JIRA, you shouldn't call yourself an engineer. You should also keep your eyes open for a better job.
Again this is mostly our biases playing us. Within HN and similar communities there are a lot of above average and caring developers. Those who don’t care aren’t going to be here toook like they do.
- Access points that don’t let WiFi devices talk to wired devices.
- Office networks that don’t repeat broadcast discovery traffic into different segments.
- People in McMansions who don’t understand extenders and use two different SSIDs for each half of their house or for indoor vs poolside.
- Phone is on cellular network, speaker is on WiFi, “controls don’t work”
- Network teams who won’t “open the ports” or configure mDNS or Zeroconf or Bonjour but will let port 443 to the internet.
- people who want to turn their speakers on at home while they are at work, for their dog.
- people who want to turn their speakers off at their second home after they returned from holiday and left the pool party music playing.
- diagnostic logs that would be nice if they went somewhere.
- People who lost their phone and want a website on their laptop to control their speakers.
- people who have their speakers nicked and want to login to your website to see their speakers checking in and “remote wipe” block them.
- People who want the status of all their speakers live in their phone app, even the ones at other properties.
- tons of buggy low quality, low cost, network devices (switches, hubs, cables, routers, firewalls, NATs).
Just about the only thing which is used enough to be reasonably trustable on any random customer network is HTTPS to the big CDNs and Cloud providers.
It's one thing not to e.g. spend the extra time to ensure everything is cached and mutations are queued up. It's another thing not to do the bare minimum to ensure what is already available and working locally is gated on the network being up.
Case in point: The other day I was checking our train tickets in an app, and the network was awful, and the train tickets which the app has local copies of took 30+ seconds to appear when the network went down. Everything I needed worked once the timeouts had been hit, it was just ridiculously slow waiting for timeouts for functionality I wasn't trying to use to be hit first.
We don't turn faster hardware into faster programs, we turn it into more program. AI isn't going to change that. We'll just get even more program because the optimisation has freed up space for that.
Unfortunately most of the time, the more program isn't for our benefit. I note that by far the heaviest program I use is my web browser. The one thing I don't get to choose what code gets thrust upon me.
If I want to watch streaming videos, I don't have a choice about how I do that.
Fine Firefox is basically a bloated YouTube app. That doesn't change the fact that it is inefficient (from the pov of my CPU) for doing that.
Andy giveth and Bill taketh away. https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law
After traveling around in places with very poor wifi/phone data speeds. I couldn't agree more with you.
It's infuriating when it's obvious that the developer of an app only ever tested it in a simulator on their dev machine on their super fast WiFi. It never seems to connect with those people developing a mobile app (or web app) that the "mobile" part has a meaning more than just on a handheld device.
We as an industry should use AI to enable a standard of software quality that was previously uneconomical.
The issue I think you’ll run into is that they’re lacking users and kind of just worse versions. Usually because people who say things like this, don’t actually want to be personally inconvenienced to switch.
If there's a connection to your services outside of the user's machine you can control all of that.
"In the Hypertext Transfer Protocol (HTTP) for the World Wide Web, servers insert a MIME header field at the beginning of any Web transmission. Clients use the content type header to select an appropriate viewer application for the type of data indicated."
A fan of "let's enable the program to do everything" philosophy I am not. This idea is embodied in the so-called "modern" web browser and a countless number of other "apps". Alas, this design, perhaps justified on "convenience" grounds (or so-called "user experience"), has been abused, e.g., for commercial purposes. One casualty of the abuse might be speed. Other sacrifices might be reliability, resource usage, "privacy", "security", etc. The most important sacrifice for me in using "do everything" software is _control_
Instead I use a number of small command line clients for making HTTP requests ("web requests").^1 I can edit the source code and compile these applications quickly with low resources
The clients are request makers, not response viewers. The historical "select an appropriate viewer application" step remains, as I prefer it
This software is not slow. I seem to avoid the dissatisfaction that I see from commenters who use software that can "do everything"
1. Generally this is one application that accepts URLs on stdin and generates HTTP on stdout and another that accepts HTTP on stdin, makes connections and sends it, typically a TCP client. But since I use a local forward proxy that has a built-in httpclient I don't necessarily need those programs to make requests, e.g.,
The proxy lets me control all the possible details of the requests (not through the built-in httpclient of course), including some details that can't be controlled using a gigantic, complicated, so-called "modern" browserThat is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes).
At the end of the day its a mindset.
It's not the only reason. I think that traffic engineers lack/lacked a feel, a certain gut understanding , an inability to feel through the skin. Just look around you. See and hear them at municipal meetings. They are unable to sensually experience the effects of their work, their praxis.
Both the traffic engineer and the author of this blogpost are well-qualified enough in their fields of expertise to be careful about the claims that they make with regards to speed and efficiency.
A big issue is how those claims are interpreted by non-technical people, especially in leadership. Would hate for my CEO to stumble upon this blog post and believe, based on the title and a brief skimming, that his engineers just aren’t using AI correctly and that’s why the website still takes a few seconds to load its content.
Simplify, then add lightness.
[1] hn favorite word
Maybe the author should be a bit more careful about what titles they give their blog posts, lest they be misinterpreted by mere mortals such as myself.
Imagine traffic not getting any better after adding 1000 lanes in a medium sized town… not just to one area but to every single damn road.
Yet we’ve done this with RAM, CPU, and storage…
Not sure how either of those things are conducive to performance
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
A story.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
> Maybe I should document what I did!
Please do!
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
Another possible confusing thing for an LLM is that getting close to max doesn't necessarily require any "tricks." A big part of getting in the ballpark is just not doing anything you don't have to. If program A is faster than program B, most of the time is not some magic algorithm. It's that program A just did less stuff.
Reasons this doesn't follow: (1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising) (2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.