It used to be that you need a good reason to make huge refactorings, because it’s often so much work. Now agent can rewrite half of your code if your prompt is vague enough and you don’t actual try to review it all. And so the “soul” of a program can change dramatically every single day. It’s both great and very much not so.
The biggest obstacle to huge refactoring has always been minimizing the risk of bugs, not losing any features, and ensuring compatibility with the existing ecosystem. The reason it's become easier in the age of AI is because we stopped caring about these things.
Yep. That’s what people are forgetting. If you have an application that many people depend on to do real work, to make money, you won’t survive if you allow AI to constantly make huge changes.
Your test suite doesn’t cover all workflows. It doesn’t cover every combination of actions a user can take. So every big AI refactor while change some of those.
If this is happening frequently, your software will feel like a janky piece of unusable crap.
If the "soul" of a program (which is a vague term but I think I get what you mean) changes daily, that's indicative of a lot of churn, and a lot of churn is not a good idea in any project.
it's like changing how the tower of babel should be built daily. Just because you can doesn't mean you should.
I interpret "keeps rising" negatively. Changes keep getting made, certainly. The AIs will perhaps never fail to fulfill your feature request. But there's no overall plan. It's just undirected, cancerous growth. It's Homer Simpson telling a team of automotive engineers to add feature after feature.
This isn't really a good way to judge things. In the future, the fondest memories someone else has about technology will be about the present. The past is not better, you're just nostalgic for it.
Every time I think the past was better, I think about how terrible ksh scripting was in 1995. And look at how great peoples' bash scripting is now compared to when we though bash had reached its apex in like 2009.
lol I was actually kidding. The past was way better, almost always (it isn't just nostalgia). I think my enjoyment of tech actually used to sort follow a wavelike pattern. Then, it sort of became a straight line a few years before COVID, and I worry that AI is cementing people into an always-forward mindset that removes the enjoyment from just about everything.
I mean, yeah, hobby coding is not going away, but the feeling of exploration for me is totally gone. I don't dislike tech, I just don't see anyone being the way they used to be. People in tech are different and too many people are in tech.
Probably only part of the joy will ever be there, now. Which is weird because I did my own thing with computers til college, and don't consider myself a super sociable person. I just used to know that it was there, and now it isn't.
> There is the appealing idea that AI-assisted programming means better tools which lets us build more ambitious software. That is certainly true at the level of the individual and without doubt a developer with an agent will be dramatically more capable of changing a codebase. But large software projects have never been limited only by how quickly an individual can produce code. They are limited by how well people can coordinate their understanding of the system they are changing.
So true.
Since Nov 30, 2022 everything has become… more complex.
I don't know. some stuff has gotten less. Major databases now ship effective HA tooling, microservices seem on their way out, structured databases seem to be back in instead of NoSQL.
HTML and pre-rendering are back in, HTMx, liveview
The degaussing of CSS and the hacks we did, hell i was trying to explain how we debugged web pages in IE6 to a younger staff member today.
Some things are more complex, some things got good enough to make them less complex.
>Since Nov 30, 2022 BC everything has become… more complex.
FTFY
Increasing complexity is the story of mankind. It's the story of civilization.
Someone from 20,000 BC would wander around the earth trying to find food, trying not to freeze, and trying not to get eaten. Someone from 5,000 BC would be trying to grow food, hoping it rains, and hoping disease didn't wipe out the village. The second one increases the complexity from all the systems required to manage people and keep the land growing. Today the vast majority of people on earth don't grow their own food at all, and instead are busy in some way managing the complexity of a large society.
Someone from 1970-80 would think our software from pre-llm days was vastly more complex. They'd just code directly to the hardware with no abstraction layer. Now almost no one does that. We abstracted the hardware away in most cases. With cryptography libraries for the vast majority of people it's complexity is abstracted away and mostly people are told "don't try to write your own crypto because you will fuck it up".
The question now becomes, how quickly will LLMs be able to coordinate their understanding of the system they are changing?
Then I think you should check in with your favorite mental health provider before you become a danger to yourself or others.
Simply put LLMs do understand some things within their crystalized intelligence. Your anthropocentric mind may not accept this, but one day it will. As LLMs have a very short context window in relation to their stored knowledge they have very limited plastic intelligence to change their minds or adapt. All of which is flushed away at the end of a session. It would be like living without the ability to turn your short term memory into new long term memories.
I would gladly use another word for what LLMs can do, but the world at large has not adopted any. The definitions we use around intelligence, comprehension, understanding, consciousness, and sapitence have already been failing us for some time before LLMs as our scientific understanding of biology has increased over the decades as it is. I am one for more exacting definitions when they exist, but humans seem to barely understand the inner workings of our own minds, in large such words escape us.
I'll meet you in the middle: an LLM "understands" words in the same way a toddler understands the phrases they say. "My want cookie!" The toddler has zero comprehension of what any of those words mean, but they know that saying them in that order might result in something desirable.
An LLM has zero understanding of "my", "want", or "cookie" because an LLM has no id/ego, has never felt desire, and has never eaten a cookie.
I believe you've made a category error in understanding, um, understanding. You've tied emotion into it. This to me are entirely different concepts where both happen to be wrapped up in meaty flesh that drives us humans. Now, these concepts are very important in sociology and human understanding of how we behave, but they also may have zero importance for the domain that encompases all understanding.
HN would commonly recommend reading the book Blindsight here.
Moreso, all you've done is recreate the Searle Chinese Room thought experiment which gets bounced around with no means of deciding if it reflects reality or not.
Ask an LLM for the difference between a definite and indefinite article, and then pronouns. Maybe just a whole run down of basic grammar. And then read the comment again.
the shy one of my shelter cats that figured out how to open and close the cabinet below the one with wet food at 3am without meowing has infinite more understanding than any LLM of how the world works
For what it's worth, nomadism and agricultural systems are not necessarily more complex - that is, the complexity of a nomadic lifestyle can be quite high. Randomization, varied food sources, hybrid lifestyles, etc.
The hierarchy is certainly higher in agriculture societies it seems, but the complexity is up for some debate
I feel like with software, things have gotten way too complicated (just layer's upon layers upon layers). But to deal with that complexity, now we're using something that just creates WAY more complexity. I've been coding for a while, and I remember the 90s and early 00s where people could make pretty powerful applications with like visual basic or php with essentially no formal training. Those technologies weren't great, but they were really simple and easy to pick up. In contrast, if you try to pick up web development or desktop app development today, it's absolutely overwhelming. Like, something like React is useful but the amount of things you need to know to use it properly is pretty high.
I think introducing AI to deal with this is overall a mistake though. We're just adding more complexity on top of the existing complexity. At best, it's a massive waste of hardware. At worst, we'll probably have agents introducing as many bugs as they fix as they also drown in complexity, and a lot of stuff built using these techniques are going to be fragile garbage while the overall skillset of humanity diminishes because people aren't learning the skills anymore.
Fundamentally, software does not need to be this complicated and it's a solvable problem, but it does require people that care about craftsmanship.
And what's ironic is that a lot of those layers and complexity were added with the stated goal of making it easier for average developers to build applications.
I had a discussion with folks at work about what information is worth retaining in the face of AI doing everything for us. A lot of what we have in our heads to qualify as "domain experts" is pretty esoteric. How to invoke command line tools, gotchas because library A uses one convention over library B, AWS vs GCP; so much is specific to a tool rather than a method. There are also a lot of entrenched tools that are effectively unfixable due to the risk of breaking changes, so you have to shrug and accept + learn that's how it works.
Catch-22 is it's still important to know the fundamentals so you know what to ask for, but if you don't know the esoterica, the model is eventually going to make an assumption and screw things up. And the models don't have much taste either in prose, or in coding/comment style.
Yes I think the problem is not that AI "has all the answers". It is that we don't have "all the questions". If we don't understand the whole of what AI produces for us, we can't ask it to make parts of it different. We are not in control any more of what exactly is produced, and we don't want to give AI all the control, because then it would not be doing what we want it to do.
Therefore it is critical that whatever AI produces is understandable to us humans. That is why we must demand that AI tools and agents produce "well designed" well-structured software. That's the bottlenexk to progress I think. Even AI can't deal with exponential complexity explosion.
> They are limited by how well people can coordinate their understanding of the system they are changing.
It's not really news, though. Programming as Theory Building (Peter Naur) was published in the 80s, I think?
Maybe the younger entrants to this field never came across it, but even if you never came across it, it was common knowledge amongst experienced devs that understanding of the system you are about to change is crucial.
The complexity of coordinating a project involving more than one entity is, of course, an issue across all industrial sectors—just look at the construction industry.
Thanks for mentioning Peter Naur’s Programming as Theory Building (1985).
I would add Fred Brooks and his The Mythical Man-Month.
It’s global madness fired up by continuous stream of news from LLM providers. It’s like The Verge that is almost about FAANG only, but multiplied as it is “magic” for most people, as vibe coding is “so easy” and dopamine-producing activity that it is similar to runners that don’t want to stop as it stimulates them (and in this case ofc it is healthy :)
I knew a runner that was wearing his body down. His joints where giving up. I predicted he would become a cripple. He wouldn’t stop running despite knowing. He was addicted. Some stories about prompting agents feel like that to me.
I've said for a long time that composability in software is a bit like playing Tetris: the lines have to clear.
I feel like that gives an even more literal tower-rising metaphor, and that's what it feels like people using agents naively (and software engineers of lower skill or earlier-career), end up violating.
Agents are getting better at folding things into themselves, especially if you direct them to... but unfortunately I've found that the architectural instincts, even of Fable and 5.6 Sol, are still wildly behind what I reflexively achieve, say.
For sure there is an ability to have agents go back over work and try to fold it into better and better abstractions until it's sort of annealed into something good. I've done something similar on codebases that I have, but the 'high reaches' of architecture with great _prediction of how the software will evolve in the future_ in _subtle_ ways – those are, for now, out of reach of agents.
There is a part of me that wonders if it's partly just how much they can hold in their head right now, though. Even with the greatest articulation and high density of feeding them, the current setups don't allow them to hold a high-quality, sparse, 'zoomable' model of the world in their head that well yet, which we can do pretty well.
But the fact that I'm talking about it in terms of that kind of subtlety is itself promising, I guess?
Unless we are planning to deploy them all individually to an expensive serverless platform like Lambda, the coordination challenges and overprovisioning are going to more than outweigh whatever architectural benefit you reap (in human-centred development, micro services are solving an entirely different problem - Conway's Law)
> if you can’t design a modular monolith, you can’t design a set of microservices.
I somewhat disagree:
Yes, it is possible to design a modular monolith, but thinking about the system in terms of a "minimum viable service" (but keep an eye on "viable", otherwise you can easily get into the "interaction problem") makes it much easier.
This is very similar to how you can write programs with no implicit state in a imperative programming language, but doing this in a pure functional programming language such as Haskell is much easier.
Microservices are about separate deployment. Regarding separating the development and maintenance of components, you can achieve that in a monolith by composing it out of corresponding modules/libraries with defined APIs. That’s good practice anyway.
What problem would it solve? They're still part of a larger system ultimately. Sure, smaller codebases with more focused scope can be good for e.g. human individuals and LLMs, but there's multiple ways to achieve that that don't require a network boundary.
Agreed, and ever since LLMs started being able to write competent code, I've noticed a massive difference in quality on codebases where I knew the technology, and ones I didn't. This is because I can much more efficiently steer the LLM on e.g. backend code, which is my expertise, vs yoloing everything on mobile, where I have no idea.
The codebases using technologies I have no idea about tend to quickly become unmaintainable and buggy, because the LLM still doesn't make good architectural choices, but the codebases that use technologies I'm familiar with basically never devolve into unmaintainability.
The difference between the two is massive, and that's why I think that a competent engineer steering an LLM in their area of expertise gets two orders of magnitude more productive, whereas someone steering an LLM in an area they know nothing about are basically producing tech debt at the speed of thought.
I just see in my usage that I can release tens of features a day, whereas I'd be able to release one or two a day usually. I don't know if it's 100x, but it's definitely more than 10x.
The biggest thing to get right is to let the LLMs do what they're great at (code implementation from very detailed specs, and code review), and you do what humans are great (architecture and making sure the high level of the implementation is sane). That way, you get the best of both worlds, and a lot of speed at high quality.
Does this speed of development only apply to the early stages, or will it slow down as the codebase grows? I have yet to see these 100x improvements (and the claims themselves keep getting bigger) on anything more than prototype scale, rather than month-long projects.
No doubt AI boosters will claim we have found an O(1) productivity enhancer that is somehow two orders of magnitude better than anything else before. This is entering lunacy territory.
I have a theory (armchair take here lmao) that AIs are trained on public code, but the biggest codebases are not public.
Although I suspect models from Google, Facebook and Microsoft can be trained on their massive internal codebases. Whether they are is another question.
I’m not sure you’d see that big of a difference in quality. There is quite a bit of cruft that can accumulate when you know the code will never be public.
But, you would probably see a difference of scale and architecture. Larger projects that need better organization are probably more likely to be in private codebases (Linux excluded). So you might be right about the lack of private code in LLM being an issue.
At least in the past (before LLM-based code contributions got socially acceptable in some circles), in open-source project you often got very direct comments on code that was of bad quality. Yes, this was always a little bit abrasive, but it did a lot for the code quality.
For internal applications used at companies, such an abrasive behaviour is typically not accepted ("not a team player" (as if this is something bad), offended snowflakes, "not socially adept" etc.). Thus the code quality suffers quite a lot for internal applications.
Have you ever thought why salespeople have such an easy time selling some LLM for coding to big companies? Because the code quality of many internal applications is so bad, which makes even shitty LLM slop code often better than what is there.
I think part of the problem is the context windows for humans are actually much smaller than what an LLM can keep track of today. The small context window of humans is a feature that forces modularity and abstraction in software engineering so that you can decompose what you're working on into something that can fit into your head. But since LLMs can fit so much more in their head, so to speak, they don't have this same incentive, and you get the unorganized mess of spaghetti code that current agents often produce.
Also a human has to explain the code. There is a social contract and producing slop gets you fired. Especially as you produce it much slower than an LLM.
Just a matter of time. Go download gpt2 or llama2 and be shocked at how bad they are compared to today. They were entirely "useless" yet we marveled at them. Go examine GPT3.5/gpt4 out which was all the rage and then marvel at how a qwen27b or gemma31b model mops the floor today. My point is that the models will eventually learn to have a great model of software system in their head, just a matter of time and proper RL.
I can't prove it but I have strong beliefs that the logic and intuition required for abstracting for future changes is not possible in a stream of predicted tokens. Something's missing that can't be measured.
The real understanding builds on not knowing. Once I realise I don't know something I cease thinking and start watching the thing. Then somehow I understand.
We do have a mechanism similar to LLMs but it provides existing knowledge; it's a search mechanism, basically. New knowledge happens when we turn that mechanism off.
Maybe not exactly the same thing, but I think one related representation of this in any good codebase is documentations in the code:
Good documentation always focuses on the why and not the what. i.e. things that are not represented by or representable in code.
It may describe not only why a certain choice was made, but also why not something else.
In general, the whole point of these documentation is to account for things that are beyond what can be understood from just reading the codebase including other existing documentation, that you may never even imagine with all the tokens in the world otherwise, which were yet a critical part of our coding choices.
If we are to believe that LLMs can build complex systems, then it's equivalent to saying that LLMs can make similar decisions equally well without any of that... which also then implies that we essentially never needed these documentation to begin with other than as redundancy for human needs.
> There is a part of me that wonders if it's partly just how much they can hold in their head right now, though. Even with the greatest articulation and high density of feeding them, the current setups don't allow them to hold a high-quality, sparse, 'zoomable' model of the world in their head that well yet, which we can do pretty well.
My personal experiments show that giving them tools to access to all past sessions over a codebase helps a lot. When I coded "feature X" 2 months ago i likely specifically mentioned some constraints expressed as abstractions and, if the coding agent checks not only the code/feature it needs to implement/change but also the past sessions over that, it picks them up, and ships code that better fits the overall project.
At least, more than "architectural/design guidelines", since they are more "concrete", to the point for the task at hand.
Sessions self-preserve them for following sessions, which helps, but might also carry over stale things, so some "pruning" helps, and can be automated. Overall, as long as corrections were also made via agents and hence in sessions, they are picked up automatically.
It's not the "human zoom" you refer to, which as humans we can drive/control, but the effect seems to be similar: I read autonomous sessions where it picked up from the past the very design/architecture/abstraction points i would have driven, had I been in the loop.
In team contexts it should not be impossible to share sessions, but I not working in teams right now :D
So maybe it's also a "single dev quirk".
Another point is that in a sense, learning to code goes from being told "you are using the_wrong_abstraction/this_abstraction_the_wrong_way/no_abstraction_where_you_should_have", to telling it to others/ourselves.
Since the number of knowable abstractions seems to be "at least one more than I already know", agents can actually be helpful in learning.
After a certain threshold, it basically zeros in a specific domain, but on novel domains agents taught me abstractions, when nudged towards doing that.
Yeah I would love to sit down and see what exactly one of the 20x people us doing. Either I spot the flaw (99%) or there is some utter magic they are doing (1%) which I could try to learn.
My comment is not directly responding to the essay, but it got me thinking about about how agentic programming is much more akin to management than it is to actual programming. Managers generally only have a high level idea of what ICs are working on and often don't have the time, bandwidth, and in some cases ability to understand everything the ICs they're supervising are doing. As more and more software gets written agentically the role of software engineer because less technical and more managerial.
It feels to me like I'm stuck doing code reviews for a junior dev all day so I use it as little as possible and mostly to look for things I may have missed.
For example, yesterday I came across some unit tests that didn't have error messages in their assertions. Normally, it takes me ~10 minutes to fix a handful of tests in this situation. In this case, I gave a 2-3 sentence prompt, went to the bathroom, and reviewed the result after I washed my hands. Saved me a bunch of time!
I encourage you to accept a feeling of "imposter syndrome" when using it, and keep trying new things with it. Don't feel like you have to be hands off, except when you're confident that you can be. (IE, if you think you need to spend 30+ minutes on mindless refactoring, see if you can explain it to an agent and then look at HN while it runs. You might get a good result, otherwise, it probably was time for a break anyway.)
BTW: It's important to try different models. The Claude 5.0 models are slow and give me bad results, so I'm sticking with 4.x for now.
Yes it's so useful for mechanical changes, refactors, and creating similar but slightly different components etc. It turns out that when you feed the word prediction machine a bunch of examples of what you want, and then some text that it can use to predict more text, it can do a good job!
The hard part is what text you feed it and how to judge the output.
We can say that about programmers, most ICs don't understand what's going on in the layers beneath were they work. Most have no idea what's going on with libraries, frameworks, remote APIs, it's all abstractions. Most people can't tell you how system calls are implemented or function. They don't have the time, bandwidth to understand it all, they just operate at their own layer to get the job done.
While this is true to a point, the other factor is that developers are hired and tested for having certain skills and specializations. While I'm sure LLMs can be good at things, the question is whether their manager can accurately judge the output. I suppose this problem applies to developers too, but developers have peers and processes.
So as someone managing LLMs you need to put those processes in place too. The risk is that one tries to do too much and loses overview / insight. Focusing on getting the tower tall instead of sturdy, if you will.
You use a shared agents.md and an auto updated architecture doc but that is the one that needs to be heavily scrutinized and everyone gets a turn to review it.
this doesn't work in any truly complex system. If the entire organization's shared understanding could be captured in a few documents, software engineering would've been a solved problem ages ago.
Would be interesting to see an example. I heard a couple times that code maturance is very similar on how a city grows naturally, but never seen an example.
I come back to Babel and the Bruegel image too, although taking from it a little less optimism.
I feel these systems rising and sprawling with wee myopic agents developing out their little corners of this unknowably vast whole… a tower with 50 parapets on one side and some wacky cantilevered maiden tower on the other, and a very serviceable adobe roof over some patio for god-knows-why, and thatch over the landing next to it…
Some grotesque fatberg of designs that make sense at the level of individual design efforts, but that lack the fractal sort of levels of policy and judgment that unify the overall enterprise.
The overall language, as it were.
And language takes discipline to establish and maintain through any sufficiently large group of people—witness the company-speak or army-speak of pretty much any successful organization.
We feel like we’ve conquered the problem of talking the same language as our “Gastown Mayors” (who in turn are talking the same language as their “polecats” and so on all the way down the chain of golems)… but it’s only when it’s all built that the good Lord will humble us… that we’ll realize the understanding we thought we’d transmitted perfectly from our thrones wasn’t quite so shared as we’d imagined.
The agent will always fill in the gaps in your understanding. It's not a compiler. It's categorically different from any of the other ways we've built software.
I'm not sure reading code is coming back. The ritual of reading code must come back, because that's the only way to build products that don't collapse under their own incoherence, both technically and visibly.
"just ask Claude" is fine, but it's not the end state
> compared to languages which demand much more effort to get anything substantial done.
It is not clear at all to me that other languages "demand much more effort" for the same end result.
It is clear that many non-lisp programmers value syntax, and many lisp programmers don't. Even many people who programmed enough lisp to have their minds blown and expanded still prefer not to program in lisp. I'm still awaiting psychological studies on this, but the rift is so large, I think there may be some significantly different brain processing going on between the two groups.
To your point, yes, it is also clear that, to the extent that lisp can match the productivity of other languages, whether it exceeds them or not, one of the tools that is needed to achieve this productivity boost in lisp is heavy usage of homoiconicity, and this results in every serious lisp program being a collection of DSLs, each of which is only understood by one person or very few people.
> Even many people who programmed enough lisp to have their minds blown and expanded still prefer not to program in lisp
For me, the answer to this is economics. Even if you love lisp, there are way more companies hiring for stacks that don’t include lisp
If you then optimize for employability, which I assume most developers do (not all, but a large percentage), you might end up with not that many people practicing lisp regularly
Au contraire if typescript and rust didn’t steal the whole show it’d be a great time to be a lisp LLM pilot: agents can explain pretty much everything without any confabulations nowadays, so the understanding problem essentially goes away, if you care, which is exactly the point of the article if you ask me.
Assembly programmers made the same argument. It seems that we revisit this same trope each time the practice of software engineering undergoes a paradigm shift.
Some have a harder time with the transition than others.
Are you implying the author of one of the best, if not the best performing agentic coding harness is having a hard time with vibecoding as a new paradigm?
Is it really the same argument? I think there's enough of people that will argue that there will be no need to come together to collaborate, just rally a bunch of agents and you can build whatever non-trivial stuff you can imagine.
A fallacy that a lot of people have is that productivity equals progress. Programmers (and their managers) using LOC as a target metric, writers using words/chapters written, etc. For some reason this fallacy is reinvented and repeated every so often.
I don't know why people hold on to all this extra software and features when with the tools its easier than ever to strip that out and refactor the end product in to a much more compact deliverable. Maybe once upon a time it was useful to keep legacy parts of the software solution around, but it can be recreated with fresh eyes if needed given the power of the new LLM models. My philosophy is if its not needed, it needs to be removed.
Three or so years ago, Omar, the creator of DSpy pointed out on Twitter that ~LLMs get better most by better internal collaboration. Wish I could find it.
It seems to me that LLMs and particularly chatbots have already allowed for bigger scale collaboration within the LLM companies versus what was possible within the prior cohort of big platform companies.
Has the result just been taller towers, or actually a change of what is possible?
AI replaces a single tower with millions of 5-over-1s[1]. The aggregate height, and speed of construction mind-boggling, but when each building is considered individually, not very impressive.
1. Perhaps with a handful of skyscrapers sprinkled in.
so the post-AI landscape as a software shanty town? That's not a bad analogy. We won't like the looks of it and it will barely function but like real life shanty towns it will function nonetheless.
at one point - future generations - will look at people who designed unix like tools - tools that do one thing well & compose with other tools as demigods.
> But it’s not the biblical story. At Babel, the loss of common language stops construction whereas in AI-assisted engineering, construction can continue after shared understanding has already collapsed. The lack of an immediate failure is what makes it curious and a bit disorienting. The tower does not fall, and so we do not notice what was lost. It just keeps rising.
I don't know whether the author thinks this is a good or a bad thing, but in my eyes it's clearly a bad thing. Intelligence is knowing that a tomato is a fruit, wisdom is knowing not to put it in a fruit salad. AI is the the ultimate form of intelligence with zero wisdom. Actually, it's not even intelligence, it's an illusion of intelligence. If there is no human who can understand what the AI is doing it's time to stop and accept that we do not have the wisdom to contain what we are building.
> Intelligence is knowing that a tomato is a fruit
Intelligence is learning to avoid using childish cliches, unless your intention was to mislead. Categorisation and understanding dependencies are hard enough problems already.
At the supermarket or taxation (Nix v. Hedden) tomatoes are a vegetable.
Speaking of "categorisation and understanding dependencies" and "intelligence" and "fruit":
Interestingly, the Bible does not specify what kind of fruit was on the Tree of Knowledge. The association between the forbidden fruit and an apple was a derivative of later translations.
Imagine the consternation if the claim had become Adam and Eve ate of the forbidden tomato. Then this incessant controversy regarding the tomato's status would take on a biblical dimension... though then perhaps the Mormon claim that the Garden of Eden is to be found in the New World would actually have some credibility. Indeed, answers to some questions are not a matter of intelligence at all. Sometimes it seems to me intelligence is more about knowing what questions to ask in the first place.
> I don't know whether the author thinks this is a good or a bad thing, but in my eyes it's clearly a bad thing.
I like that the author lets the image do the work, rather than preach at me even if I were to agree. History never repeats itself but it always rhymes.
You picked a bad example because this is a thing where LLMs excel. A human doesn't know that tomato doesn't play well in a fruit salad until he learns it; not that different from LLMs.
What LLMs lack, from my experience, is two things: First, a long term memory (even a zoomed out one) and Second, the ability to execute a multi-step task without fizzling out. Upon adversity, LLMs breakdown and start making shit up. The newer models are better (and that's the difference between Fable/5.6 and GLM 5.2) but still have a very limited ability to execute any tedious planning tasks.
178 comments
[ 3.1 ms ] story [ 56.2 ms ] threadYour test suite doesn’t cover all workflows. It doesn’t cover every combination of actions a user can take. So every big AI refactor while change some of those.
If this is happening frequently, your software will feel like a janky piece of unusable crap.
it's like changing how the tower of babel should be built daily. Just because you can doesn't mean you should.
I mean, yeah, hobby coding is not going away, but the feeling of exploration for me is totally gone. I don't dislike tech, I just don't see anyone being the way they used to be. People in tech are different and too many people are in tech.
Probably only part of the joy will ever be there, now. Which is weird because I did my own thing with computers til college, and don't consider myself a super sociable person. I just used to know that it was there, and now it isn't.
Is that because of the technology or because of who you were at the time?
So true.
Since Nov 30, 2022 everything has become… more complex.
HTML and pre-rendering are back in, HTMx, liveview
The degaussing of CSS and the hacks we did, hell i was trying to explain how we debugged web pages in IE6 to a younger staff member today.
Some things are more complex, some things got good enough to make them less complex.
Which ones? PostgreSQL doesn't have HA in core.
FTFY
Increasing complexity is the story of mankind. It's the story of civilization.
Someone from 20,000 BC would wander around the earth trying to find food, trying not to freeze, and trying not to get eaten. Someone from 5,000 BC would be trying to grow food, hoping it rains, and hoping disease didn't wipe out the village. The second one increases the complexity from all the systems required to manage people and keep the land growing. Today the vast majority of people on earth don't grow their own food at all, and instead are busy in some way managing the complexity of a large society.
Someone from 1970-80 would think our software from pre-llm days was vastly more complex. They'd just code directly to the hardware with no abstraction layer. Now almost no one does that. We abstracted the hardware away in most cases. With cryptography libraries for the vast majority of people it's complexity is abstracted away and mostly people are told "don't try to write your own crypto because you will fuck it up".
The question now becomes, how quickly will LLMs be able to coordinate their understanding of the system they are changing?
https://en.wikipedia.org/wiki/VM_(operating_system)
I think the next time I see "LLMs" and "Understanding" in the same sentence, I am going to lose it....
Then I think you should check in with your favorite mental health provider before you become a danger to yourself or others.
Simply put LLMs do understand some things within their crystalized intelligence. Your anthropocentric mind may not accept this, but one day it will. As LLMs have a very short context window in relation to their stored knowledge they have very limited plastic intelligence to change their minds or adapt. All of which is flushed away at the end of a session. It would be like living without the ability to turn your short term memory into new long term memories.
I would gladly use another word for what LLMs can do, but the world at large has not adopted any. The definitions we use around intelligence, comprehension, understanding, consciousness, and sapitence have already been failing us for some time before LLMs as our scientific understanding of biology has increased over the decades as it is. I am one for more exacting definitions when they exist, but humans seem to barely understand the inner workings of our own minds, in large such words escape us.
An LLM has zero understanding of "my", "want", or "cookie" because an LLM has no id/ego, has never felt desire, and has never eaten a cookie.
HN would commonly recommend reading the book Blindsight here.
Moreso, all you've done is recreate the Searle Chinese Room thought experiment which gets bounced around with no means of deciding if it reflects reality or not.
How'd your toddler do at IMO last year?
The hierarchy is certainly higher in agriculture societies it seems, but the complexity is up for some debate
I think introducing AI to deal with this is overall a mistake though. We're just adding more complexity on top of the existing complexity. At best, it's a massive waste of hardware. At worst, we'll probably have agents introducing as many bugs as they fix as they also drown in complexity, and a lot of stuff built using these techniques are going to be fragile garbage while the overall skillset of humanity diminishes because people aren't learning the skills anymore.
Fundamentally, software does not need to be this complicated and it's a solvable problem, but it does require people that care about craftsmanship.
Drowning in complexity. Paralysis of choice.
I read a comment (joke) that if you want to follow all LLM development you should have to be unemployed.
Catch-22 is it's still important to know the fundamentals so you know what to ask for, but if you don't know the esoterica, the model is eventually going to make an assumption and screw things up. And the models don't have much taste either in prose, or in coding/comment style.
Therefore it is critical that whatever AI produces is understandable to us humans. That is why we must demand that AI tools and agents produce "well designed" well-structured software. That's the bottlenexk to progress I think. Even AI can't deal with exponential complexity explosion.
It's not really news, though. Programming as Theory Building (Peter Naur) was published in the 80s, I think?
Maybe the younger entrants to this field never came across it, but even if you never came across it, it was common knowledge amongst experienced devs that understanding of the system you are about to change is crucial.
Thanks for mentioning Peter Naur’s Programming as Theory Building (1985).
I would add Fred Brooks and his The Mythical Man-Month.
The news is that Agentic Programming has made this always challenging task even more challenging.
It’s global madness fired up by continuous stream of news from LLM providers. It’s like The Verge that is almost about FAANG only, but multiplied as it is “magic” for most people, as vibe coding is “so easy” and dopamine-producing activity that it is similar to runners that don’t want to stop as it stimulates them (and in this case ofc it is healthy :)
I feel like that gives an even more literal tower-rising metaphor, and that's what it feels like people using agents naively (and software engineers of lower skill or earlier-career), end up violating.
Agents are getting better at folding things into themselves, especially if you direct them to... but unfortunately I've found that the architectural instincts, even of Fable and 5.6 Sol, are still wildly behind what I reflexively achieve, say.
For sure there is an ability to have agents go back over work and try to fold it into better and better abstractions until it's sort of annealed into something good. I've done something similar on codebases that I have, but the 'high reaches' of architecture with great _prediction of how the software will evolve in the future_ in _subtle_ ways – those are, for now, out of reach of agents.
There is a part of me that wonders if it's partly just how much they can hold in their head right now, though. Even with the greatest articulation and high density of feeding them, the current setups don't allow them to hold a high-quality, sparse, 'zoomable' model of the world in their head that well yet, which we can do pretty well.
But the fact that I'm talking about it in terms of that kind of subtlety is itself promising, I guess?
In other words, if you can’t design a modular monolith, you can’t design a set of microservices.
I somewhat disagree:
Yes, it is possible to design a modular monolith, but thinking about the system in terms of a "minimum viable service" (but keep an eye on "viable", otherwise you can easily get into the "interaction problem") makes it much easier.
This is very similar to how you can write programs with no implicit state in a imperative programming language, but doing this in a pure functional programming language such as Haskell is much easier.
The codebases using technologies I have no idea about tend to quickly become unmaintainable and buggy, because the LLM still doesn't make good architectural choices, but the codebases that use technologies I'm familiar with basically never devolve into unmaintainability.
The difference between the two is massive, and that's why I think that a competent engineer steering an LLM in their area of expertise gets two orders of magnitude more productive, whereas someone steering an LLM in an area they know nothing about are basically producing tech debt at the speed of thought.
Shipping 100x more features per day?
I've written up my process here:
https://www.stavros.io/posts/how-i-write-software-with-llms/
The biggest thing to get right is to let the LLMs do what they're great at (code implementation from very detailed specs, and code review), and you do what humans are great (architecture and making sure the high level of the implementation is sane). That way, you get the best of both worlds, and a lot of speed at high quality.
https://www.writelucid.cc/
No doubt AI boosters will claim we have found an O(1) productivity enhancer that is somehow two orders of magnitude better than anything else before. This is entering lunacy territory.
For example, I built this over a month or so and never hit a slowdown or quality issue:
https://github.com/skorokithakis/stavrobot
Sorry, the lines have to clear what? Surely there must be some kind of constraint on "lines" that they have to overcome.
In code the thing has to become stable, can't just keep packing more and more noise onto it.
I assume one can't benchmaxx multi-year long efforts, clean architecture, taste etc as easily as these "make tests pass" tasks
Although I suspect models from Google, Facebook and Microsoft can be trained on their massive internal codebases. Whether they are is another question.
But, you would probably see a difference of scale and architecture. Larger projects that need better organization are probably more likely to be in private codebases (Linux excluded). So you might be right about the lack of private code in LLM being an issue.
At least in the past (before LLM-based code contributions got socially acceptable in some circles), in open-source project you often got very direct comments on code that was of bad quality. Yes, this was always a little bit abrasive, but it did a lot for the code quality.
For internal applications used at companies, such an abrasive behaviour is typically not accepted ("not a team player" (as if this is something bad), offended snowflakes, "not socially adept" etc.). Thus the code quality suffers quite a lot for internal applications.
Have you ever thought why salespeople have such an easy time selling some LLM for coding to big companies? Because the code quality of many internal applications is so bad, which makes even shitty LLM slop code often better than what is there.
I am pretty sure rate of improvement will be there quite long in the future — even if it will be smoke and mirrors ;)
We do have a mechanism similar to LLMs but it provides existing knowledge; it's a search mechanism, basically. New knowledge happens when we turn that mechanism off.
Maybe not exactly the same thing, but I think one related representation of this in any good codebase is documentations in the code: Good documentation always focuses on the why and not the what. i.e. things that are not represented by or representable in code. It may describe not only why a certain choice was made, but also why not something else. In general, the whole point of these documentation is to account for things that are beyond what can be understood from just reading the codebase including other existing documentation, that you may never even imagine with all the tokens in the world otherwise, which were yet a critical part of our coding choices.
If we are to believe that LLMs can build complex systems, then it's equivalent to saying that LLMs can make similar decisions equally well without any of that... which also then implies that we essentially never needed these documentation to begin with other than as redundancy for human needs.
My personal experiments show that giving them tools to access to all past sessions over a codebase helps a lot. When I coded "feature X" 2 months ago i likely specifically mentioned some constraints expressed as abstractions and, if the coding agent checks not only the code/feature it needs to implement/change but also the past sessions over that, it picks them up, and ships code that better fits the overall project.
At least, more than "architectural/design guidelines", since they are more "concrete", to the point for the task at hand.
Sessions self-preserve them for following sessions, which helps, but might also carry over stale things, so some "pruning" helps, and can be automated. Overall, as long as corrections were also made via agents and hence in sessions, they are picked up automatically.
It's not the "human zoom" you refer to, which as humans we can drive/control, but the effect seems to be similar: I read autonomous sessions where it picked up from the past the very design/architecture/abstraction points i would have driven, had I been in the loop.
In team contexts it should not be impossible to share sessions, but I not working in teams right now :D So maybe it's also a "single dev quirk".
Another point is that in a sense, learning to code goes from being told "you are using the_wrong_abstraction/this_abstraction_the_wrong_way/no_abstraction_where_you_should_have", to telling it to others/ourselves.
Since the number of knowable abstractions seems to be "at least one more than I already know", agents can actually be helpful in learning.
After a certain threshold, it basically zeros in a specific domain, but on novel domains agents taught me abstractions, when nudged towards doing that.
Great metaphor. When I hear people claim 20x productivity with AI assistance, I imagine a Tetris game where pieces fall 20x faster.
As you say, those lines still have to clear.
For example, yesterday I came across some unit tests that didn't have error messages in their assertions. Normally, it takes me ~10 minutes to fix a handful of tests in this situation. In this case, I gave a 2-3 sentence prompt, went to the bathroom, and reviewed the result after I washed my hands. Saved me a bunch of time!
I encourage you to accept a feeling of "imposter syndrome" when using it, and keep trying new things with it. Don't feel like you have to be hands off, except when you're confident that you can be. (IE, if you think you need to spend 30+ minutes on mindless refactoring, see if you can explain it to an agent and then look at HN while it runs. You might get a good result, otherwise, it probably was time for a break anyway.)
BTW: It's important to try different models. The Claude 5.0 models are slow and give me bad results, so I'm sticking with 4.x for now.
The hard part is what text you feed it and how to judge the output.
I finally learned to let go of the code. I dont even run my C++ editor anymore.
I run frequent code and architectural reviews. Its awesome.
So as someone managing LLMs you need to put those processes in place too. The risk is that one tries to do too much and loses overview / insight. Focusing on getting the tower tall instead of sturdy, if you will.
Where the "tower" was once a company (or team?) of human devs, it can now be a single dev and their agents.
The right engineer can likely replace non-technical co-founders with a couple LLMs. Geez, I can't wait to write that article...
I feel these systems rising and sprawling with wee myopic agents developing out their little corners of this unknowably vast whole… a tower with 50 parapets on one side and some wacky cantilevered maiden tower on the other, and a very serviceable adobe roof over some patio for god-knows-why, and thatch over the landing next to it…
Some grotesque fatberg of designs that make sense at the level of individual design efforts, but that lack the fractal sort of levels of policy and judgment that unify the overall enterprise.
The overall language, as it were.
And language takes discipline to establish and maintain through any sufficiently large group of people—witness the company-speak or army-speak of pretty much any successful organization.
We feel like we’ve conquered the problem of talking the same language as our “Gastown Mayors” (who in turn are talking the same language as their “polecats” and so on all the way down the chain of golems)… but it’s only when it’s all built that the good Lord will humble us… that we’ll realize the understanding we thought we’d transmitted perfectly from our thrones wasn’t quite so shared as we’d imagined.
I'm not sure reading code is coming back. The ritual of reading code must come back, because that's the only way to build products that don't collapse under their own incoherence, both technically and visibly.
"just ask Claude" is fine, but it's not the end state
"we can, so we should".
It ended badly.
It is not clear at all to me that other languages "demand much more effort" for the same end result.
It is clear that many non-lisp programmers value syntax, and many lisp programmers don't. Even many people who programmed enough lisp to have their minds blown and expanded still prefer not to program in lisp. I'm still awaiting psychological studies on this, but the rift is so large, I think there may be some significantly different brain processing going on between the two groups.
To your point, yes, it is also clear that, to the extent that lisp can match the productivity of other languages, whether it exceeds them or not, one of the tools that is needed to achieve this productivity boost in lisp is heavy usage of homoiconicity, and this results in every serious lisp program being a collection of DSLs, each of which is only understood by one person or very few people.
For me, the answer to this is economics. Even if you love lisp, there are way more companies hiring for stacks that don’t include lisp
If you then optimize for employability, which I assume most developers do (not all, but a large percentage), you might end up with not that many people practicing lisp regularly
What made Lisp cool and powerful goes away when you do it through an LLM.
Some have a harder time with the transition than others.
I’d look him up.
Padmé: "For the better, right?"
Anakin: (gazes in silence)
Padmé: "For the better, right?"
It seems to me that LLMs and particularly chatbots have already allowed for bigger scale collaboration within the LLM companies versus what was possible within the prior cohort of big platform companies.
Has the result just been taller towers, or actually a change of what is possible?
Why being one (I see as collaborative) was it not desired? Interpretations? Why is it seemed *more* harmful rather than good?
https://www.vatican.va/content/leo-xiv/en/encyclicals/docume...
It's odd that TFA doesn't acknowledge this.
1. Perhaps with a handful of skyscrapers sprinkled in.
I don't know whether the author thinks this is a good or a bad thing, but in my eyes it's clearly a bad thing. Intelligence is knowing that a tomato is a fruit, wisdom is knowing not to put it in a fruit salad. AI is the the ultimate form of intelligence with zero wisdom. Actually, it's not even intelligence, it's an illusion of intelligence. If there is no human who can understand what the AI is doing it's time to stop and accept that we do not have the wisdom to contain what we are building.
Intelligence is learning to avoid using childish cliches, unless your intention was to mislead. Categorisation and understanding dependencies are hard enough problems already.
At the supermarket or taxation (Nix v. Hedden) tomatoes are a vegetable.
Interestingly, the Bible does not specify what kind of fruit was on the Tree of Knowledge. The association between the forbidden fruit and an apple was a derivative of later translations.
Imagine the consternation if the claim had become Adam and Eve ate of the forbidden tomato. Then this incessant controversy regarding the tomato's status would take on a biblical dimension... though then perhaps the Mormon claim that the Garden of Eden is to be found in the New World would actually have some credibility. Indeed, answers to some questions are not a matter of intelligence at all. Sometimes it seems to me intelligence is more about knowing what questions to ask in the first place.
I like that the author lets the image do the work, rather than preach at me even if I were to agree. History never repeats itself but it always rhymes.
https://thebullshitmachines.com/
What LLMs lack, from my experience, is two things: First, a long term memory (even a zoomed out one) and Second, the ability to execute a multi-step task without fizzling out. Upon adversity, LLMs breakdown and start making shit up. The newer models are better (and that's the difference between Fable/5.6 and GLM 5.2) but still have a very limited ability to execute any tedious planning tasks.