For me, if I can make a kickass testing system that people love so much that they actually build features with it and it’s not an afterthought, then maintenance becomes much easier. It’s often called test driven development but I’ve rarely seen it done in such a way that the dev ex is good enough for it to work.
But say you have that. Then you have great profiling. At that point you can measure correctness and performance. Then implementation becomes less of a focal point. And that makes it a lot easier to concede coding to ai
I wonder if AI could make code reviews more presentable.
for example, with human code reviews, developers learn quickly
not to visually change code like reflowing code or comments,
changing indent (where the tools can't suppress it), moving
functions around or removing lines or other spurious changes.
And don't refactor code needlessly.
also, could break reviews up into two reviews - functional changes
and cosmetic changes.
Do any refactorings in separate reviews, and say things like "REFACTOR_ONLY:", with a rule that none of the code changes behavior.
That makes reviews a lot easier. The review starts from "nothing should be changing" and then reviewers can pattern match on that.
Otherwise, the reviewer is re-evaluating every line of code to make sure nothing has changed. That's really hard to do properly.
The version control systems I've worked with have allowed queues of changes, each one reviewed independently. As I'm developing, if I need a refactor, I go up a commit, refactor, send out for review, rebase my in progress work and continue.
I send out a continual stream of "CLEANUP:" "REFACTOR_ONLY:", and similar changes with the final change being a lot smaller than a big monster of a change.
Your reviewers will appreciate the effort.
Plays the metric game (if you're working in that type of org) without being evil too.
First Agent I used: Do a proper code review of the changeset, it adds comments in my merge requests. Then the junior devs paste these into their IDEs and loop forever :-P
In my experience AI reduces maintenance costs. Though, context might matter here, I'm working on a multi decade set of projects, while there is a lot of greenfield feature development, the old code / older projects have suddenly become a lot easier to work with, modernize, and in a bunch of cases, eliminated. Dependency on old libraries, build tools, in some cases updated, in other cases just eliminated, builds are faster, easier for developers, etc. End to end testing has become a lot easier to setup and automate. DevOps have been improved a lot, diagnosing production issues drastically improved, we have a ton of logs and information, and while we have various consolidated dashboards / monitoring to capture critical things, now we can do a lot more analysis on our deployed system (~50 ish projects)
This rings true for me too, but I don't think it counts if your just using AI to aid maintenance. The basic argument in the article is around how many hours of maintenance you have to do for each hour of "value-add" feature development. So A. your only measuring maintenance costs not the ratio and B. The "old code" whp wasn't written with AI in the first place.
I agree - AI makes it easier to wrangle legacy code. I think the author's point is that if you lose access to the AI tools, everything becomes more daunting -- because you've been comfortably moving mountains with heavy equipment, and now it's back to hand tools.
I have a very very opposite experience in a large company where everyone shits code all over parts of the codebase they don't understand with AI assistance.
We have had outages increase in tandem with lines of code shipped and outages are getting more and more severe. Yes we have improved much old code, deleted more old code, can automate code modernization, can better diagnose issues, have more options for mitigations, etc.
But all that has not offset the sheer magnitude of code being shipped which no one really understood.
Unfortunately, maintainability is simply bucketed as a "non-functional" requirement.
Maintainability (and similar NFRs) should actually be considered what preserves and enables the delivery of future functional requirements -- in contrast to framing non-functional requirements as simply "how" the software must do what it does vs. the "what"/functional requirements that "actually matter".
From that standpoint, if a steady flow of features/improvements is important for a project, maintainability isn't really a non-functional requirement at all, and amounts to being a functional requirement, in practice, over anything except the shortest of time horizons.
> amounts to being a functional requirement, in practice, over anything except the shortest of time horizons
Right! The unfortunate thing is that many software companies don't seem to think much further than a quarter ahead, not really.
Sure they might have a product roadmap that extends for a year or two into the future, but let's be honest. Often that roadmap is mostly for sales purposes, not engineering planning purposes. Product and engineering will pivot if sales slump. The earlier in the company's lifespan, the more likely this will happen often
However if companies get out of this startup mode then they should start to stabilize... But many don't. They continue this pattern of short sighted short term planning, which means product stability remains a low priority effort.
Ultimately I guess many companies just either do not have the resources to build good software or do not actually care to
I've found the first, and most important, step for any team or organisation to eliminate concerns with NFRs, "tech debt", and whatever else it may be called, is to stop giving it a name.
I'm being completely serious. By giving it some kind of distinct name, you are giving license to it being ring-fenced and de-prioritised by someone who doesn't (but, arguably, probably should) know better.
Quality matters. It hits your P&L very quickly and very hard if you don't maintain it. So it is as important as any other factor.
I treat it like housekeeping and treat features like hosting a party. Guests/stakeholders are people who want what you can make. The party is the feature they want.
They don't care whether it was difficult or easy for you to clean the house. They just assume keep your own house tidy ... and they know you don't when you only host once a quarter instead of once a month.
They assume you're a functional adult who manages his own space.
Tech debt is like that.
Thus - the business folk don't get a say in whether it's in the sprint - cuz it's not "the party". Instead it's your Scrum Master or whatever saying "hey kids - clean the mirrors and Jane this time you're sanitizing the toilet."
The maintenance cost argument cuts both ways. We ran into this building our own project AI moves fast, but the bugs it introduces are weirdly hard to spot. Not the obvious stuff. The logic that looks completely reasonable until three weeks later, when something breaks in production, and you trace it back to a subtlety the AI got wrong. My honest take: AI doesn't reduce maintenance costs, it shifts them. Less time writing, more time reviewing. And reviewing AI code is harder than reviewing human code because it's fluent and confident even when it's wrong. Whether that's a net win depends entirely on how good your team is at reading code vs writing it
The review burden scales with the agent's decision space (which almost nobody is doing anything to limit today) and it's a variable you can control. When an agent has a discrete state, a subset of tools it can work with and constraints on the quality of the result the surface area of what could go wrong shrinks in proportion. The article treats this shift as inherent. It's more like we're all surprised pikachu that we gave the agents access to everything and expected it not to happen ever.
Forget the term “non-functional”. Who wants software that doesn’t function? Use Kevlin Henney’s terms: operational and developmental characteristics. Maintainability is a fundamental developmental issue.
I think AI is great for the soul destroying boring stuff that makes me want to quit my job like wrapping legacy code in test cases. Hey I’ll take on any idiot who’s willing to do that job, even if he’s artificial.
You can only type at 50WPM and read one file at a time, the LLM doesn't have the physical limits, use it at your advantage so you can actually focus on the work that matter
The maintenance-cost framing is the useful constraint. I’d rather see agents default to smaller diffs, test scaffolding, and explicit assumptions than maximize lines changed per prompt.
My low value comment. This feels directionally correct to me. The problems I've been struggling with in my dev job for the past 6 months have been 80% maintenance/legacy code interfering with new feature development.
Some of our developers are overly aggressive about using AI and I've started going down that path because I need to keep up and actually enjoy the flow of working with AI in my IDE.
I put a lot of work into keeping my area of the codebase understandable and coherent but I do not see that from the others on our team. I'm not perfect but I and extremely sensitive to incoherent, or un-grok-able at a glance.
Anyway, I like the novel (to me at least) framing of this article!
So what are all of these agentic based strategies going to do once the infinite money spigot of investment into AI ends and they need to start charging prices that actually make a profit?
I get that most of the cost is in training and not inference, but I don’t see how models stay useful once the worlds software updates in a few months post training since the models can’t learn without said training.
Are we just going to have shops do the equivalent of old COBOL shops where everything is built to one years standards and the main language/framework is mostly set in stone?
This is what I've been preaching to my team. With 5.5 and 4.7 the coding agents are good enough know to almost never take any tech debt. Any new feature or fixes should come with a cleanup or refactor, on the same PR.
Yeah, but to be honest, I sometimes just tell Claude to cleanup / refactor stuff; it finds a lot of things, discusses it with me and I approve the plan, and it churns away my tokens for some time. I do this once in a while, and I've been doing this for over 6 months and I don't feel like my development has significantly slowed down. Yeah my token usage is more for sure, but my codebase also is, so I'm not worried about that. To me AI seems to make maintenance very easy, like the rest. You just need to do it.
Edit: I make it sound a bit simple maybe. I do more extensive redactors also, where I'm more involved and opinionated. But I don't feel the need to do that very often very deeply. But yeah sometimes it's definitely necessary to prevent the project from going off rails.
I feel like AI might let us model some of the things that we initially didn't scope that led to these problems (e.g. "Decided not to fix every bug, or upgrade every dependency") - being able to more easily ask a system that can dig into "how much time are we spending on stuff related to foo"
AI tooling can also be a place where we start building our view of what maintainable software practices look like so we don't make decisions that have these same tail effort profiles. That can be things like building out tooling to handle maintenance updates
I think the real thing that comes out of AI tooling is probably that the tooling needs to be trained (or steered) towards activities that enhance human attention management.
The incitives for remote LLMs are off with providing defaults which optimize for maintenable sound architecture though. Same way Claude is going to produce overview of the indexes of the summaries of comprehensive reports, no one is going to read. No doubt this feels like excellent KPI on how much output was generated.
Great Article! I think ultimately we are heading towards a world where much better software will be created. This is the major roadblock we need to cross over before that can be true, but I think it is a very tractable problem!
I created a video that talks about this in more detail:
With AI, you can hypothesise what can potentially break with each new addition (which your regression tests do not even capture at present). Then, you can write tests for each of those hypotheses, ask AI to deploy a canary, ask AI to divert 5% of traffic to the canary. Ask AI to analyse the logs for any signs of regression in performance, ask AI to roll it out to 100% if everything is good. Congrats! At this point, you've become a slave to AI and cannot do without it. Even logging into a remote server now causes mental pain; having to do anything by hand causes pain. You just wait for your limit to be reset to return to slavery again. A master of a slave is as much of a slave to his salve as the slave is to the master itself.
Would be an interesting concept and read were it grounded in reality. Unfortunately, it's data and graphs pulled out of someone's imagination. Reality is nowadays with the right skillset you can take state of the art AI tools and get a complete language rewrite and or refactor and be done the same afternoon.
but the dirtier truth is that llm-generated code skews the maintenance curve worse than human code because it optimizes for compiles and passes the happy path rather than for the boring stuff that makes future-you's life easier
1. software doesn't only have tech maintenance - there is also user support and it increases as software grows.
2. I'm not convinced maintenance costs scale linearly. And even if it scales linearly, you will eventually get to a point where maintenance takes up all your time.
> Your crowd might tell you that, for each month you spend writing code, you’ll spend... 10 days on maintenance in the first year; and 5 days on maintenance each year after that
Someone is an optimist! I'd estimate those significantly higher, and even worse if you are in a field that has to do any sort of SOC/HIPAA/GDPR audit
The bet that he misses, which a lot of companies are starting to make or at least think about, is that AI will get better at coding. So the model / harness / whatever is next takes care of the maintenance burden.
One thing I like about framing this as maintenance cost is that it moves the measurement boundary. The usual AI coding metric is something like accepted diff per hour, but the more interesting unit is probably future decisions created per hour.
An agent can reduce typing while increasing the number of things nobody really owns later: rationale, invariants, tradeoffs, half-meaningful tests, files that changed because they were nearby, etc. The PR can pass and still leave the team with more intent to rediscover.
The useful agent workflows I keep coming back to are less about "write more code" and more about making every change come with a maintenance handle: what invariant changed, what should fail if this is wrong, what files should not have changed, what rollback looks like. It feels slower in the moment, but it gives future-you something to grab onto.
This could have been a good piece of writing if the author chose not to be so smugly overconfident in their belief and show real evidence to support their claim. Mentioning the front page of HN as your source is glib and immediately made me doubt the conclusions. I was interested to see what work the author put into researching this but apparently they didn’t do any work at all.
When an LLM provides you with an overconfident piece of writing with no sources to back it up, what do you do?
interesting perspective, but i'd throw in some caveats:
- productivity isn't the be-all end-all, it's just one metric and a consultant's mantra. taking a productivity hit can be more than fine if it gives you a tactical/strategic advantage or opportunity.
- i'm not convinced at all that agents will become prohibitively expensive. that's indeed some companies' wet dream but a) good cheap competition is emerging and b) you don't really need the latest models or massive computing power to get shit done.
i do agree though with the emphasis on code quality and debt, and for not blindly going for the silver bullet fad and throwing money at it like there's no tomorrow in the hopes of some "productivity figures boost". then again i doubt that companies going for that would heed such advice, we've been there many times.
71 comments
[ 1.4 ms ] story [ 95.9 ms ] threadBut say you have that. Then you have great profiling. At that point you can measure correctness and performance. Then implementation becomes less of a focal point. And that makes it a lot easier to concede coding to ai
I wonder if AI could make code reviews more presentable.
for example, with human code reviews, developers learn quickly not to visually change code like reflowing code or comments, changing indent (where the tools can't suppress it), moving functions around or removing lines or other spurious changes.
And don't refactor code needlessly.
also, could break reviews up into two reviews - functional changes and cosmetic changes.
That makes reviews a lot easier. The review starts from "nothing should be changing" and then reviewers can pattern match on that.
Otherwise, the reviewer is re-evaluating every line of code to make sure nothing has changed. That's really hard to do properly.
The version control systems I've worked with have allowed queues of changes, each one reviewed independently. As I'm developing, if I need a refactor, I go up a commit, refactor, send out for review, rebase my in progress work and continue.
I send out a continual stream of "CLEANUP:" "REFACTOR_ONLY:", and similar changes with the final change being a lot smaller than a big monster of a change.
Your reviewers will appreciate the effort.
Plays the metric game (if you're working in that type of org) without being evil too.
We have had outages increase in tandem with lines of code shipped and outages are getting more and more severe. Yes we have improved much old code, deleted more old code, can automate code modernization, can better diagnose issues, have more options for mitigations, etc.
But all that has not offset the sheer magnitude of code being shipped which no one really understood.
Unfortunately, maintainability is simply bucketed as a "non-functional" requirement.
Maintainability (and similar NFRs) should actually be considered what preserves and enables the delivery of future functional requirements -- in contrast to framing non-functional requirements as simply "how" the software must do what it does vs. the "what"/functional requirements that "actually matter".
From that standpoint, if a steady flow of features/improvements is important for a project, maintainability isn't really a non-functional requirement at all, and amounts to being a functional requirement, in practice, over anything except the shortest of time horizons.
Right! The unfortunate thing is that many software companies don't seem to think much further than a quarter ahead, not really.
Sure they might have a product roadmap that extends for a year or two into the future, but let's be honest. Often that roadmap is mostly for sales purposes, not engineering planning purposes. Product and engineering will pivot if sales slump. The earlier in the company's lifespan, the more likely this will happen often
However if companies get out of this startup mode then they should start to stabilize... But many don't. They continue this pattern of short sighted short term planning, which means product stability remains a low priority effort.
Ultimately I guess many companies just either do not have the resources to build good software or do not actually care to
I'm being completely serious. By giving it some kind of distinct name, you are giving license to it being ring-fenced and de-prioritised by someone who doesn't (but, arguably, probably should) know better.
Quality matters. It hits your P&L very quickly and very hard if you don't maintain it. So it is as important as any other factor.
I treat it like housekeeping and treat features like hosting a party. Guests/stakeholders are people who want what you can make. The party is the feature they want.
They don't care whether it was difficult or easy for you to clean the house. They just assume keep your own house tidy ... and they know you don't when you only host once a quarter instead of once a month.
They assume you're a functional adult who manages his own space.
Tech debt is like that.
Thus - the business folk don't get a say in whether it's in the sprint - cuz it's not "the party". Instead it's your Scrum Master or whatever saying "hey kids - clean the mirrors and Jane this time you're sanitizing the toilet."
Some of our developers are overly aggressive about using AI and I've started going down that path because I need to keep up and actually enjoy the flow of working with AI in my IDE.
I put a lot of work into keeping my area of the codebase understandable and coherent but I do not see that from the others on our team. I'm not perfect but I and extremely sensitive to incoherent, or un-grok-able at a glance.
Anyway, I like the novel (to me at least) framing of this article!
I get that most of the cost is in training and not inference, but I don’t see how models stay useful once the worlds software updates in a few months post training since the models can’t learn without said training.
Are we just going to have shops do the equivalent of old COBOL shops where everything is built to one years standards and the main language/framework is mostly set in stone?
Edit: I make it sound a bit simple maybe. I do more extensive redactors also, where I'm more involved and opinionated. But I don't feel the need to do that very often very deeply. But yeah sometimes it's definitely necessary to prevent the project from going off rails.
AI tooling can also be a place where we start building our view of what maintainable software practices look like so we don't make decisions that have these same tail effort profiles. That can be things like building out tooling to handle maintenance updates
I think the real thing that comes out of AI tooling is probably that the tooling needs to be trained (or steered) towards activities that enhance human attention management.
The incitives for remote LLMs are off with providing defaults which optimize for maintenable sound architecture though. Same way Claude is going to produce overview of the indexes of the summaries of comprehensive reports, no one is going to read. No doubt this feels like excellent KPI on how much output was generated.
I created a video that talks about this in more detail:
https://www.youtube.com/watch?v=G3Q7Y-nrUbk
1. software doesn't only have tech maintenance - there is also user support and it increases as software grows.
2. I'm not convinced maintenance costs scale linearly. And even if it scales linearly, you will eventually get to a point where maintenance takes up all your time.
Someone is an optimist! I'd estimate those significantly higher, and even worse if you are in a field that has to do any sort of SOC/HIPAA/GDPR audit
That's the theory anyway.
An agent can reduce typing while increasing the number of things nobody really owns later: rationale, invariants, tradeoffs, half-meaningful tests, files that changed because they were nearby, etc. The PR can pass and still leave the team with more intent to rediscover.
The useful agent workflows I keep coming back to are less about "write more code" and more about making every change come with a maintenance handle: what invariant changed, what should fail if this is wrong, what files should not have changed, what rollback looks like. It feels slower in the moment, but it gives future-you something to grab onto.
When an LLM provides you with an overconfident piece of writing with no sources to back it up, what do you do?
- productivity isn't the be-all end-all, it's just one metric and a consultant's mantra. taking a productivity hit can be more than fine if it gives you a tactical/strategic advantage or opportunity.
- i'm not convinced at all that agents will become prohibitively expensive. that's indeed some companies' wet dream but a) good cheap competition is emerging and b) you don't really need the latest models or massive computing power to get shit done.
i do agree though with the emphasis on code quality and debt, and for not blindly going for the silver bullet fad and throwing money at it like there's no tomorrow in the hopes of some "productivity figures boost". then again i doubt that companies going for that would heed such advice, we've been there many times.