Ask HN: How to deal with AI generated sloppy code
I run a successful tech business and I also consult others who want to start their own businesses.
Now this is a problem I'm seeing other tech CEOs start doing which is use AI to have the code written for them by either using some sort of editor plugin or by using something like Cursor.
The problem is that the code that is generated is usually humongous. There is just huge amounts of types and indirection and functions, calling functions and doing all sorts of nonsense which can be done manually, much simpler with fewer lines of code.
This is creating huge amounts of AI-generated slop.
Now when I come in to consult some of these tech architectures it takes me a really long time to figure out how to improve things because there is so much of indirection in the code base and the more troubling thing is that there are errors hidden in the architecture which earlier I would easily find out, but now it requires me to go through every single line of code.
In some of the worst cases, there have been bugs which have taken down their system and then they come and blame me for not being able to find it.
This is the same problem that I had with Java shops. A lot of Java programmers immediately start using a lot of classes and objects because they had for a very long time superior tooling and IDEs.
My theory is that Java is actually a pretty reasonable good language but because there is such easy tooling and easy autocomplete you can almost always increase the huge web of classes and objects because you can immediately reach for them by pressing a dot.
Now take that and increase it like 10x with all of this AI generated code. The medium is what influences what the code generated or written is.
So how are you all handling this problem? Do you find this to be a big problem in the code bases that you see?
It's also hard to tell them not to use AI because the code does work. I would say even most of the times the code does work.
But it's just written in the worst possible manner and maintaining it long term is going to be so much harder if instead they had just handwritten the code.
67 comments
[ 5.4 ms ] story [ 123 ms ] threadIncreasing my per-hour rate until I'm comfortable spending hundreds of hours unspaghettifying code.
If they are still looking to have humans maintain the code, and hire people to optimize it, the at use case needs to be taken into account. From that perspective, I would argue that it doesn’t work.
You could/should charge a higher rate for companies that have a bigger mess to clean up. A cleaning crew is going to charge more to clean up a hoard than to do some light cleaning and re-organizing. The price reflects the effort involved, and I don’t see this as any different. They can choose to pay to write clean code up front, or pay for it later with difficult or impossible maintenance. The choice is theirs.
Personally, everyone on my team hates AI and it’s rarely used. I probably use it the most, which is maybe 4 or 5 times a month to write a single line here or there, mostly to save me a search for syntax.
(When I tried with Zig it was a disaster..)
I’d argue that relying on a tool like cursor to bang out whole routines and types, just using it as “advanced intellisense”, takes away from you truly understanding the solution you’re putting together. It’s fantastic if they make you more productive in the short term, but in the long term, if the generated code is convoluted, difficult to revisit, that’s where you (or a future maintainer like OP) are going to kill time.
Essentially, you need to stringent the code review and testing processes. You cannot have AI doing code generation and code review and/or testing together. Either you have code generated by AI, and have Humans review and test the code, or have humans write the code and AI supporting with code review and/or testing the code. Never both.
I find the second approach better - AI supporting with testing and code review processes. Code generation is still Developer's domain and I don't see AI writing Production level code anytime soon. Developers can use AI to generate snippets of code which is then adjusted to make it production ready. That is probably best of both worlds at the moment.
Building software is like building a house.
If you are given a house with a bad foundation, and need to build on it, literally your only option for improvement is to see if anything can be salvaged, and then you start over.. ideally not making the same mistakes the next time.
Stick to the scams and whatever else, or go work at a PE firm.
Source: my experience working with such types.
If you're working on a Java project, consider prompting the AI to first write a "pseudocode solution" in a more concise/low boilerplate/"highly expressive" language — Ruby, for example — and then asking it to translate its own "pseudocode" into Java.
(Mind you, I'm not sure if you can modify the implicit system prompt used by automatic coding-assistance systems like Cursor. [Can you? Anyone know?] I'm more just thinking about how you'd do this if you were treating a ChatGPT-like chatbot as if it were StackOverflow — which is personally where I find most of the value in LLM-assisted coding.)
Alternately, consider prompting the AI to "act like a senior $lang engineer forced to write in Java for this project, who will attempt to retain their existing opinionated coding style they learned from decades of experience in $lang, in the Java code they write" — where $lang is either, once again, a more expressive language; or, more interestingly, a language with a community that skews away from junior engineers (i.e. a language that is rarely anyone's first programming language) and toward high-quality, well-engineered systems code rather than slop CRUD code. For example, Rust, or Erlang/Elixir.
(Funny enough, this is the exact same mental through-line that would lead a company to wanting to hire people with knowledge of these specific languages.)
As far as I could tell the OP was not saying, at all, that the LLM slop was written in Java.
I used to work at a Java shop a few years ago. Hated my time there since a. I was coming from a more pragmatic way of writing software in C and Java and its ecosystem made me pull my hair out and b. came across several very senior engineers who insisted on writing code with factories and builders and obscure design patterns which made no sense to me. They were all straight from praying at the altar of Design Patterns and always found a way to give comments about a future scenario where this would be refactored.
I was fairly convinced that design patterns in OOP languages were conceived by relatively senior folks as a form of gate keeping.
I wish C was better. A company I have to interact with at $work managed to involve 10 different processes just to change wifi network name (it all boils down to a single netlink message to the driver). And eventually there were too many daemons so they replaced the entire thing with a new architecture where everything ran in one process but with many threads, with no regard for thread safety and blindly copy pasting hostapd code all over the place.
Everything is passed and parsed as a string, so you get none of the advantages of C (speed, simplicity) and all of the drawbacks (ABI breaking, etc.). To make matters worse, these people apparently haven't heard of databases or even plain text files. Data is stored in XML or JSON "databases", but of course having a full XML parser is too expensive so it's actually their own homemade XML parser which requires line breaks in certain places and literally does strchr('<').
The funniest thing was when they replaced dbus with their proprietary data bus (with zero debugging capabilities of course) because dbus was "too slow", meanwhile at the same time their "hashmap" is actually a linked list (!!!) because they thought it would be premature optimization to write a proper map type. All written in C and yet somehow slower than python.
I'm really looking forward to the stone age of programming, because we're definitely not there yet.
In a large-enough Java codebase, with good software engineering being done, you will eventually hit the problems for which these weird kinds of indirection are the solution (because Java is dumb and forces them) — but it tends to take a while, and not involve concerns that are the first priority in a greenfield project. I'll just address the factory-pattern here, but the builder-pattern has a similar story.
The Java factory-uber-alles mode of thought (i.e. "make all your Java classes factories; and all your static methods, including constructors, into instance methods on those factories") exists because doing any kind of sum-typing on Class objects in Java is painful.
Java has interfaces. But Java has no concept of type-level interfaces.
In theory, Java could have:
1. extended the definition of interfaces, to validate that certain static methods and constructor signatures exist within the Class objects of implementing classes;
2. and then introduced something like a ClassWithInterface<T> type, that a Class object could be cast to if-and-only-if that class (or one of its ancestors) implements that interface.
But Java never did this.
And even if it did, by definition, a Java interface must still be known to the class defining it, and the membership of the class in the interface declared explicitly by the implementing class.
Which means it's a bit too late to make this change now, for it to ever have any kind of useful impact — there's so much existing Java code that can't be changed [old stable JDK versions everyone depends on; third-party libraries that are basically abandonware; etc], which would therefore not have defined membership in whichever later-defined type-level interfaces these types really should have type-level membership in.
And even if this were part of Java from the beginning, you'd still have the situation where you want to "make" a third-party class part of a type that the maintainer of that class doesn't think it should be a member of. (Maybe because that type-level interface is a private one you're defining within your app!)
With type-level interfaces, you could put a Class object into a type-level-interface-typed variable, and then use that variable to invoke the static methods of that class, construct instances of the class, etc.
But because you don't have type-level interfaces, to interact with a Class object "generically", you instead have to resort to using the java.lang.reflect package — which, depending on the method, either 1. accepts and returns plain Objects (resulting in extremely error-prone code); or 2. requires that you gradually find the right method handle (and catch five possible exceptions); pack the parameters for calling it (and catch five more possible exceptions); call it (and catch five possible exceptions besides the ones the typing yourself); do any return-type coercion yourself (three more exceptions)... and where all of those exceptions are extremely opaque and obscure the "actual" error (e.g. the caller passing an argument of the wrong type.)
Which means that any time you might want to take code that calls a static method on a Class, and instead make that code call a static method on the value of a Class-typed variable... you're now in for a world of hurt. Dependency injection? Mocks in unit tests? Strategy/adapter patterns? Plugins systems? All become the hairiest Java code in the world.
...until eventually you get fed up, and realize that you can have type-level interfaces today, by just taking the statics and constructors on these classes (that you may or may not own); creating equivalent wrapper instance methods on little proxy classes you do own; and then make all those little proxy classes implement an interface. ...
There's also an implicit bias towards blathering in LLM output, especially in open-ended scenarios like code completion. Language models prefer high-probability outputs; if it's appropriate for a model to generate highly stereotypical content like boilerplate getters/setters and documentation comments, it's more likely to generate that than something more application-specific like the actual implementation of an algorithm.
It won't be dissimilar to being a consultant now. I've spent years consulting where my job was to tame legacy code (that is, code which is not under test and nobody knows how it works). The value in roles like this is that you work at a level higher than the code in order to get the team able to understand the system and be able to start extending it again.
It sometimes "works on their machine". As you aptly pointed out - it often has serious design issues.
> So how are you all handling this problem?
Do nothing. The problem will resolve itself. Eventually tech debt will crush them. Probably soon.
Bold.
The ideas underlying the implementation are not necessarily tied to the implementation given the skill gap.
You might as well get good at this now, because the future is AI generated code and AI assisted code maintenance. And by the future, I mean the present.
If it saves you time to fix it, it's still a win. If you find yourself spending more time debugging, just write it by hand. There's a lot of investors that want to "make AI happen" but the truth is more nuanced. If it works, use it. LLMs aren't good or bad, they are just another tool.
It was an uphill battle, and honestly, for that time, I just switched to the editors they were using. Otherwise, I'd just be seen as being less productive.
For my own personal projects, I could spend more time just keeping things simple, and using vim. Though recently, I've switched to VScode based Cursor. I've found that it works best when I use it as a guide, rather than as a code writer. I'm often working on an atypical stack, and the code it generates just uses a wrong or outdated API. Even cursor is terrible at pulling in the right context and documentation. The mental habit of thinking it'll "just solve the problem" is wrong too, and wastes my time. It's much better if I think of it as a guide, and I ask it to explain everything I need to do step by step, so I'm following along, rather than expecting it to one-shot it all for me.
So, in some sense, you'll probably need to go with the flow. Using some sourcegraph or other tooling to help you navigate will help in the short term.
But I think in this sense, I agree with the handmade people--that keeping things short, simple, and fast will be a business advantage over time. But that's given that you've validated the need. And find companies that understand that AI won't be able to one-shot tasks (beyond very typical junior programming tasks) for a while to come.
That’s why the 2000s outsourcing wave didn’t really impact the market for software engineers in the US despite the apocalyptic rhetoric at the time. If Company A has a productive local dev team, their competitor B has to either hire their own or hire a bigger outsourced team to stay competitive. Then company A has to hire even more to meet exec’s growth targets and it becomes a competitive spiral unless economic conditions change (like recently with elimination of ZIRP).
I think in the near future, LLMs will put that arms race on overdrive and all of us will have to adapt. They’re here to stay, for better or worse.
I handle it by using AI tools for understanding codebases and code reviews. Properly set up, LLMs can even give decent refactoring advice - i.e. by feeding it a callgraph - but the UX isn’t there yet. I use cli apps and custom scripts to do a lot of codebase work with LLMs like claude-sync and aider.
I’d start by trying out aider/Cursor to see whether they can help you manage the code. Use aider’s repomap or Cursor’s repo indexing features and ask the LLM questions about the codebase. Use the Cursor code review feature, which has caught a lot of subtle bugs for me (I ignored the feature for my first month and was pleasantly surprised when I first used it in lieu of debugging a PR). Experiment with asking the LLM to refactor the codebase since now their token windows are large enough to fit ten thousand plus lines. Maybe try Github Copilot Chat if you can get beta access, since they do codebase indexing too.
My advice boils down to “fight fire with fire”. In the hands of competent engineers, LLMs are a lot more powerful than the slop you’re seeing. (For context I’m working at the intersection of C++/Rust/Qt for which training data is basically nonexistent)
I think this isn't particularly true, in general. I'll give an example of how this assumption can bite an early stage startup rather badly.
Company X rapidly develops some idea, raises seed round, hires some devs. At this point they have no customers or traffic so infrastructure costs are low and software development costs dominate. Moreover, they have $100k of free Amazon bucks to play with so they're not really thinking about infrastructure costs. Then they get a customer or two, and all of a sudden infrastructure costs are huge and the small dev team has to reorient in a panic towards figuring out how to make their thing efficient--basically an unplanned total rewrite. The opportunity cost of that could actually sink the company or force them to raise a round on bad terms.
In the public cloud, there are many ways you can very quickly rack up a huge bill if you don't do things efficiently. Some examples:
1. Let's just use AWS Lambda for everything!
2. Let's use AWS managed kuberneetus!
3. Let's write a bunch of small files to S3!
4. Let's not think too hard about what our data looks like on the wire, and just ignore data transfer fees!
These costs may look small at first, but they probably scale more rapidly than your revenue does. There's absolutely no substitute for just doing it consistently right, from the very beginning, and not deviating.
Just because companies are bad at cost management and bloat their stacks in clouds doesn't mean that the marginal cost is high, just that their absolute budget is. The marginal cost of compute and the marginal cost of each new customer using it are different issues.
So a system which automatically generates a bunch of inefficient, poorly wrought crap quickly for low effort will make you pay for it in different ways. So it's not obviously advantageous, and it seems (to me at least) the disadvantages are greater.
Low marginal cost is built into the cloud products like Lambda, S3, and autoscaling VMs. That's why they're popular, even though the servers running those services are much more expensive.
Edit: I think I see the impedance mismatch here. Yes the marginal cost of spinning up another RDB instance or big VM is relatively high, but unless you’re spinning up one for each customer, the marginal cost per user is much lower. Sometimes the marginal cost is recalculated like when scaling up to more big DB instances, but the cost per unit is very low compared to say a factory that has to buy machines up front and train machinists when they need to grow. If a software company has enough servers already, the marginal cost of a new customer is often just the extra electricity used by existing compute
I agree that for example, serving a web page to users where almost everything happens on the client, the marginal cost of running the software is low. But it's not always true, and it's really important to measure and track these costs otherwise you can end up in a bad way. I've seen it happen ;).
What actual problem are you running into?
If it’s purely that gen AI produces slop, that’s par for the course in consulting with startups. You will find a lot of humans willing to generate slop all on their own. Some will even do it for free. Fundamentally, if your client could figure it out themselves they wouldn’t be paying your hourly so slop represents part of the opportunity space.
LLMs will just create new opportunities in consulting. But the game is fundamentally the same - communicate about what you understand versus what you don’t, ask a lot of questions and underpromise in the beginning.