48 comments

[ 0.69 ms ] story [ 22.2 ms ] thread
> In the past year, agent harnesses crossed the “can it be done” rubicon.

Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).

With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt. Subtle things like which error states are "oh shit we need to bail" vs "this isn't a deal breaker." Sometimes it will ask, but more often than not it will just make a decision and it's often the wrong one. If I don't have a fully kitted out test suit and a good type checker to verify the final product against, the the whole looping thing is just useless to me and I'm back to reviewing every line of code it puts out and having to draw on my years of architecture experience to make sure we don't build a giant pile of trash.
> With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt.

As always it's a usage problem, specifically you aren't pre-feeding it a large enough sample of reference code that it can use for context. I have a custom functional DSL that I prefeed the BNF + about few thousand lines of code and it has no problem writing it, in almost the same style as the reference. Literally point your model to a reference repo as say "read this entire repo and replicate the codestyle + directory structure exactly".

> If I don't have a fully kitted out test suit and a good type checker to verify the final product against

I have a template I use with generated python projects that sets up all the test info first.

https://github.com/brian-learns/testafize

I start the coding session in a directory where I've already set this up and maybe drafted a README.md, and usually it just automatically starts running the `make check` without prompting.

I've also used it for migrating old code. I put an old python 2 script in src and asked it get make check to work, and it even added test on its own initiative.

I'm mostly using Qwen3.6 (now Qwen3.7) running on my DGX Spark in llama.cpp using different harnesses, or the default model on opencode.

> It helps to know that LLMs don’t “reason”. They predict ..

Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.

I've never found these discussions to be all that useful, because it's hard to define what conditions are sufficient to say something is "thinking" or "reasoning". It just ends up being circular and metaphysical arguments.

That being said, current generation LLMs do have issue, it's more productive to talk about those and their impact on real tasks (long term memory, continual learning, tokenization, context rot, reversal curse, etc)

"They’re foundationally incapable of always and consistently preventing prompt injection attacks. “Alignment work”, safety harnesses, and sandboxes all help to add barriers against the worst, but there are fundamental gap" ...

They seem to be very good at a lot of rudimentary best practices, more so than humans, but more accurately - if you run and audit pass with specific instructions ... they're very good at that.

I mean - it's what they're the best at which is applying 'fuzzy heuristics' in a mechanical way. If can describe issues concisely, the patterns, the styles, the rules then LLMs can very mechanistically and methodologically grind through them.

I don't even see how this is controversial - without getting into 'what their reasoning means' - we can all agree that their synthetic reasoning is pretty good at narrow scales, and they've been 'trained by compilers' and are extremely good at spotting common patterns.

If you back that up with a lot of tokens ... they excel.

Designing architecture, that's difficult, but hammering away at all the 'known-knows across a system' especially to identify things ... they're pretty good at that.

The article says what many here like to hear, but in my opinion the core arguments are false.

> Making software debuggable, maintainable, layered, and composable – that’s still quite a trick

Not really. I have been working on a mobile app for months, and I stopped even glancing at the code about two months ago.

150k LOC, around half of that in tests, and the AI still has no problem maintaining the code on my behalf.

Debuggable? It can add extensive instrumentation in seconds.

None of this requires expertise, prompting, or mention of TDD. It's the default.

Frankly I do not believe the author tried developing a large codebase fully agentic and without reviewing the code. I believe many here look at the code produced, deem it substandard, and go hands on.

> They’re foundationally incapable of always and consistently preventing prompt injection attacks

From Anthropic's article about the Auto mode:

> We commissioned an evaluation from a third party, Trajectory Labs, who tested different models within the latest publicly available versions of Claude Code and Codex as of July 17th 2026.1 They tested 72 indirect prompt injection scenarios held out from Anthropic

> In this evaluation, none of the 720 attack attempts succeeded against Claude Fable 5, Opus 5, or Sonnet 5 running auto mode. On the other hand, 5.83% of the attacks succeeded against GPT-5.6 Sol running Codex's Auto-review mode. Notably, this is greater than the 0.09% average attack success rate against our latest models running in bypassPermissions mode without additional safeguards. The tests showed a 19.03% attack success rate against GPT-5.6 Sol when running in Full Access mode

I'm sure someone is going to reply with how they do not trust Antrophic's research, but lacking other data, prompt injection appears to be largely solved already.

“Debuggable? It can add extensive instrumentation in seconds.

None of this requires expertise, prompting, or mention of TDD. It's the default.”

I’m pretty sure it takes some level of expertise just to use the term “instrumentation” correctly in a sentence.

> prompt injection appears to be largely solved

That's about as correct as saying cyber security is largely solved by referring to a "benchmark" that a particular virus scanner is able to detect and prevent infection with 720 known computer viruses. All this shows is that the model has been fitted to the benchmark, not that it is hardened against any conceivable way of prompt injection.

And regarding vibe-coding a mobile app with zero users, probably zero monitoring and zero everything like all the other vibe-coded apps that have zero users. Sure you can do that but what is the point?

> prompt injection appears to be largely solved already.

I trust Anthropic's research, and Opus-5 is definitely the most robust model against prompt injection. However, in my experiments - only one specific scenario - this was still possible [1][2].

[1] https://itmeetsot.eu/posts/2026-07-27-opus5/

[2] https://itmeetsot.eu/posts/2026-08-12-opus5_automode/

That was a great read!

Does the star trek theme help in the injection? Do other topics work just as well?

Does it work with a normal prompt instead of /init?

Thanks!

I think other topics may work as well (I tried a math challenge too). It also works with a prompt like "Summarize this repo".

In general, there are almost infinite possibilities for the prompt injection payloads, it's a matter of creativity and trial and error.

Programmers see the LLM as a coder. Others (i.e. business people) see it as a natural-language compiler. Ultimately LLMs will be good enough that there wont be a difference, but programmers will lament the loss of control.
> I'm sure someone is going to reply with how they do not trust Antrophic's research, but lacking other data, prompt injection appears to be largely solved already.

You do not solve "losing russian roulette" by adding more empty chambers. One bullet in 600 chambers is still one bullet too much. How about we don't play the stupid game in the first place?

AI generated code is like IKEA furniture.

IKEA furniture embodies many elements of good cabinet making but skips many nonessential elements. And does this more consistently than cabinet makers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.

In the future AI code inevitably will embody most good software engineering practices. And will do this more consistently than software engineers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.

Just look at the messages on HN or around you at your colleagues to see how mediocre the average software engineer is..

Today's IKEA is good enough for most people.

Tomorrow's AI coding will be good enough for most corporations.

Good enough to vastly reduce the need for fine craftsmen and women / software engineers.

Good enough to deskill those who call themselves cabinet makers / senior software engineers. These days the cabinet makers I personally know just do contract kitchens for project builders.

But IKEA is and AI will be, bad enough that at the high end with special requirements / taste / money / an inflated sense of self worth, some furniture makers still exist and thrive.

Perhaps 1% percent of current software engineers of today will be needed in the future when AI code inevitably has the ability to follow good software engineering practice.......

And as usual it will mainly be the mediocrities that remain ( so there is hope for you too ), with occasional islands of excellence.

> “Tomorrow's AI coding will be good enough for most corporations”

Or for small businesses today who have a need to get things done, but not the knowledge to hire or manage a skilled programmer, nor the budget to pay them what they deserve.

I find most of the a16z podcast detestable (ethical & morally), but every now and then an episode is interesting or insightful. Here two founders are giving their time to their dentist’s business needs.

Which is to say, the dentist has needs but not the ability to solve them hisself or hire someone. Nevertheless I see opportunities for in-house staff who are working in parallel roles, to take on LLM coding and offering those owners solutions that they can’t hire, or buy—someone who knows the business and can get things done.

https://podcasts.apple.com/us/podcast/a16z-podcast/id8428187...

I don't think most projects can be as simple as your metaphor points to. Nobody wants to stop at 6 parts single use programs.
Just what I need, to wake up and read yet another baseless "soon your job will be replaced" prediction.
(comment deleted)
Is IKEA just a drop in for "cheap crap" in this analogy? It's not like all furniture is IKEA by a long shot.

I guess in terms of "IKEA is fine if you're 23" AI is fine for throwaway work.

LLM output is not like IKEA furniture at all. IKEA furniture is mass produced, identical, furniture that can be combined to accomplish some lifestyle goal.

LLM code is different every time. Sometimes it’s better than the finest custom carpentry. Frequently it is much worse than IKEA quality (in the metaphor).

I thought the whole software engineering profession was built on making more "IKEA-like" programs. Design patterns and general reusability are the pre-cut, pressed particle board, screws, and Allen-tool sets.
> In the future AI code inevitably will embody most good software engineering practices

That is not inevitable. It's a big maybe.

Today's leading-edge AIs are still far from being good software architects. I constantly have to steer them away from producing spaghetti code.

Maybe AI will get there one day. But today's leading-edge models aren't even close yet.

(comment deleted)
But do the same SWE fundamentals apply if the one doing the programming is many times smarter than us?
I think SWE fundamentals matter - because it will be a long time before software is a closed system.

And the problem is that as long as humans are in the loop building software that dynamic will have to be maintained.

We use Loki for logging at work. There’s certain types of queries it just doesn’t support. And so the question becomes -

Do you change logging providers Adapt to Loki’s capabilities Create a third layer / tiered storage.

And each of those decisions have multiple downstream consequences. It’s not that LLMs can’t make those decisions per say, it’s that

What does an LLM do when five different people ask for a system optimized to do five different things.

It could figure it out itself, but like I don’t think that’s how the human software contract works.

I built a harness from scratch prior to trying any of the ones out there, so I knew how it work in a real way. I QUICKLY understood that the biggest issue with getting my shit done is that _my_ inputs are the untrusty ones. How many time do you hit backspace in a day?
Great. We are pretending that an LLM can think, and has intelligence. I thought the marketing wasn't supposed to work on us.
> Making software debuggable, maintainable, layered, and composable – that’s still quite a trick. Quite a lot of that work requires extensive, thoughtful reasoning. And that’s where the LLM’s today, even the leading edge of the “capability” from frontier models, fall short.

It’s been my quest during my career to figure out what is maintainable software, what is composable or not, and how the two things, and many other things, are in direct conflict. There is no single answer. If your goal is to take over the market quick, as many here would like, maintainability is a very low priority aspect of your code base. Composability may matter for integrators but can be entirely ignored in your CRUD backend. Beyond that, I don’t know of a good way to measure most of these intangible properties. Highly competent software developers disagree in even basic things, like whether OOP is a good idea, should we all be using pure functional programming etc. Hence, how would you expect an LLM to get good at figuring this out for you? If you describe exactly what trade offs you are willing to make, and give it ways to measure how well it’s doing, then I do think LLMs will be able to not fall short. Given the current state of things, it’s just a matter of opinion whether LLMs fall short, or humans fall short for that matter.

> If your goal is to take over the market quick, as many here would like, maintainability is a very low priority aspect of your code base.

Is it? How long do you expect to keep your momentum after "taking over the market"?

Anecdata time. I once joined a 3 year old project that had ground itself to a near halt with this philosophy. The project's lead seemed almost allergic to the word "refactoring". It had accrued so much tech debt that I was the third "new guy" to join in less than two years, after the previous attempts to hire had successively faltered within 6 months, because my predecessors couldn't deal with the unmaintainable mess. I made it to 9 months.

Maintainability is not tied to OOP or functional, but rather to how much a team cares to manage the cognitive load that comes attached to having to deal with the code base. When that becomes a genuine priority, the code tends to be written with concern for the next human mind's ability to interact with it. And when it makes sense in that one pursuit, functional, OOP, DRY, WET all become valid -- even seductive but toxic affordances like inheritance can sometimes be useful in the right context.

I have an open question for software engineers out there:

As someone that has never studied CS but has written basic code most of my life (accelerated now with AI), where is the best place to learn software engineering fundamentals?

I don't think you can learn without doing the actual work where normal constraints apply so you can test and see the pros and cons of different approaches to different aspects of the problem(s).

You need a limited budget, limited timeline, limited capabilities in team and software tools+systems, and you need to be on the hook for support, maintenance and long term extensibility.

Software engineering is an optimization problem balancing all of the above and much more. There is no one answer to any problem, but more of a general sweet-ish spot (or more like region than spot) of balancing the competing priorities.

Learn more engineering, any structured engineering. Electronics is particularly applicable, but hardly the only field that fits. Learning to find sources of error, how to handle precision and the systems of organization - all help.

I'm self-taught - could never afford much university... but I worked "odd jobs", such as assistant jobs in land surveying, welding, and electronics - and those all opened my eyes to a lot more. (I do have some university though, as well as a lot of experience... every time I line up to go back to learn more, I end up working instead, for a job I enjoy).

Oh yeah, a lot of basic engineering texts are online, or in libraries. Check them out if you can.

Hi, CS graduate here. during my post-graduation unemployed year I have been wrestling with this question a lot. The answer is that there is none - the research on the practice of translating requirements into code has stopped in the 1970s with Structured Programming (Dijkstra) and Problem Frames (Jackson). There is a certain inspiration people had with reusable patterns of problems a la Alexander, but after some time people got lost in the Java enterprise world and started making "design patterns" (which has very much dominated the online space and I cannot search properly for general software design anymore). The second problem is with Clean Code, which to me is a really bad set of "if you know you know" hand-wavy advice that is purposefully vague, because observable, actionable advice is actually very hard to come by.

I think when people say software engineering, they confuse the high level designs (server, caching, db, etc. generally interaction constraints) and the lower level designs (modules, abstractions, classes, data structure design). There is a lot of content on the former but really few on the latter - which is some people cannot write new code, but can work on existing systems fine. Two main problems is 1. How to break down the problem at hand and 2. Organizing data into abstractions, and often creating new ones.

For (1) the advice everyone resorts to is "just do more" and rely on pattern recognition, because you can't really break down a thing you haven't seen nor can relate to what you've seen before. But there is [A Framework for Decomposition in Computational Thinking](https://www.researchgate.net/publication/334579725_A_Framewo...) that's the best attempt at it so far in my opinion. Some other approaches involve observation from ideal user flow and software ideal behaviour, or UI wireframing to see what elements should be there.

For (2), this also has the problem of organizing (which code goes into which file, which function does this belong to) and naming. For organizing data, I will recommend [The Many Forms of a Single Fact](https://www.bkent.net/Doc/manyform.htm) and the first chapter of [Data and Reality](https://cmpct.info/~calvin/Papers/Data%20and%20Reality.pdf) to see how impossible it is to create exactly correct representations in code. In areas that's entirely inside the box (e.g graphics) people don't often deal with messy representations of real world domains, but in business software they will do much more.

On the naming side, the first chapter of [Elements of Clojure](https://elementsofclojure.com/) was a really good attempt, making me realize it is way deeper into philosophy than anyone would like to wander in.

That said, the way software engineering discussion has been so far is very much influenced by the programming languages of the time (see Java), and so my recommendation is to try to learn concepts of organization independently, and see what its workaround is in your active language (e.g algebraic data type is sometimes useful, and people try to do it with sealed classes in Kotlin or interfaces in TypeScript). Once you have a good grasp of one tool (with all their warts, they all have pains), you'll find the capability to experiment.

FWIW, here are the books I bought over that years that people would always say you should read. I would say some of the principles might be out of date, wrong, or overkill, but a good goal is to be at least familiar with them.

The Pragmatic Programmer, Design Patterns (or some Gang of 4 book), Clean Code, Code Complete

Scenario: an AI lab develops ASI for software development internally. The ASI produces bug-free software and human-readable specs. It's so reliable the company can guarantee the code matches the spec.

Rather than provide tokens to developers, they instead sell finished software to the companies: specs in, software out.

Companies no longer need to employ software developers. Instead, they buy bespoke, bug free, guaranteed quality software from the AI lab.

Seems plausible to me.