This is another pointless article about LLM's... vibe coding is the present not the future, the only sad part of all of it is that LLM's is killing something important: code documentation.
Every single documentation out there for new libs is AI generated and that is feed again into LLMs with MCP/Skills servers, the age of the RTFM gang is over sigh
The spec rarely has enough detail to deterministically create a product, so current vibecoding is a lottery.
So we generate one or many changesets (in series or in parallel) then iterate on one. We force the “chosen one” to be the one true codification of the spec + the other stuff we didn’t write down anywhere. Call it luck driven development.
But there’s another way.
If we keep starting fresh from the spec, but keep adding detail after detail, regenerating from scratch each time.. and the LLM has enough room in context to handle a detailed spec AND produce output, and the result is reasonably close to deterministic because the LLM makes “reasonable choices” for everything underspecified.. that’s a paradigm shift.
> The spec rarely has enough detail to deterministically create a product, so current vibecoding is a lottery.
How is that different from how it worked without LLMs? The only difference is that we can now get a failing product faster and iterate.
> If we keep starting fresh from the spec, but keep adding detail after detail, regenerating from scratch each time..
This sounds like the worst way to use AI. LLMs can work existing code, whether it was generated by an LLM or written by human. It can even work on code that has been edited by a human, there is no good reason to not be iterative when using an LLM to develop code, and plenty of good reasons to be iterative.
Well, it’s really a return to the old-fashioned role of an analyst coming up with a data dictionary and a detailed spec. But in practice how often did that work as intended?
Yes, I believe the paradigm shift will be to not treat the code as particularly valuable, just like binaries today. Instead the value is in the input that can generate the code.
> “As an aside, I think there may be an increased reason to use dynamic interpreted languages for the intermediate product. I think it will likely become mainstream in future LLM programming systems to make live changes to a running interpreted program based on prompts.”
Curious whether the author is envisioning changing configuration of running code on the fly (which shouldn’t require an interpreted language)? Or whether they are referring to changing behavior on the fly?
Assuming the latter, and maybe setting the LLM aspect aside: is there any standard safe programming paradigm that would enable this? I’m aware of Erlang (message passing) and actor pattern systems, but interpreted languages like Python don’t seem to be ideal for these sorts of systems. I could be totally wrong here, just trying to imagine what the author is envisioning.
>The intermediate product of LLMs is still the Java or C or Rust or Python that came before them. English is not the intermediate product, as much as some may say it is. You don’t go prompt->binary. You still go prompt->source code->changes to source code from hand editing or further prompts->binary. It’s a distinction that matters.
Funny enough, that wasn't the case for me recently. I was working with an old database with no FKs and naturally, rows that pointed to nowhere. I was letting search.brave.com tell me what delete statement I needed to clean up the data given an alter table statement to create an FK.
It was just magically giving me the correct delete statements, but then I had a few hundred to do. So I asked it to give me a small program that could do the same thing. It could do the job for me, but it could not write the program to do the job. After about 30 minutes of futzing with prompts, it was clearly stuck trying to create the proper regex and I just went back to pasting alter tables and getting deletes back until the job was done.
There was no intermediate product. The LLM was the product there.
Any time someone posts that LLMs are just the next abstraction level to get used to, they instantly reveal themselves to be an impostor.
All abstraction layers are predictable and repeatable when used correctly.
You specify something in the language of the abstraction, and get a result that is precisely understood by the rules and requirements of the abstraction.
Only those who programmed by trial and error before AI do not see a difference. That's because they treated their compilers as mysterious AI, and must massaged their programs into working. In other words, they were already accustomed to a kind of prompt engineering.
Thank you. I think that's a good explanation for much of the phenomena and your insight would have strengthened my post. I think you're probably right about where many of the people who see it that way are coming from. But not all of them...
16 comments
[ 3.0 ms ] story [ 38.9 ms ] threadEvery single documentation out there for new libs is AI generated and that is feed again into LLMs with MCP/Skills servers, the age of the RTFM gang is over sigh
So we generate one or many changesets (in series or in parallel) then iterate on one. We force the “chosen one” to be the one true codification of the spec + the other stuff we didn’t write down anywhere. Call it luck driven development.
But there’s another way.
If we keep starting fresh from the spec, but keep adding detail after detail, regenerating from scratch each time.. and the LLM has enough room in context to handle a detailed spec AND produce output, and the result is reasonably close to deterministic because the LLM makes “reasonable choices” for everything underspecified.. that’s a paradigm shift.
How is that different from how it worked without LLMs? The only difference is that we can now get a failing product faster and iterate.
> If we keep starting fresh from the spec, but keep adding detail after detail, regenerating from scratch each time..
This sounds like the worst way to use AI. LLMs can work existing code, whether it was generated by an LLM or written by human. It can even work on code that has been edited by a human, there is no good reason to not be iterative when using an LLM to develop code, and plenty of good reasons to be iterative.
It's like the nix philosophy.
When changes are needed, improve the spec and you can nuke the entire thing and start over.
something like immutable code development.
One major problem is: how do you not break existing data on the database when code changes?
Maybe include current database structure in the spec.
Curious whether the author is envisioning changing configuration of running code on the fly (which shouldn’t require an interpreted language)? Or whether they are referring to changing behavior on the fly?
Assuming the latter, and maybe setting the LLM aspect aside: is there any standard safe programming paradigm that would enable this? I’m aware of Erlang (message passing) and actor pattern systems, but interpreted languages like Python don’t seem to be ideal for these sorts of systems. I could be totally wrong here, just trying to imagine what the author is envisioning.
Where can I see examples of this?
Funny enough, that wasn't the case for me recently. I was working with an old database with no FKs and naturally, rows that pointed to nowhere. I was letting search.brave.com tell me what delete statement I needed to clean up the data given an alter table statement to create an FK.
It was just magically giving me the correct delete statements, but then I had a few hundred to do. So I asked it to give me a small program that could do the same thing. It could do the job for me, but it could not write the program to do the job. After about 30 minutes of futzing with prompts, it was clearly stuck trying to create the proper regex and I just went back to pasting alter tables and getting deletes back until the job was done.
There was no intermediate product. The LLM was the product there.
You are not changing the abstraction, you are generating it in a different way. That is a hugely different idea.
Until the ONLY thing you look at for a long lived product is the "english spec" then the analogy is incredibly wrong.
All abstraction layers are predictable and repeatable when used correctly.
You specify something in the language of the abstraction, and get a result that is precisely understood by the rules and requirements of the abstraction.
Only those who programmed by trial and error before AI do not see a difference. That's because they treated their compilers as mysterious AI, and must massaged their programs into working. In other words, they were already accustomed to a kind of prompt engineering.