Amodei's work history indicates that his background as a software developer is a single part-time job that he held for a year-and-a-half after college. As far as I'm concerned, he wouldn't even make it as a junior on my team. I'm not inclined to believe anything he says about what it takes to write production-ready code.
I mean, trivially true, if you consider that AI-enhanced programming requires 1000% more code written, deleted and rewritten, countless times, by AI itself in it's feeble attempt to "reason" about the problem.
This industry is full of snake oil salesmen and false promises, it's reliant on hype and an army of fanboys trained on twitter to fan the flames and create the impression of a revolution that isn't there and might never be there, just to prop the valuations of the likes of OpenAI and Anthropic.
It always follows the same playbook:
- The guru type e.g muskie, sama, adam neumann, elizabeth holmes, SBF, says something on twitter: "XYZ is the future"
- Army of same-thinkers lurkers start getting excited about it.
- VC fund bros who are also forming their opinion on vibes stalking twitter, also get excited about it.
- Big money poured in XYZ thing.
- Guru keeps claiming that XYZ version 2.0 is just behind the corner (AGI/Superintelligence, Autonomous Self Driving Cars, the Boring Tunnel, cure for whatever) projecting insane and out of touch with reality timelines.
- XYZ thing either becomes the future (not likely), or gets abandoned (remember when buying pixelized pics of apes on the internet was an "investment"?) and the collective amnesia of the fanbase doesn't keep the guru accountable. It's easy to make hundreds of claims if getting it right once, even by chance, makes you some kind of a prophet and all your wrong claims of the past are dismissed due to your fanbase's dog-like adoration for you.
- Cycle repeats.
The LLM craze reminds me of the NFT, web3, crypto grift all over again, but this time with a product that at least has some value.
Did some bigger admin tool to do search and matching datapoints etc and decided to only use claude code. First version was wow awesome this saved me so much time .. now 2 weeks later putting back code that was deleted several times, removing 3 copy's of the same code, way to complicated sqls, verbose code and looking how its mixing htmx with some wird own JavaScript.
It really looks as a junior developer solutions, so I'm done using it for more then boiler plate things..
I will come back in 3-6 months and hope its better to understand its own limitations.
I'm guessing it would be less then a week for me to write it on my own.
The biggest issue is that I've now seen the shit it creates so I have zero trust in the code I now have from a security and stability standpoint. I know many have better experiences then mine.
The thing is that this is probably currently indeed happening. The bizarre output of LoC by LLMs probably now eclipses whatever humans write by hand. I don't believe 90% passes the reviews though; a lot more gets written and a lot more gets discarded. My fear is more that this will change, and code will pass because less and less people know what is good or not. Not yet, but soon-ish, EVEN if the output quality does not improve.
For CRUD/UI heavy web apps, it can. I have probably wrote ~5% of the code I produced in the last 2 months, but I've read and verified and corrected every line of code. Output increase is substantial.
But I'm not expecting that to hold true for linux kernel or postgres codebase or equivalent anytime soon.
For me this appears to be true actually, these days. It struggles more with large and terrible codebases though, because it has to ingest so much garbage code in order not to break anything, wheras with nicely strucured code it can do great work on a module level, as long as you direct it well.
36 comments
[ 4.3 ms ] story [ 41.6 ms ] threadOr in other words, of course he says that; it's not really a useful analysis is it?
https://www.techradar.com/pro/nearly-half-of-all-code-genera...
Or another way, if Rust didn't exist, could an AI create Rust?
https://www.businessinsider.com/github-ceo-developers-embrac...
https://www.theverge.com/news/757461/microsoft-github-thomas...
This industry is full of snake oil salesmen and false promises, it's reliant on hype and an army of fanboys trained on twitter to fan the flames and create the impression of a revolution that isn't there and might never be there, just to prop the valuations of the likes of OpenAI and Anthropic.
It always follows the same playbook:
- The guru type e.g muskie, sama, adam neumann, elizabeth holmes, SBF, says something on twitter: "XYZ is the future"
- Army of same-thinkers lurkers start getting excited about it.
- VC fund bros who are also forming their opinion on vibes stalking twitter, also get excited about it.
- Big money poured in XYZ thing.
- Guru keeps claiming that XYZ version 2.0 is just behind the corner (AGI/Superintelligence, Autonomous Self Driving Cars, the Boring Tunnel, cure for whatever) projecting insane and out of touch with reality timelines.
- XYZ thing either becomes the future (not likely), or gets abandoned (remember when buying pixelized pics of apes on the internet was an "investment"?) and the collective amnesia of the fanbase doesn't keep the guru accountable. It's easy to make hundreds of claims if getting it right once, even by chance, makes you some kind of a prophet and all your wrong claims of the past are dismissed due to your fanbase's dog-like adoration for you.
- Cycle repeats.
The LLM craze reminds me of the NFT, web3, crypto grift all over again, but this time with a product that at least has some value.
Tom Renner wrote an excellent article on the "LLM inevitabilism" https://tomrenner.com/posts/llm-inevitabilism/
I will come back in 3-6 months and hope its better to understand its own limitations.
I'm guessing it would be less then a week for me to write it on my own.
The biggest issue is that I've now seen the shit it creates so I have zero trust in the code I now have from a security and stability standpoint. I know many have better experiences then mine.
Project managers be like sending me AI code snippets like they know shit.
But I'm not expecting that to hold true for linux kernel or postgres codebase or equivalent anytime soon.