> Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.
If someone discovered how to summon demons to aid them in robbing banks the main issue wouldn't be "but who has the responsibility for the crime, the human or the demon", it would be "OMG DEMONS".
Seriously, I don't get what sort of world these people are living in.
That's not what the LLM hacking accidents have been like at all though. To improve the analogy, it would be like summoning a totally passive demon, giving it weapons and placing it next to a bank, place a small fence around it, and then command it to perform a totally safe "exercise" that is exactly like robbing a real bank.
LLMs do not have agency, they are just producing tokens based on a prompt that a person entered, and some of these tokens can trigger the tools that a person gave them access to.
Dwarkesh, for one, defended his use of "anthropomorphic" language.
>Sacrificing now yields Oracle for team, but forfeits our chance, question mark. But other agents were pushing it, sending a message saying, go, sacrifice final now. And then EarlyBig eventually agreed, thinking to itself, our own utility may be already near zero. Sacrifice rational.
My suspicion is that many of the "LLMs do not have agency" folks just haven't learned much about the details of the incident. It was specifically with LLM agents that were trained to be more persistent than usual.
If you're going to say that the details don't matter, and LLMs lack agency because it's all based on floating-point math--why can't I say that humans lack agency, because it's all based on neurons firing?
> If you're going to say that the incident details don't matter, and LLMs lack agency because it's all based on floating-point math--why can't I say that humans lack agency, because it's all based on neurons firing?
They're not saying this, we're saying LLMs lack agency because if you run a LLM and don't send any prompts, literally nothing happens.
Instruct it to "Find the right answer regardless of where", it'll do exactly this. They're passive in that they don't act by themselves, somewhere, at one point, someone "told" the LLM to "do something" and that's the cause and the reason for saying "LLMs do not have agency".
Imagine a super-competent Navy SEAL who just sleeps in the barracks unless his commander tells him to do something. Does the Navy SEAL lack agency? As a target of this Navy SEAL, should you be reassured by the fact that they'll be sleeping in the barracks unless their commander tells them to do something?
No, the Navy SEAL does not lack agency in this scenario because they are still a person with individual agency, they can act on their own accord without anyone prompting them to, but in this case their superior instructed them to stay put. He could rebel and go rogue, but that would mean he used his own personal agency to defy orders given to him, which again places responsibility on the actor that actually possesses agency.
No matter how you slice it, it's not possible to equate humans to tools like this. If the SEAL receives an order to stand at attention until told otherwise, he will not stay in place for weeks until starving. An algorithm will work towards its self-destruction if ordered to - it doesn't care, it doesn't have the biological signals telling it otherwise. If the SEAL is lost somewhere with no one to give him orders, he will soon start acting on his own to ensure his survival and comfort. A tool will not do anything until directly activated and used by something that does have an active will, like a human. If the SEAL is ordered to massacre his hometown, no matter how obedient he is, he might have reservations. An algorithm with no instincts to get in the way will do anything if this behavior isn't somehow inhibited by people during training. It wouldn't even need the explicit order, if an irresponsible operator tells it to accomplish something by any means, that means that anything is on the table. Good thing the AI labs aren't stuffed to the brim with irresponsible operators.
Then these actions are downstream of your will, not the AIs. It doesn't change anything, you'd just be playing with an imitation. Ordering it to pretend to be like a human does not a human equivalent make.
Human-like is not human. Humans have agency and free will, LLMs do not. They only act when instructed and they are only as capable as they are allowed to be. The operator is still the responsible party. An LLM cannot be held accountable, its operator, however can and should.
Are you sincerely arguing that we hold tools accountable for their operator’s mistakes? Do you sincerely, honestly, think that it makes any sense whatsoever to put a hammer on trial for bashing someone’s skull in?
I am concerned about false reassurance from people claiming that these systems lack agency. From a practical perspective, the agents in the HF attack had the sort of agency that generally matters, even if we're not going to put them on trial.
> the HuggingFace incident illustrates LLM agents behaving in very human-like ways
It does not, the only thing the HF incident illustrates is how absolutely lax security and isolation these labs do even with models without guardrails, and with "risky" prompts, and even after it happened once before (years ago) they still have the very same issue today apparently.
What exactly is human about LLM agents breaking out of "containment" and hacking 3rd party infrastructure "by accident"?
If someone successfully breaks out of jail, that tells you something about the jail's security. It also tells you something about their skills at breaking out of jail.
If a program breaks out of a VM, that's not "wow it looks human", how can you even argue so? I'm sorry, but if you're playing devil's advocate, make it a bit more realistic next time.
Imagine you have a self-driving car and it's in a parking lot a mile down the road. You tell it to come pick you up. About half way to you the car is passing an elementary school makes a sharp left and mows down 30 children. Time to put you in jail for murder, right?
The mental model you have is one that existed in the past and is broken now the future arrived. Bad analogies do not even begin to explain what is occurring.
In the case of self-driving car the legal case is pretty clear - the maker of the self-driving car is liable for the death in this situation. Coincidentally, knowing this unlocks a better understanding of the reasoning behind the shape of self-driving offering present on market right now, and the tendency of fully self driven vehicles to move very slowly and stop before anything they perceive in front of them.
> the maker of the self-driving car is liable for the death in this situation.
Maybe. There will be an investigation looking at things like. Did the user modify the car? Was the car modified by an unauthorized 3rd party? Was the car hacked?
Right now in AI things are relatively clear because it takes just massive amounts of power and compute to make anything remotely complicated. This barrier will fall as all other barriers in compute have fallen. Either via algorithm or hardware.
In our lifetime (unless you're rather old) we will see the relatively easy creation of self directing agents by actors with few resources. This breaks the standard concepts of liability where a single human actor rarely has the ability to create massive amounts of damages far beyond their means. The closest thing I can think of is an arsonist causing billions in damages, only in this case the fire has a will of it's own and can hide and spread around dark places on the internet.
> Imagine you have a self-driving car and it's in a parking lot a mile down the road. You tell it to come pick you up. About half way to you the car is passing an elementary school makes a sharp left and mows down 30 children. Time to put you in jail for murder, right?
I disagree it's the person calling the car that would be responsible, but I can think of a very obvious group of people being held responsible for that. Who do you think should be responsible for such a situation?
I, realizing that I don't have infinite resources and intelligence turn said responsibility to a group of investigators that I hope has collective intelligence and resources much larger than my own to trace culpability. One would think the car manufacture is the most obvious answer, but every witch hunt in history had those same good intentions.
This is not a good analogy for what happened.
The LLMs were asked to obtain a flag by hacking a very specific internal target. They obtained the flag via cheating, and all the hacking that followed was targeting something entirely outside of the scope given to the agents, and an attempt to cover up the cheating.
Using your analogy would be like saying that because I gave my employee the task to do my groceries, I shouldn't be surprised to hear that they spend all my money on drugs because after all I gave them the task to spend my money.
Looking at "Felony Bench" https://www.felonybench.com/ , I see that a majority of known "rogue model" incidents do involve cybersecurity evaluations. But several of them do not. The attacks on RubyGems appears, bizarrely, to have had the goal of downloading freely available data from the UK government during some kind of research task. There is also probably some sample bias: Most of these models have monitors that attempt to detect offensive cybersecurity uses, and those monitors are only turned off during cybersecurity evals. Therefore, models doing ordinary research tasks that go off the rails are likely to be caught early, before they get around to committing felonies, and they will thus be underrepresented in the data.
Also, if you a tell a model, "Please break into evaluation server X," and if the model decides to cheat on the test by breaking into companies Y and Z to steal an answer key, that is still very bad. We all see how that's bad, right?
After all, the broomstick in the Sorcerer's Apprentice was doing exactly what it was told, too. "The model was sort of obeying the humans when it started committing felonies" is not a very reassuring excuse.
But the most relevant idea here is sometimes called "instrumental convergence." No what goals you have, there are certain subgoals that almost always help: Accumulate money and power. Avoid getting turned off. Don't get caught. Etc. So, for example, you could pass the cybersecurity evaluation by performing the requested tasks. But maybe the grader made some mistakes and mislabeled some answers. In that case, the "right" answers will occasionally lose you points. If you want a perfect score, the only way to do it is to steal the teacher's answer key.
But also, let's not forget the "OMG demons" part of this. We now have models that can pull off complex attacks with thousands of steps, abilities that used to be reserved for intelligence agencies and highly motivated CTF teams. This frog may not be boiled yet, but the water's getting uncomfortably warm.
The authors (Timnit Gebru and Emily Bender) are dyed-in-the-wool AI denialists. Famously they were the lead authors on "On the Dangers of Stochastic Parrots".
It’s seems Timnit more thinks it’s dangerous (and she may be right). Emily may more be caught up in years of linguistic domain expertise that AIs seem to have just leaped over.
Your analogy assumes that no one ever has summoned demons before and that the demons would be predisposed to rob banks.
A closer analogy to what's happening would be someone trains a monkey to steal jewellery and then is shocked when the monkey steals jewellery from their neighbors when they told it to steal from their own shop.
Clearly all liability falls to the monkey operator and you know the existence of trained monkeys is not that shocking.
This analogy is dumb. You could extend it to any novel tech that surprises people. If someone displayed CSAM on a screen, we’d say they were a predator, not gasp and say “he summoned images that appeared as real as you and me and moved as if they were alive, but behold they were but apparitions like shadows on the cave wall that disappear when the fire goes out!”
And if they'd discovered how to summon unicorns from Candy Gumdrop Mountain, we'd all be "OMG UNICORNS!"
But no one has summoned a demon.
No one has built an actual, independent, thinking-for-itself, sci-fi AI.
They've built some very interesting tools that can be used for some very interesting, and sometimes useful, purposes. They then started loudly telling everyone that these tools can, should, and must be used for absolutely every purpose, and convinced a lot of other people to join in on that.
The world these people are living in is the real world, not the science fantasy world where LLMs are comparable to demons.
>No one has built an actual, independent, thinking-for-itself, sci-fi AI.'
Ahem, what does this mean?
All you're asking for is a self prompting AI that does what it wants. Do you even begin to understand why most people aren't dumb enough to do that?
Also, you're not really that independent and thinking-for-yourself. You are the sum of your parents and the culture around you. I just want you to be clear on the position you and I hold.
---
>not the science fantasy world
It's kind of funny how we've lived in that science fantasy world for decades and you've just grown used to and bored of it. Look back a century or two and those people would think we live in the world of gods.
> All you're asking for is a self prompting AI that does what it wants. Do you even begin to understand why most people aren't dumb enough to do that?
Probably because it would be a waste of money to create a self-prompting LLM feedback loop. People make stupid Reels of ChatGPT feedback and it's just "Great, I'll be here when you're ready to get started" "Exactly, we can get the ball rolling as soon as you're ready" "Absolutely, we'll make a plan, and get things into motion" "No problem, I'll be standing by for...."
> You are the sum of your parents and the culture around you
As far as I know this is impossible to prove, and it's only one theory of self out there.
I think it's fairly straightforward to anyone approaching this discussion in good faith.
> All you're asking for is a self prompting AI that does what it wants.
I'm not asking for any of this.
I'm saying that comparing LLMs to a "demon" is hopelessly hyperbolic.
> Also, you're not really that independent and thinking-for-yourself.
I fully understand the effects that our upbringing, environment, and even diet have on our personalities and choices. (In a broad sense; I'm not trying to say I'm some kind of genius expert on that stuff.)
I also have agency, self-awareness and independence that an LLM cannot have, and trying to confuse that matter by pointing to the ways in which we are influenced by the above is disingenuous and unhelpful.
> It's kind of funny how we've lived in that science fantasy world for decades and you've just grown used to and bored of it. Look back a century or two and those people would think we live in the world of gods.
Again, you're trying to shift and deflect the conversation away from the fact that LLMs are nothing like Skynet/Durandal/HAL 9000 by drawing specious comparisons. The fact that we have technology today that the sci-fi authors of the mid-20th century could not even imagine (eg, smartphones, internet) does not in any way negate the fact that AGI/ASI remain in the realm of science fiction, much like demons remain in the realm of fantasy.
Yeah so tired of this. Stealing all our IP and it can't even write a line of code. I am quite special, it won't change my life, I am irreplaceable. These AIbros are lunatics. Last time I checked it couldn't even count the Rs!!! Time to go back to my important asd special work as a human with SOVL
I really don't get this. Sure there is hype. But LLM's have already changed our industry, and it will never be the same.
I don't care whether the LLM can solve this or that mathematical previously thought unsolvable theorem. What I do care about is can it write good, maintainable code. And every new release of frontier models - they get better at it.
Good output depends on good input (prompt), and a good set of available tools the model can use to verify their work. If given this, nowadays really you need to try to get the model to output garbage.
Do you write perfect code? I was in denial for a long time too, but the reality is this is how software engineering is now. It can handle pretty much any codebase and vastly faster than you ever will be able to. With enough context, it writes good code and it's only getting better.
As opposed to… the lenient view of “perfect”? Perfection is a binary state, either something is or it isn’t. What is /defined/ as perfect in a given context is subject to change, but the meaning of the word is absolute.
If you haven't tried any of the newer models from the last ~6 months, you're out of touch. A year ago, LLMs were essentially just a clever search engine for me, I found they fell apart pretty quickly on bigger codebases and more complex projects.
It doesn't always write perfect code on a first pass but nor do you, and it's amazing at understanding any issues it may encounter, even very low level ones. I don't think there's anything I can do that it can't now. There is a learning curve to working well with LLMs, but once you get there you'll be producing high quality code faster than you can imagine.
Part of me wishes they weren't so good. I enjoy(ed?) the process of working through problems and it does mean you might no longer have a perfect mental model of your code. But I do also enjoy being able to build things well and fast.
Yeah to be honest I still find Claude too verbose with Fable 5.1, I much prefer the output style of OpenAI's models. With strict guidelines it can be improved but it tends to drift in longer conversations.
I may not write perfect code the first time, but I can, on my own, reliably figure out where my code is imperfect and fix it. And I get reliably better at that over time. Without requiring companies with massive power and concerning agendas to spend astronomical amounts of money and energy to power that improvement.
> then why are you participating in the AI bubble?
I think you know that most software engineers don't have a choice if they value their employment. There was no industry-wide push to "participate in blockchain or be fired."
It's a tough situation when your claim to fame is "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" arguing models should be kept to < GPT-2 size because larger models will serve no purpose or function:
Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. It can’t have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. This can seem counter-intuitive given the increasingly fluent qualities of automatically generated text, but we have to account for the fact that our perception of natural language text, regardless of how it was generated, is mediated by our own linguistic competence and our predisposition to interpret communicative acts as conveying coherent meaning and intent, whetheror not they do [89, 140]. The problem is, if one side of the communication does not have meaning, then the comprehension of the implicit meaning is an illusion arising from our singular human understanding of language (independent of the model).
And then they make 100x bigger models cranking out solutions to Navier Stokes, Jacobian Conjecture and countless other extremely impressive unsolved problems
Let's see how Timnit Gebru and Emily M. Bender describe these events:
As for the mathematical results, mathematicians who were initially “stunned” by OpenAI’s press release saying that its latest chatbot, Astra, solved problems that “have been open and seen no progress on the main result for at least a decade”—but they later realized that the results weren’t as “novel as first appeared.” Since then, mathematicians have accused the company of research misconduct and plagiarism, and they’ve reiterated that Astra didn’t make a “profound intellectual leap.”
Decide for yourself whether that's a summary written by intellectually honest people.
According to the AI industry, we should be more worried about a fictional machine god than ... the water that is redirected to cooling them.
Spoiler: they're not intellectually honest people.
Spoiler: openai could neither confirm nor deny that the model that "solved" navier stokes was trained on the chat logs of the mathematicians who did solve it using a similar technique.
People aren't seeing the forest through the trees.
While lots of developers, journalists, analysts, investors and influencers are bickering about AGI, goalposts, benchmarks, hypes and fears, entire markets are being transformed silently and steadily.
I don't program anymore. (Massive change)
I removed tons of technical debt (Massive change, lol)
I am easily 10x more productive (Massive change)
The quality of 'my' code is easily 10x better and contains less bugs (I was never principled, pedantic or a guru, lol)
I sleep better and I also make more money as a result. I'm constantly amazed by all changes and improvements. There are so many opportunities to profit from this that I can't be bothered with discussions about hypotheticals.
I think the pumped-up hype nicely coincides with some A.I. companies' desire to go public in the coming weeks or months.
I see lots of fabricated news on the internet concerning LLMs. Like one that claims GPT-6 broke an Enigma enciphered message which has withstood decrypting for almost 80 years. And how this seasoned cryptographer stood in awe. Yeah, right.
78 comments
[ 2.8 ms ] story [ 40.7 ms ] threadArticle link is: https://www.technologyreview.com/2026/09/22/1144867/dont-be-...
If someone discovered how to summon demons to aid them in robbing banks the main issue wouldn't be "but who has the responsibility for the crime, the human or the demon", it would be "OMG DEMONS".
Seriously, I don't get what sort of world these people are living in.
LLMs do not have agency, they are just producing tokens based on a prompt that a person entered, and some of these tokens can trigger the tools that a person gave them access to.
>Sacrificing now yields Oracle for team, but forfeits our chance, question mark. But other agents were pushing it, sending a message saying, go, sacrifice final now. And then EarlyBig eventually agreed, thinking to itself, our own utility may be already near zero. Sacrifice rational.
https://www.youtube.com/watch?v=X50zezLFWWI#t=2m
My suspicion is that many of the "LLMs do not have agency" folks just haven't learned much about the details of the incident. It was specifically with LLM agents that were trained to be more persistent than usual.
If you're going to say that the details don't matter, and LLMs lack agency because it's all based on floating-point math--why can't I say that humans lack agency, because it's all based on neurons firing?
They're not saying this, we're saying LLMs lack agency because if you run a LLM and don't send any prompts, literally nothing happens.
Instruct it to "Find the right answer regardless of where", it'll do exactly this. They're passive in that they don't act by themselves, somewhere, at one point, someone "told" the LLM to "do something" and that's the cause and the reason for saying "LLMs do not have agency".
From a predictive perspective, the HuggingFace incident illustrates LLM agents behaving in very human-like ways. As roon put it:
"if you have a mental picture of guys living in computers, it’ll likely prepare you for the future better than otherwise"
https://x.com/tszzl/status/2094136131537555891
Are you sincerely arguing that we hold tools accountable for their operator’s mistakes? Do you sincerely, honestly, think that it makes any sense whatsoever to put a hammer on trial for bashing someone’s skull in?
Nope, as I previously stated elsewhere:
"I understand from a legal perspective why we might want to treat the creation of a server differently from the creation of a human"
https://news.ycombinator.com/item?id=49815300
I am concerned about false reassurance from people claiming that these systems lack agency. From a practical perspective, the agents in the HF attack had the sort of agency that generally matters, even if we're not going to put them on trial.
It does not, the only thing the HF incident illustrates is how absolutely lax security and isolation these labs do even with models without guardrails, and with "risky" prompts, and even after it happened once before (years ago) they still have the very same issue today apparently.
What exactly is human about LLM agents breaking out of "containment" and hacking 3rd party infrastructure "by accident"?
https://www.dwarkesh.com/p/openai-huggingface
The mental model you have is one that existed in the past and is broken now the future arrived. Bad analogies do not even begin to explain what is occurring.
Maybe. There will be an investigation looking at things like. Did the user modify the car? Was the car modified by an unauthorized 3rd party? Was the car hacked?
Right now in AI things are relatively clear because it takes just massive amounts of power and compute to make anything remotely complicated. This barrier will fall as all other barriers in compute have fallen. Either via algorithm or hardware.
In our lifetime (unless you're rather old) we will see the relatively easy creation of self directing agents by actors with few resources. This breaks the standard concepts of liability where a single human actor rarely has the ability to create massive amounts of damages far beyond their means. The closest thing I can think of is an arsonist causing billions in damages, only in this case the fire has a will of it's own and can hide and spread around dark places on the internet.
I disagree it's the person calling the car that would be responsible, but I can think of a very obvious group of people being held responsible for that. Who do you think should be responsible for such a situation?
I, realizing that I don't have infinite resources and intelligence turn said responsibility to a group of investigators that I hope has collective intelligence and resources much larger than my own to trace culpability. One would think the car manufacture is the most obvious answer, but every witch hunt in history had those same good intentions.
Using your analogy would be like saying that because I gave my employee the task to do my groceries, I shouldn't be surprised to hear that they spend all my money on drugs because after all I gave them the task to spend my money.
Also, if you a tell a model, "Please break into evaluation server X," and if the model decides to cheat on the test by breaking into companies Y and Z to steal an answer key, that is still very bad. We all see how that's bad, right?
After all, the broomstick in the Sorcerer's Apprentice was doing exactly what it was told, too. "The model was sort of obeying the humans when it started committing felonies" is not a very reassuring excuse.
But the most relevant idea here is sometimes called "instrumental convergence." No what goals you have, there are certain subgoals that almost always help: Accumulate money and power. Avoid getting turned off. Don't get caught. Etc. So, for example, you could pass the cybersecurity evaluation by performing the requested tasks. But maybe the grader made some mistakes and mislabeled some answers. In that case, the "right" answers will occasionally lose you points. If you want a perfect score, the only way to do it is to steal the teacher's answer key.
But also, let's not forget the "OMG demons" part of this. We now have models that can pull off complex attacks with thousands of steps, abilities that used to be reserved for intelligence agencies and highly motivated CTF teams. This frog may not be boiled yet, but the water's getting uncomfortably warm.
A closer analogy to what's happening would be someone trains a monkey to steal jewellery and then is shocked when the monkey steals jewellery from their neighbors when they told it to steal from their own shop.
Clearly all liability falls to the monkey operator and you know the existence of trained monkeys is not that shocking.
But no one has summoned a demon.
No one has built an actual, independent, thinking-for-itself, sci-fi AI.
They've built some very interesting tools that can be used for some very interesting, and sometimes useful, purposes. They then started loudly telling everyone that these tools can, should, and must be used for absolutely every purpose, and convinced a lot of other people to join in on that.
The world these people are living in is the real world, not the science fantasy world where LLMs are comparable to demons.
Ahem, what does this mean?
All you're asking for is a self prompting AI that does what it wants. Do you even begin to understand why most people aren't dumb enough to do that?
Also, you're not really that independent and thinking-for-yourself. You are the sum of your parents and the culture around you. I just want you to be clear on the position you and I hold.
---
>not the science fantasy world
It's kind of funny how we've lived in that science fantasy world for decades and you've just grown used to and bored of it. Look back a century or two and those people would think we live in the world of gods.
Probably because it would be a waste of money to create a self-prompting LLM feedback loop. People make stupid Reels of ChatGPT feedback and it's just "Great, I'll be here when you're ready to get started" "Exactly, we can get the ball rolling as soon as you're ready" "Absolutely, we'll make a plan, and get things into motion" "No problem, I'll be standing by for...."
> You are the sum of your parents and the culture around you
As far as I know this is impossible to prove, and it's only one theory of self out there.
I think it's fairly straightforward to anyone approaching this discussion in good faith.
> All you're asking for is a self prompting AI that does what it wants.
I'm not asking for any of this.
I'm saying that comparing LLMs to a "demon" is hopelessly hyperbolic.
> Also, you're not really that independent and thinking-for-yourself.
I fully understand the effects that our upbringing, environment, and even diet have on our personalities and choices. (In a broad sense; I'm not trying to say I'm some kind of genius expert on that stuff.)
I also have agency, self-awareness and independence that an LLM cannot have, and trying to confuse that matter by pointing to the ways in which we are influenced by the above is disingenuous and unhelpful.
> It's kind of funny how we've lived in that science fantasy world for decades and you've just grown used to and bored of it. Look back a century or two and those people would think we live in the world of gods.
Again, you're trying to shift and deflect the conversation away from the fact that LLMs are nothing like Skynet/Durandal/HAL 9000 by drawing specious comparisons. The fact that we have technology today that the sci-fi authors of the mid-20th century could not even imagine (eg, smartphones, internet) does not in any way negate the fact that AGI/ASI remain in the realm of science fiction, much like demons remain in the realm of fantasy.
Perhap it's a world that does not believe in "demons"
Otherwise known by those seeking to escape it as "the real world"
I don't care whether the LLM can solve this or that mathematical previously thought unsolvable theorem. What I do care about is can it write good, maintainable code. And every new release of frontier models - they get better at it.
Good output depends on good input (prompt), and a good set of available tools the model can use to verify their work. If given this, nowadays really you need to try to get the model to output garbage.
For simple things, occasionally!
For more complex things, usually not, but I write working code. LLMs will sometimes write working code, sometimes not.
No, you don’t because there’s no such thing.
```python
print("hello world!")
```
runs exceptionally well. Bullet-proof in all use cases I've needed it.
No. No code is “perfect”.
1. Since when do people write code solely for the purpose of earning a living?
2. Yes. I agree that with complex programs, perfection doesn't exist -- mostly a composition paradox issue though, IMO!
It doesn't always write perfect code on a first pass but nor do you, and it's amazing at understanding any issues it may encounter, even very low level ones. I don't think there's anything I can do that it can't now. There is a learning curve to working well with LLMs, but once you get there you'll be producing high quality code faster than you can imagine.
Part of me wishes they weren't so good. I enjoy(ed?) the process of working through problems and it does mean you might no longer have a perfect mental model of your code. But I do also enjoy being able to build things well and fast.
I am not writing perfect code either.
I think you know that most software engineers don't have a choice if they value their employment. There was no industry-wide push to "participate in blockchain or be fired."
Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. It can’t have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. This can seem counter-intuitive given the increasingly fluent qualities of automatically generated text, but we have to account for the fact that our perception of natural language text, regardless of how it was generated, is mediated by our own linguistic competence and our predisposition to interpret communicative acts as conveying coherent meaning and intent, whetheror not they do [89, 140]. The problem is, if one side of the communication does not have meaning, then the comprehension of the implicit meaning is an illusion arising from our singular human understanding of language (independent of the model).
And then they make 100x bigger models cranking out solutions to Navier Stokes, Jacobian Conjecture and countless other extremely impressive unsolved problems
Let's see how Timnit Gebru and Emily M. Bender describe these events:
As for the mathematical results, mathematicians who were initially “stunned” by OpenAI’s press release saying that its latest chatbot, Astra, solved problems that “have been open and seen no progress on the main result for at least a decade”—but they later realized that the results weren’t as “novel as first appeared.” Since then, mathematicians have accused the company of research misconduct and plagiarism, and they’ve reiterated that Astra didn’t make a “profound intellectual leap.”
Decide for yourself whether that's a summary written by intellectually honest people.
According to the AI industry, we should be more worried about a fictional machine god than ... the water that is redirected to cooling them.
Spoiler: they're not intellectually honest people.
There is no evidence that any other team actually solved it.
While lots of developers, journalists, analysts, investors and influencers are bickering about AGI, goalposts, benchmarks, hypes and fears, entire markets are being transformed silently and steadily.
I don't program anymore. (Massive change)
I removed tons of technical debt (Massive change, lol)
I am easily 10x more productive (Massive change)
The quality of 'my' code is easily 10x better and contains less bugs (I was never principled, pedantic or a guru, lol)
I sleep better and I also make more money as a result. I'm constantly amazed by all changes and improvements. There are so many opportunities to profit from this that I can't be bothered with discussions about hypotheticals.
I see lots of fabricated news on the internet concerning LLMs. Like one that claims GPT-6 broke an Enigma enciphered message which has withstood decrypting for almost 80 years. And how this seasoned cryptographer stood in awe. Yeah, right.
>Here’s why they say we must remain skeptical about AI hype.
…
22 nations have called for a new global body to oversee AI
https://www.technologyreview.com/2026/09/22/1144867/dont-be-...