Imagine a world where the human-in-the-loop was deemed to inefficient, and code does not get reviewed anymore. The next generation of worms just need to spread into the data centers that host these models and flip neurons. Would be a pretty cool supply chain hack.
Imagine hypothetical scenario where model take care life support system for spaceship - then single neuron flip due bit flip and decides to vent air out.
I am more afraid of accidental situation (as opposite to customer support bots, designed to give no quarter) where such bug propagates (by slop being included in training set) and all LLMs agree with other while humans are considered the weird ones, and nothing can cause the former to change their mind.
This type of research should put the "do LLMs think" to rest, not to mention the ludicrous claim of consciousness cough Dawkins cough. It is quite clearly a pattern matching algorithm at scale.
It's still narrow AI but massively scaled to look like general AI.
How does this follow? Isn't it fair to say that what someone's interacting with at the 'personality surface'-level is a form of emergent complexity / emergent behavior, not readily apparent or discernible at the neuron level (just like how human 'thinking' or 'consciousness' seems to work)? I'm not even arguing these models are conscious, I'm just saying that it doesn't seem like we can take the fact that a single neuron flipping their behavior or triggering 'alter egos' is enough to say they don't "think", etc. We might discover that flipping - if not a single neuron but perhaps really tiny clusters of neurons/synapses - in a human brain, might result in similar effects. We haven't had human connectomics studies to really show this (to my knowledge).
How bout we think about consciousness as a VM; that VM can load one, many, several types of context processing. The environment (nurture) is what trains/expands the different ones and potentially unplug the other ones, etc.
One of those types of consciousness processing is indistinguishable from what an LLM does.
Everyone knows someone who seems to just call and respond based on whatever popular media of their choice of culture is on about.
I'm not sure why you think so. We know that very local brain injuries can result in personality changes, sometimes dramatic ones. And that's in a system evolved to deal with constant damage and rebuilding. With LLMs trained to achieve the opposite of that (minimal redundancy, maximum performance), the effect is similar but more extreme.
27 comments
[ 10.0 ms ] story [ 42.9 ms ] threadIt's still narrow AI but massively scaled to look like general AI.
Well, compared to the scale of an LLM it’s actually massive.
You might have a different preference today than yesterday.
I think all this study shows is that a human brain is more resilient to minuscule changes/damage.
One of those types of consciousness processing is indistinguishable from what an LLM does.
Everyone knows someone who seems to just call and respond based on whatever popular media of their choice of culture is on about.
But not damage to a single neurons. You are talking about damage to thousands of neurons.
To make it equivalent you would need at least 100,000 of these models.
It therefore provides a correct answer in 96% of cases, and displays a sense of humor in the remaining cases.
I think the answer is coming close to yes.
And the scale of these models are minuscule compared to the human brain.
Can a jumping spider do everything thing a human can? No.
Can a jumping spider think? Yes.
Is a jumping spider conscious?