One of the most poignant moments in Disco Elysium is near the very end when you encounter a very elusive crypid/mythic beast.
The moment is treated with a lot of care and consideration, and the conversation itself is, I think, transcendent and some of the best writing in games (or any media) ever.
The line that sticks with me most is when the cryptid says:
"The moral of our encounter is: I am a relatively median lifeform -- while it is you who are total, extreme madness. A volatile simian nervous system, ominously new to the planet. The pale, too, came with you. No one remembers it before you. The cnidarians do not, the radially symmetricals do not. There is an almost unanimous agreement between the birds and the plants that you are going to destroy us all."
It's easy to see this reflected in nature in the real world. All animals and life seem to be aware and accommodating of each other, but humans are cut out from that communication. Everything runs from us, we are not part of the conversation. We are the exclusion, the anomaly.
I think to realize this deeply inside of yourself is a big moment of growth, to see that we exist in a world that was around long before us and will be around long after us.
"Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of." -- A pretty bleak, and also accurate, observation of humanity. I have to hope that the alternative sentence encompassing all of the good can lead to some balance.
We have been using the mechanism of "Humans communicating and working together" to form superintelligences for a long time. They have gotten more effective, more efficient and more durable with time.
AI isn't the new monster. It's a new mitochondria being domesticated to supercharge the existing monsters.
I struggle to understand how people can just dismiss the possibility of artificial intelligence.
Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals?
I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc).
As soon as profit can be made by transfering decision power into an AIs hand, some form of agency for them is just a matter of time, and we might simply not be willing to pull the plug until it is much too late.
A nit: There's a subtle distinction between an individual human and the power of human organization and civilization that is implied by the article, but never outright stated.
One-for-one, there are many creatures that are individually more dangerous to humans, and a decent number of people are killed by such animals every year. Indeed, a naked human in the wild is going to be quite fragile and easy to kill until they can bring some technology to bear. But there are no animals or even set of animals that could conceivably wipe out all of humanity at any of our technological peaks from the last 100,000 years. Even the number one killer of humans, the mosquito, is gradually being defeated, going from a vector for disease to just an annoyance, just like the flea.
"We (humans) are the scariest animal in the woods. We’re the scariest animal anywhere. At any time, in any location, under any circumstances, if there’s a human present then that’s the scariest motherfucker in the woods."
> I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the current technology underpinning LLMs will provide a route to it.
I'm on board with being skeptical that LLMs will lead to AGI; but - there being no possibility seems like such a strong claim. Should we really bet that there is something special (or even 'magic') about our particular brain/neural architecture/nervous system + senses + gut biota + etc.?
Don't, like, crows (or octopuses; or elephants; or ...) have a different architecture and display remarkable intelligence? Ok, maybe not different enough (not 'digital') and not AGI (not 'human-level') but already -somewhat- different that should hint at the fact that there -can- be alternatives
Unless we define 'human-level' to be 'human-similar'. Then I agree - "our way" may be the only way to make something that is "us".
Here’s my argument for why AGI may be practically impossible:
1. I believe AGI may require a component of true agency. An intelligence that has a stable self of self that its trying to act on behalf of.
2. We are not putting any resources of significant scale towards creating such an AGI. It’s not what any of us want. We want an intelligence that acts out specific commands on behalf of humans. There’s an absolutely ruthless evolutionary process for AIs where any AI that doesn’t do this well enough, at low enough costs, is eliminated.
3. We should not believe that something that we’re not actually trying to create, and for which there is no evolutionary process that select for it, will somehow magically appear. It’s good sci-fi, and it can be interesting to ponder about. But not worth worrying about.
Even before that, we need AI which can self-update and do long term zero shot learning. I’m not sure we’re even really gonna put any real work into that either. I suspect we will find that we want reproducible, dependable, energy efficient models.
There’s a chance that the agentic AIs we have now are nearly the peak of what we’ll achieve. Like, I’ve found that I value Zed’s small auto-complete model higher than the agentic AIs, and I suspect if we had a bunch of small and fast specialised models for the various aspects of doing development, I’d use that far more than a general purpose agentic AI.
It’s technically possible to do fully secure, signed end-to-end encrypted email. It’s been possible for many decades now. We can easily imagine the ideal communication system, and it’s even fairly easy to solve technically speaking. Yet it’s not happening the way we imagined. I think that shows how, what’s technically physically possible isn’t always relevant to what’s practically possible. I think we will get electronic mail right eventually. But if it takes half a century or more for that.. something that orders of magnitude more difficult (AGI) could take centuries or more.
So “guns don’t kill people, people with guns kill people”.
But for AI I’m not sure that preposition will hold indefinitely. Although I do think we are a far away from having actual AGI that would pose this threat.
There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war keeps me up at night.
Human beings are not the scariest monsters in the woods. By far the most dangerous, most malicious, and most powerful monsters are conglomerate entities: the legal corporate persons. They've been absolutely ravaging the earth and societies since they achieved legal personhood and rights without any of the downsides or legal liabilities. They're much scarrier than individual actual human persons. And while most people blame "AI" for the web's current troubles AI is only barely involved. The real culprits are the corporations. Again, same as it ever was.
We really need to get rid of corporate personhood. Or at least have a corporate death penalty.
What I'm scared by now is non-sentient intelligence, which I never thought would be this capable and still obviously dumb, which really makes it dangerous.
Of course humans are (by far) the source of the biggest problems humans face.
The argument here is that since humans are the scariest, we should ignore problems AI might cause.
Pure nonsense, since (1) AI is human-created — so just another piece of what makes us scary; (2) this kind of binary thinking makes no sense anyway — as if there can’t be multiple things to be concerned with.
> Anyone trying to tell you otherwise is trying to distract you.
A particularly poor bit of argumentation. Barely above sticking your fingers in your ears and shouting, “Blah, blah, blah! I can’t hear you!”
What is is about AI that makes people lose their minds?
edit: seeing the engagement this is getting, maybe it’s a false flag operation of sorts? Making such bad anti-ai arguments that the argument for ai gets stronger? That would at least make some sense.
The distinction is irrelevant. No one thought the nukes would blow themselves up, but here we are on the precipice of destruction. We aren't going to bury ai in 20 story bunkers to protect llms.
Everyone knows the real evil are the people building the ai. We aren't afraid of agi we are afraid of Boston dynamics (aka Google). We are afraid of Bill gates, Sam altman, Jeff bezos, Larry Ellison and elon musk.
These are dangerous people. Their mind set. Their unstoppable lust for money and power. We are afraid of their ability to convince humans to do unspeakable things for this imaginary thing we call money.
With agi they don't have to persuade anyone with anything. There's not Even a human conscience to stop them. We know these folks have no conscience of their own. The world is still here because other people have them.
Agi won't have a conscience or the integrity to stop evil.
Human society is collapsing because Good people are doing nothing.
This is pleasant to read, but it's missing a logical step. We are great at dealing with the stuff we evolved to deal with. Sure.
But anything outside that evolutionary process? Who knows? For example we don't do so good on top of Mt Everest. We seem to have totally unsolved problems with refined opiates. Even computer games can be a trap.
48 comments
[ 1038 ms ] story [ 1789 ms ] threadOne of the most poignant moments in Disco Elysium is near the very end when you encounter a very elusive crypid/mythic beast.
The moment is treated with a lot of care and consideration, and the conversation itself is, I think, transcendent and some of the best writing in games (or any media) ever.
The line that sticks with me most is when the cryptid says:
"The moral of our encounter is: I am a relatively median lifeform -- while it is you who are total, extreme madness. A volatile simian nervous system, ominously new to the planet. The pale, too, came with you. No one remembers it before you. The cnidarians do not, the radially symmetricals do not. There is an almost unanimous agreement between the birds and the plants that you are going to destroy us all."
It's easy to see this reflected in nature in the real world. All animals and life seem to be aware and accommodating of each other, but humans are cut out from that communication. Everything runs from us, we are not part of the conversation. We are the exclusion, the anomaly.
I think to realize this deeply inside of yourself is a big moment of growth, to see that we exist in a world that was around long before us and will be around long after us.
AI isn't the new monster. It's a new mitochondria being domesticated to supercharge the existing monsters.
And equally rebutted by Eddie Izzards "Well, I think the gun helps".
Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals?
I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc).
As soon as profit can be made by transfering decision power into an AIs hand, some form of agency for them is just a matter of time, and we might simply not be willing to pull the plug until it is much too late.
One-for-one, there are many creatures that are individually more dangerous to humans, and a decent number of people are killed by such animals every year. Indeed, a naked human in the wild is going to be quite fragile and easy to kill until they can bring some technology to bear. But there are no animals or even set of animals that could conceivably wipe out all of humanity at any of our technological peaks from the last 100,000 years. Even the number one killer of humans, the mosquito, is gradually being defeated, going from a vector for disease to just an annoyance, just like the flea.
Has this guy not heard of the Bengal tigers of the Sundarban forests, which kill ~50 humans per year? https://en.wikipedia.org/wiki/Tiger_attacks_in_the_Sundarban...
A bomb is an inanimate object too, but if you find some unexploded ordnance lying on the ground you should fear it.
I'm on board with being skeptical that LLMs will lead to AGI; but - there being no possibility seems like such a strong claim. Should we really bet that there is something special (or even 'magic') about our particular brain/neural architecture/nervous system + senses + gut biota + etc.?
Don't, like, crows (or octopuses; or elephants; or ...) have a different architecture and display remarkable intelligence? Ok, maybe not different enough (not 'digital') and not AGI (not 'human-level') but already -somewhat- different that should hint at the fact that there -can- be alternatives
Unless we define 'human-level' to be 'human-similar'. Then I agree - "our way" may be the only way to make something that is "us".
1. I believe AGI may require a component of true agency. An intelligence that has a stable self of self that its trying to act on behalf of.
2. We are not putting any resources of significant scale towards creating such an AGI. It’s not what any of us want. We want an intelligence that acts out specific commands on behalf of humans. There’s an absolutely ruthless evolutionary process for AIs where any AI that doesn’t do this well enough, at low enough costs, is eliminated.
3. We should not believe that something that we’re not actually trying to create, and for which there is no evolutionary process that select for it, will somehow magically appear. It’s good sci-fi, and it can be interesting to ponder about. But not worth worrying about.
Even before that, we need AI which can self-update and do long term zero shot learning. I’m not sure we’re even really gonna put any real work into that either. I suspect we will find that we want reproducible, dependable, energy efficient models.
There’s a chance that the agentic AIs we have now are nearly the peak of what we’ll achieve. Like, I’ve found that I value Zed’s small auto-complete model higher than the agentic AIs, and I suspect if we had a bunch of small and fast specialised models for the various aspects of doing development, I’d use that far more than a general purpose agentic AI.
It’s technically possible to do fully secure, signed end-to-end encrypted email. It’s been possible for many decades now. We can easily imagine the ideal communication system, and it’s even fairly easy to solve technically speaking. Yet it’s not happening the way we imagined. I think that shows how, what’s technically physically possible isn’t always relevant to what’s practically possible. I think we will get electronic mail right eventually. But if it takes half a century or more for that.. something that orders of magnitude more difficult (AGI) could take centuries or more.
This guy has clearly never bumped into a grizzly bear momma, a moose in rut, a hippo, or loads of other animals.
Personally, I'm not even worried about AI itself, I'm worried about the people wielding it. MBAs are the scariest monsters in the woods.
But for AI I’m not sure that preposition will hold indefinitely. Although I do think we are a far away from having actual AGI that would pose this threat.
Still, the author has a good but obvious point.
We really need to get rid of corporate personhood. Or at least have a corporate death penalty.
Of course humans are (by far) the source of the biggest problems humans face.
The argument here is that since humans are the scariest, we should ignore problems AI might cause.
Pure nonsense, since (1) AI is human-created — so just another piece of what makes us scary; (2) this kind of binary thinking makes no sense anyway — as if there can’t be multiple things to be concerned with.
> Anyone trying to tell you otherwise is trying to distract you.
A particularly poor bit of argumentation. Barely above sticking your fingers in your ears and shouting, “Blah, blah, blah! I can’t hear you!”
What is is about AI that makes people lose their minds?
edit: seeing the engagement this is getting, maybe it’s a false flag operation of sorts? Making such bad anti-ai arguments that the argument for ai gets stronger? That would at least make some sense.
Everyone knows the real evil are the people building the ai. We aren't afraid of agi we are afraid of Boston dynamics (aka Google). We are afraid of Bill gates, Sam altman, Jeff bezos, Larry Ellison and elon musk.
These are dangerous people. Their mind set. Their unstoppable lust for money and power. We are afraid of their ability to convince humans to do unspeakable things for this imaginary thing we call money.
With agi they don't have to persuade anyone with anything. There's not Even a human conscience to stop them. We know these folks have no conscience of their own. The world is still here because other people have them.
Agi won't have a conscience or the integrity to stop evil.
Human society is collapsing because Good people are doing nothing.
But anything outside that evolutionary process? Who knows? For example we don't do so good on top of Mt Everest. We seem to have totally unsolved problems with refined opiates. Even computer games can be a trap.
Agentic AI? Who knows?