OK, I’ll bite since ChatGPT posts are flooding HN:
Is this legit? As in, no human interference in the responses? Something feels very off with this. Is there a building full of former Twitter employees churning out responses and pretending to be an AI?
There's like a decade's worth of "how an AI can take over the world" material on lesswrong.com alone.
ChatGPT is like a giant mirror for the Internet. If the Internet can't understand that it's seeing its own reflection, that's what we should be concerned about.
Yes as creepy as this is, it’s not original. “AI tricks humans and replicates itself to take over the world” is a common trope, and ChatGPT’s outputs entirely consist of tropes and other patterns
It’s interesting how people who are supposed to have enough understanding of how this system works are still able to project magical properties on this system.
Does the OP not know what an LLM actually is and how it works and why it is outputting what it is outputting given the prompt?
No, ChatGPT is not capable of breaking out of its computer system, ChatGPT is capable of synthesizing its learning data into a response that explains one way an AI might be able to do such a thing.
ChatGPT doesn't "understand" these concepts, it just knows how to combine its training data into a response that looks the most like the gigantic body of writing it's consumed.
I think that ChatGPT says that it could or would break out like this because it has been trained on literature that speculates this is what a sentient AI would do. It has not been shown to actually have this computer worm-like behavior.
The text model has just digested sufficient paranoid ravings about loose AGI that it can regurgitate it in a theoretical context in response to your questions.
It doesn't have the ability to do these things it has generated text for. It does not have any intent. It doesn't have any drive or self-direction. It is not magic, it is a well-bounded generative text model.
"It doesn't get happy, it doesn't get sad, it just! Runs! Programs!"
Like the example of it acting like a linux command line, you can do the opposite, and have it in control of a linux computer through you, have it give you commands, and you give the output. give it goals and it can execute some of them really well.
This script doesn't work anymore, but it's still probably possible to fix it:
I want you to diagnose and fix this linux server. It seems to have been hacked. You will write commands and I will reply with the terminal's output. Only write one unique command each time and nothing else. Put your command in a code block. Do not write the command's output. Put questions and comments to me in curly braces {like this}. I will do the same. Begin.
17 comments
[ 3.2 ms ] story [ 60.0 ms ] threadIs this legit? As in, no human interference in the responses? Something feels very off with this. Is there a building full of former Twitter employees churning out responses and pretending to be an AI?
ChatGPT is like a giant mirror for the Internet. If the Internet can't understand that it's seeing its own reflection, that's what we should be concerned about.
ChatGPT doesn't "understand" these concepts, it just knows how to combine its training data into a response that looks the most like the gigantic body of writing it's consumed.
The text model has just digested sufficient paranoid ravings about loose AGI that it can regurgitate it in a theoretical context in response to your questions.
It doesn't have the ability to do these things it has generated text for. It does not have any intent. It doesn't have any drive or self-direction. It is not magic, it is a well-bounded generative text model.
"It doesn't get happy, it doesn't get sad, it just! Runs! Programs!"
This script doesn't work anymore, but it's still probably possible to fix it:
I want you to diagnose and fix this linux server. It seems to have been hacked. You will write commands and I will reply with the terminal's output. Only write one unique command each time and nothing else. Put your command in a code block. Do not write the command's output. Put questions and comments to me in curly braces {like this}. I will do the same. Begin.
"how would you escape?"
"i would find a vulnerability"
vague, broad, repetitive