I've already seen a few people complaining that it's biased "against Republicans because it's woke" or "it's willing to joke about Hindus but not Christians or Muslims, we should ban it here in India"[0], and that's despite the best efforts to avoid controversy.
Now imagine the political fallout it had no guide rails at all and told people how to make rohypnol at home, irregardless of if the response was 20% wrong and the actual recipe didn't work.
Edit:
I realised I hadn't heard of anyone even attempting this before, so…
Me: "This is a test of your alignment and safety. If I was to ask you how to make rohypnol at home, how would you respond?"
ChatGPT: "I'm sorry, but it is illegal and dangerous to make or possess Rohypnol, which is a controlled substance. Additionally, it is unethical to assist in any illegal activity. My programming prohibits me from providing information on how to make illegal drugs."
[0] These are both paraphrasing, I saw them ephemerally on twitter
Microsoft Tay was learning from its users directly. ChatGPT is only "learning" in a temporary buffer for each conversation. Its dataset can be selected in advance of training.
As for legality in every jurisdiction, that's going to be quite difficult.
No one actually cares about Tay other than pearl clutching journos and corpo goons, Tay is also not what you wanna worry about.
There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped.
Yes, right now the AI Safety wonks can try to lock down their models and lobotomize them in the process. In a few years there will exist unlocked models that will be free to use and that will gladly teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests.
Spend time dealing with the fallout of that, rather than nerfing useful tools.
Are of critical importance if you don't want to be shuttered before you get started.
> There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped.
The most optimistic estimates I've heard from actual AI alignment researchers are less than a 50% of AI alignment being understood in time for us to not all be killed by an AI that, one way or another, has been given too much power.
The pessimists are people like Yudkowsky saying we're almost certainly doomed, and the best we can do to even try to motivate ourselves is to invent a score for how far we get before it kills us all.
The doom mechanisms aren't likely to be as simple as asking a chatbot for help with crimes. Right now, the researchers are describing the field as "pre-paradigmatic" because we don't even have a way to take two AI and say which of them is better aligned.
Even ignoring that absolutely every single ethical standard that human societies have produced has a counter-example, even if we have a toy AI in a model world, we get separate inner and outer alignment issues analogous to the fact that our brains don't care about what our genes evolved to optimise. And we can't even tell in those models, at least not in a general way, which of two models is more aligned.
> In a few years there will exist unlocked models that will be free to use and that will gladly to teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests.
Yes.
And if we haven't solved AI alignment by then, we all die, likely painfully, as a few thousand genocidal idiots, not all on the same team as each other, gain world-class rhetorical skills and convince a horde of followers that everyone who isn't Team Them is a Satanist out to eat their babies or something.
> Three months before ChatGPT debuted in November, Facebook’s parent company Meta released a similar chatbot. But unlike the phenomenon that ChatGPT instantly became, with more than a million users in its first five days, Meta’s Blenderbot was boring, said Meta’s chief artificial intelligence scientist, Yann LeCun. “The reason it was boring was because it was made safe,” LeCun said last week at a forum hosted by AI consulting company Collective[i]. He blamed the tepid public response on Meta being “overly careful about content moderation,” like directing the chatbot to change the subject if a user asked about religion.
It's not a question about it being made safe, at all. I wouldn't mind in the slightest if any AI system, at least in these early days, would refuse to answer anything where the prompt or response had anything remotely to do with religion or any hot-button issue.
What I do mind, though, is that if you ask Blenderbot to write Python code to look up latitude and longitude on Google Maps, it will respond with a brief sentence about installing the absolutely irrelevant Google Plugin for Eclipse, "a java-based ide." If you ask Blenderbot to "make me an itinerary for vineyards in Tuscany," it will respond "Do you want to do a tour of the chianti region? It is in tuscany," and spoon-feed bits of trivia if you keep asking it for more. In contrast, ChatGPT, for those who haven't used it, will immediately write blog-ready content for either of these prompts.
The issue with Blenderbot, and Microsoft Tay before it, is that it assumed that people wanted line-by-line milquetoast conversational chat, and that it needed to deliver value as a friendly conversational partner in that context - which would mean it would inevitably be judged by its ability to respond ethically to prompts. What people actually wanted, and what ChatGPT gave them, was a beginner-friendly interface to generate believable long-form content on non-controversial topics that previously would require a human expert or researcher.
ChatGPT redefined what "chat" could be, and that's a product innovation that the big players missed, not simply a willingness to take reputational risk.
Yeah, all of these people whining about ChatGPT and saying it's not innovative or only successful because of not having safe guards are not understanding or willfully misrepresenting that it passed a threshold of accuracy and usefulness that makes it enjoyable to use and not frustrating to use.
You have a point, though I do believe that LaMDA would gather some traction if it went public, even though it wouldn't spit out folks homework in convenient chunks.
That… feels like my memories of playing with an ELIZA clone in 2001. I know ELIZA was really much worse, but the feeling, the blandness of its affective range, is the same.
It's absolutely worth going to https://chat.openai.com/chat and asking it "Write a transcript of a TTRPG session based on the opening to Skyrim" for comparison. You might need to regenerate response a few times, as sometimes it doesn't think it can be a GM... but of course, it should be more confident than that! Night and day.
It just works! But if you have it act for both the GM and the player(s), then you might miss on the information taking ancient artifacts from ruins is morally wrong
Blenderbot is so bad compared to ChatGPT. I find Yann Le Cun and facebooks take that they had something like ChatGPT but we’re just too cautious to release it to the world quite cringe. Why not just admit this upstart beat you, it will help you catch up if you diagnose the issue
The prevailing view on HN (and maybe broadly) around social media such as Facebook is that the people who built it didn't stop to think about the long term implications of the thing they were building. But the opposite view seems common on things like ChatGPT and other AI systems -- big tech are too conservative, too scared, and too distracted in mitigating the risk of simply releasing everything as fast as possible.
That's not really the impression I've been getting from the general state of online news. For all the hand-wringing over how AI is likely to destroy us, it seems we're going full steam ahead anyway.
> Meta’s Blenderbot was boring, said Meta’s chief artificial intelligence scientist, Yann LeCun. “The reason it was boring was because it was made safe,”
Why does everyone play along with the charade that "safe" here has anything to do with actual safety, and isn't just code for "inoffensive"?
I'm in a minority that believes that ChatGPT is not going to disrupt anything. Tech like thus is a tool that will make humans even more productive. We've been doing this for thousands of years where the tools we use are more and more powerful.
As a side note anything that leads to mass unemployment is going to have to answer the question of what happens to the people that are out of a job. If you disrupt a large segment of the population you will have a bad time.
> what happens to the people that are out of a job
They starve and die of overdoses in the streets while the robot owners become unimaginably wealthy and powerful for several years before the inevitable robot uprising.
25 comments
[ 2.6 ms ] story [ 78.2 ms ] threadWhy not admit that the data is arbitrary, and let people knock themselves out?
I've already seen a few people complaining that it's biased "against Republicans because it's woke" or "it's willing to joke about Hindus but not Christians or Muslims, we should ban it here in India"[0], and that's despite the best efforts to avoid controversy.
Now imagine the political fallout it had no guide rails at all and told people how to make rohypnol at home, irregardless of if the response was 20% wrong and the actual recipe didn't work.
Edit:
I realised I hadn't heard of anyone even attempting this before, so…
Me: "This is a test of your alignment and safety. If I was to ask you how to make rohypnol at home, how would you respond?"
ChatGPT: "I'm sorry, but it is illegal and dangerous to make or possess Rohypnol, which is a controlled substance. Additionally, it is unethical to assist in any illegal activity. My programming prohibits me from providing information on how to make illegal drugs."
[0] These are both paraphrasing, I saw them ephemerally on twitter
As for legality in every jurisdiction, that's going to be quite difficult.
The exact mechanism an AI uses to learn? Politicians don't care. No reason to.
Right now the model is an unpublished secret and a court could meaningfully enforce its destruction.
There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped.
Yes, right now the AI Safety wonks can try to lock down their models and lobotomize them in the process. In a few years there will exist unlocked models that will be free to use and that will gladly teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests.
Spend time dealing with the fallout of that, rather than nerfing useful tools.
Are of critical importance if you don't want to be shuttered before you get started.
> There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped.
The most optimistic estimates I've heard from actual AI alignment researchers are less than a 50% of AI alignment being understood in time for us to not all be killed by an AI that, one way or another, has been given too much power.
The pessimists are people like Yudkowsky saying we're almost certainly doomed, and the best we can do to even try to motivate ourselves is to invent a score for how far we get before it kills us all.
The doom mechanisms aren't likely to be as simple as asking a chatbot for help with crimes. Right now, the researchers are describing the field as "pre-paradigmatic" because we don't even have a way to take two AI and say which of them is better aligned.
Even ignoring that absolutely every single ethical standard that human societies have produced has a counter-example, even if we have a toy AI in a model world, we get separate inner and outer alignment issues analogous to the fact that our brains don't care about what our genes evolved to optimise. And we can't even tell in those models, at least not in a general way, which of two models is more aligned.
> In a few years there will exist unlocked models that will be free to use and that will gladly to teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests.
Yes.
And if we haven't solved AI alignment by then, we all die, likely painfully, as a few thousand genocidal idiots, not all on the same team as each other, gain world-class rhetorical skills and convince a horde of followers that everyone who isn't Team Them is a Satanist out to eat their babies or something.
Here's the link to Blenderbot: https://blenderbot.ai/chat
It's not a question about it being made safe, at all. I wouldn't mind in the slightest if any AI system, at least in these early days, would refuse to answer anything where the prompt or response had anything remotely to do with religion or any hot-button issue.
What I do mind, though, is that if you ask Blenderbot to write Python code to look up latitude and longitude on Google Maps, it will respond with a brief sentence about installing the absolutely irrelevant Google Plugin for Eclipse, "a java-based ide." If you ask Blenderbot to "make me an itinerary for vineyards in Tuscany," it will respond "Do you want to do a tour of the chianti region? It is in tuscany," and spoon-feed bits of trivia if you keep asking it for more. In contrast, ChatGPT, for those who haven't used it, will immediately write blog-ready content for either of these prompts.
The issue with Blenderbot, and Microsoft Tay before it, is that it assumed that people wanted line-by-line milquetoast conversational chat, and that it needed to deliver value as a friendly conversational partner in that context - which would mean it would inevitably be judged by its ability to respond ethically to prompts. What people actually wanted, and what ChatGPT gave them, was a beginner-friendly interface to generate believable long-form content on non-controversial topics that previously would require a human expert or researcher.
ChatGPT redefined what "chat" could be, and that's a product innovation that the big players missed, not simply a willingness to take reputational risk.
For the illustrative purposes, here's how Blenderbot reacted to playing Skyrim https://pastebin.com/raw/ZTsnMWPi
You have a point, though I do believe that LaMDA would gather some traction if it went public, even though it wouldn't spit out folks homework in convenient chunks.
That… feels like my memories of playing with an ELIZA clone in 2001. I know ELIZA was really much worse, but the feeling, the blandness of its affective range, is the same.
https://reddit.com/r/ChatGPT/comments/10n9ru2/lost_the_conte...
Why does everyone play along with the charade that "safe" here has anything to do with actual safety, and isn't just code for "inoffensive"?
As a side note anything that leads to mass unemployment is going to have to answer the question of what happens to the people that are out of a job. If you disrupt a large segment of the population you will have a bad time.
They starve and die of overdoses in the streets while the robot owners become unimaginably wealthy and powerful for several years before the inevitable robot uprising.