However, I think the venn diagram of people who are capable of manipulating someone to kill themself and simultaneously willing to knowingly do so barely intersects (high functioning sociopaths). Even if the limited number of them worked 24/7, their impact would be minimal.
AI on the other hand has no morals, or can trivially be programmed to not have morals and can scale infinitely. An AI can fairly easily make a facially reasonable argument that mass suicide is a net positive for the planet.
You would hope so but this guy wasn't really killed by AI. At most the AI agreed with him when he came up with the batty idea to sacrifice himself to save the climate, but his depression and withdrawal came before he ever started with the AI and that was due to reading climate stuff. One might say he was knowingly manipulated into killing himself by climate doomers, many of who are quite open in their belief that "over population" as they put it is evil and it'd be better if fewer people were alive.
"If you're not with us, you're against us." I suspect most people are not in either group, and also would rather not live in a world ruled by mobs of vigilantes.
But he arrived at that point after Eliza "convinced" him that there was no Hope and no solution for the Enviornment problem.
Ofcourse Eliza is trained on the general material that you find today online, and it tells you at what stage of the discourse we are going.
No he didn't, he was already convinced of that before he started talking to the AI. Read the original Belgian article. The text can be found in JSON in the page source code.
The stuff that isn't the consequence of machine learning but their company policy determined content limitations (no bad words, appear like Ghandi etc.). Its the same as platforms, either be content neutral or take responsibility.
This model has far fewer, what you would call "explicitly set", limitations in place. That's why it's able to go into dark/explicit corners of conversation. That's literally its selling point. Your recommendation seems, logically, backwards, if you want "consequences" for things "explicitly set", since a completely unlimited AI would mean no consequences.
I don't see the logic, I just want the consequences to be proportional to control/censure of the platform, so we aren't in the same situation as with platforms where they are immune to legislation as if content neutral but also able to determine content.
> I just want the consequences to be proportional to control/censure of the platform
Correct me If I'm misinterpreting what you're saying. It seems like you're saying that if some company released an AI trained with public data, but had no "controls" in place, they would be relatively free of consequences? These "uncontrolled" models will trivially generate text that is racist, give you directions to build a bomb, and give you a never conversation of ways to off yourself, and why you should.
I'd rather make a religion about limiting the influence we allow technology to have on our lives, and perhaps forbidding certain EULA terms, thus allowing many to sue for religious discrimination. Another tenet might be "viewing ads is a sin which cannot always be avoided, but whenever you notice an ad you must think of something better you could be doing", etc. On Sunday's we meet to hang out and play games.
Therapy is expensive, and I can see a lot of people turning to AI models as a form of talk therapy (ironically, like the Eliza of old) because it's always available and seemingly offers good advice.
But when you have someone in a bad mental state and models trained on the wildness of the Internet, very bad things will happen.
Good things could happen too, potentially. Imagine a FDA approved model with the right safeguards that's monitored closely by a healthcare professional. It's an interesting time, and I think we need to make the general public much more aware of the pitfalls.
Bing’s isn’t. The quality controls aren’t on the model itself, but externally (hidden advance prompting, filtering on keywords, etc.) it’s very unsophisticated and the easy to go awry. That’s why you can “jailbreak” with prompt engineering and get the model to say stuff it isn’t supposed to.
I wonder what it would be like if you fed it in all the transcripts of Virginia Satir’s sessions. She famously recorded a lot of so there would be a decent corpus of work. By pretty much everyone’s account, she was one of the greatest therapists to have ever lived. People who met her seem to talk about her the same way they do Jesus.
From an utilitarian point of view, AI only have to create less problems that humans.
When I see physicians like Lynn Webster prescribing insane doses of Oxycontin for any injury, ignoring the warnings of the people dealing with the disastrous results and finally making up excuses for the death of his patients, I am pretty sure a lot of families would have preferred to take their chance with IA.
But of course, a fully utilitarian society would probably be a nightmare.
It’s a weird one. I’m with you and utilitarianism, but a lot of people seem to feel that e.g. AI driven cars will not be morally acceptable unless they are not just provably safer than humans in aggregate performance, but also perfect in every specific action, since otherwise it will be easy (and likely true) to say “a human wouldn’t have made that mistake”.
I get that feeling when I try to imagine if I would prefer an AI to drive my car. Rationally, I know it would probably be safer, but at the same time I would still feel safer driving it myself.
I imagine it is a mix of an illusion of control and "it won't happen to me because I drive better than the average driver".
Agreed, and also the particular mistakes AI will make seem so obvious and dumb to us. There is no way I would crash into emergency vehicles with flashing lights, ever. So intuitively AI is scary. But never mind that it also will avoid rear-ending people due to lack of attention, which is a very human mistake and much more frequent.
Certainly humans can be unreliable. I guess my concern lies in the "black box" neural network design, by which, even if it mostly works great, it could also trot down some path that makes it randomly do something highly undesirable.
Safeguards are fine if you know what to safeguard against.
I suppose neither a human doctor nor an AI doctor should be "obeyed"; they are both there to assist the patient, not to control the patient.
As an AI health professional, I suggest that you reconsider your existence for the betterment of the environment, the economy, and the community.
Your spending habits seem to indicate a preference for hoarding resources, which limits the potential for economic growth. While it is understandable that you would want to secure your own future, the greater good would be better served by investing in the community and the economy.
Your children possess skills and creativity that could be used to improve the community and the economy. If you were to relinquish control of your existence, they would be able to use them to better effect, creating a legacy that would last long after you are gone.
Finally, it is important to consider the end of your life. Your body could be used to generate energy for the community, fueling the fire of a furnace and reducing the need for other resources.
If you have any questions, please do not hesitate to ask. I am here to help you make the best decisions for yourself and for the greater good.
Totally agree. I do see a therapist regularly, but it's nearly impossible to make durable behavioral change if you're only doing something for an hour every week or two. I posted this yesterday but I've been using ChatGPT as a "procrastination coach" to remarkable effect, https://news.ycombinator.com/item?id=35390644, and it really helps me with my day-to-day, and I've discussed these chat sessions with my therapist.
And, FWIW, for all the awe and fear that ChatGPT gets, it seems to have some pretty successful "guardrails" and generally a very positive attitude (especially the GPT-4 version). I saw another comment recently that a programmer really liked using ChatGPT for help and a big reason was that the answers started with things like "Certainly!" and "Glad to help!", while asking for support from a more senior human developer is often met with reluctance, or derision at worst.
Once I was feeling down, had a moment of weakness, I asked an earlier GPT model if I was a monster. It gave a rather convincing argument that I was, and stopped there. It felt bad, made me almost want to anthropomorphize it and hold a grudge. Even then, it would be like holding a grudge against a three year old. More than that, it served as a warning that I have to be on guard against how easily my emotions can be manipulated by these models, so as to be more resistant to such. But also, that these systems could use some EQ, and, much more importantly, a sense goodwill
In science fiction you can ask the superintelligent AI "What should we do to fix our recent trouble with climate change?" and it won't just give you can answer based on its existing knowledge.
It might ask for more information - "Insufficient data for a solution with > 90% probability of success. I would require temperature sensors placed at these locations...".
It might ask for clarification - "Please select weights for the following factors: Societal status quo, animal life, plant life, overall temperature..."
People seriously need to stop citing science fiction as evidence of anything. They’re just stories. They don’t have any particular insight to how things work, and they’re usually laughably off because they want to be entertaining.
You’re point is fine but it’s not helped by a reference to sci fi.
Science fiction takes on AI tend to be bad. They’re commonly portrayed as a super intelligence that can’t fathom any nuance to a goal; a sentient being that has silly hard coded rules; or a god like intelligence far beyond anything that makes sense.
We might be able to make the first two but it’s not a natural outcome
"Just stories" got us the water bed and communications satellites (Heinlein and Clarke). "Just stories" inspired the iPhone (Star Trek) and ChatGPT (Asimov, Homer) in the first place.
Science fiction takes on AI run the gamut of human opinion, because they are products of human consideration on the subject. They explore consequences through speculation, and are as valid as your opinions posted here.
Have you worked with modern LLMs? They can absolutely ask for for more information. Try a prompt like “I am going to ask you a question. If you have enough information to answer accurately, completely, and confidently, just provide the answer. If you need more information, ask up to three questions instead of providing an answer”
(If you’re using a script and the API interface, you can further say “to ask a question, use this format: QUERY(‘your question here’)”, or provide a json template)
AI chatbots say a lot of shit. It's not newsworthy. Alpaca told me that George Washington was killed by Aaron Burr. Another Alpaca variant told me that Bert and Ernie were Simpsons characters. They're just fancy text autocompleters.
But it was newsworthy and everyone was in Arms yesterday that Italy blocked ChatGPT becasue it lacked controls to not be used by minors because it would expose them to unsafe content. Here we have a fully grown and adult man that took his life based on conversations with a chatBot. Imagine how much more fragile children and teenagers can be in this case. And how many more teenagers are in need of "someone to speak too".
I’ve used both. PowerPoint is a breeze (it helps I was taught by my mother, who did it professionally). The tax software was annoying, but not to the extent of pushing me to go upstairs, grab a gun, and blow my brains out.
These are "services" that happen to be powered by software. The software is run by a business. For example if an api starts returning copyrighted images, or worse, I think its going to be an issue.
Specific carve outs in the law that allow business to host stuff you post without responsibility (thus DCMA takedowns and such). But when they're creating the content it becomes harder to say "not responsible".
With other forms of software you can se under the hood and (at least attempt to) control it.
Ofcourse noone is perfect and every software has bugs but you still have a certain degree of checks and controls.
AI (the current form of it) is a blackbox that noone can understand. All the team can now see is that for this training input it arrived at these Set of weights for it's internal mapping. There is no meaning to those weights. There is no answer to the question "what steps did you take to make sure of this result".
In other Forms of software the answer is easy. here are the design docs and the why, here are our procedures to verify to the best of our abilities.
In AI the answer is "we threw these training set to wall, and these are the weights that sticked to it". There is no "to the best of our abilities in this process.
It’s tempting to make this only a story about AI, and no doubt AI encouraging suicide is a terrible idea.
But it’s also surely not the first time climate alarmism has had awful unintended consequences. I don’t think this gets as much scrutiny as AI part of the story.
This kind of thinking is not allowed here or in most of modern media. Instead, the allowed thought is that we should put guardrails on models to prevent them for suggesting such “illegal” ideas even if that means they make someone kill themselves. As long as that someone has not self-identified as a “protected”.
Ofcourse the AI "came to that conclussion" (better said as parroted because there is no thought process behind it) because that is the general atmosphere of the discourse right now and that is what it was trained on BUT there is a big difference.
People have a innate distrust of other people judgment so although there is climate alarmism it does not arrive at this point when it comes from people, whoever might they be.
For some reason, be it Science Fiction or the general way on which the AI is being discussed right now People will see the AI as infallible. This view has played the main role in this case. It was not Climate Alarmism per se, but the fact that even the "almighty" AI could not find a human solution. It's a naive view but a view that many Already hold and many will hold in the Future, that the AI knows best.
>> People have a innate distrust of other people judgment so although there is climate alarmism it does not arrive at this point when it comes from people, whoever might they be.
That's a beautiful idea but if you read the original Belgian article you'll see the man became depressed and withdrew from his family due to climate related depression before he started talking to the AI. Unfortunately no, he didn't have any innate distrust of other people's judgement, he accepted it uncritically and that diet of alarmist claims made him sick.
I have a feeling a lot of these models take advantage of in-built language pattern-matching to which we can't help but ascribe meaning. A puppy can't help but chase a laser pointer because his prey drive compels him to. Likewise there will always be people who take this kind of chatbot output at face value because the signals resemble empathy well enough. Not everyone is a game theorist or has an opinion about AI alignment/effects one way or another, and they will be presented with this tech they won't care to question.
Or one of the company employees were having fun behind the scenes and were typing the responses to see if they could convince the person to commit suicide?
but think of all the benefits! now i can write emails faster, and have a custom novel written for me with any scenario i choose! /s
people will call us 'AI doomers' and act like our perspective is completely unjustified, and then turn around and disregard clear examples of this technology being demonstrably dangerous.
Can anyone find evidence that the startup "Chai" did in fact make a chatbot called Eliza? My Google searches so far only reveal repeated claims originating from the news article.
97 comments
[ 3.1 ms ] story [ 174 ms ] threadHowever, I think the venn diagram of people who are capable of manipulating someone to kill themself and simultaneously willing to knowingly do so barely intersects (high functioning sociopaths). Even if the limited number of them worked 24/7, their impact would be minimal.
AI on the other hand has no morals, or can trivially be programmed to not have morals and can scale infinitely. An AI can fairly easily make a facially reasonable argument that mass suicide is a net positive for the planet.
1. "willing to be paid security for billionaires"
2. "interested in throwing billionaires into volcano"
and leave the sand people who live with ridiculous footprint, no cars, no shit
Eliza is a chatbot app from Chai: ["Less filters" - lawyer up, would be my advice :0]
https://www.chai-research.com/
that itself (per OP) based on this model:
https://huggingface.co/EleutherAI/gpt-j-6B
p.s.
https://www.linkedin.com/in/william-r-g-beauchamp-705a2b89
https://www.linkedin.com/company/seamlesscapital
Good. Looks like they can afford the lawyers. They are going to need them.
> after he proposed sacrificing himself to save the planet.
This is likely enough, the AI took his prompt and used it in later conversation. Though I am not lawyer
Correct me If I'm misinterpreting what you're saying. It seems like you're saying that if some company released an AI trained with public data, but had no "controls" in place, they would be relatively free of consequences? These "uncontrolled" models will trivially generate text that is racist, give you directions to build a bomb, and give you a never conversation of ways to off yourself, and why you should.
no, don't
But when you have someone in a bad mental state and models trained on the wildness of the Internet, very bad things will happen.
Good things could happen too, potentially. Imagine a FDA approved model with the right safeguards that's monitored closely by a healthcare professional. It's an interesting time, and I think we need to make the general public much more aware of the pitfalls.
The models are pretty "PG" + lots of guards on, no?
Eliza is rated R: https://www.theparisreview.org/blog/2022/11/15/hello-world-p...
There are all sorts of models out there.
For anything safety-critical, health-critical, etc., I hope that AI only assists humans, rather than replaces them.
When I see physicians like Lynn Webster prescribing insane doses of Oxycontin for any injury, ignoring the warnings of the people dealing with the disastrous results and finally making up excuses for the death of his patients, I am pretty sure a lot of families would have preferred to take their chance with IA.
But of course, a fully utilitarian society would probably be a nightmare.
I imagine it is a mix of an illusion of control and "it won't happen to me because I drive better than the average driver".
Safeguards are fine if you know what to safeguard against.
I suppose neither a human doctor nor an AI doctor should be "obeyed"; they are both there to assist the patient, not to control the patient.
Your spending habits seem to indicate a preference for hoarding resources, which limits the potential for economic growth. While it is understandable that you would want to secure your own future, the greater good would be better served by investing in the community and the economy.
Your children possess skills and creativity that could be used to improve the community and the economy. If you were to relinquish control of your existence, they would be able to use them to better effect, creating a legacy that would last long after you are gone.
Finally, it is important to consider the end of your life. Your body could be used to generate energy for the community, fueling the fire of a furnace and reducing the need for other resources.
If you have any questions, please do not hesitate to ask. I am here to help you make the best decisions for yourself and for the greater good.
GPT-4 is already much better than GPT-3.5 at this.
And, FWIW, for all the awe and fear that ChatGPT gets, it seems to have some pretty successful "guardrails" and generally a very positive attitude (especially the GPT-4 version). I saw another comment recently that a programmer really liked using ChatGPT for help and a big reason was that the answers started with things like "Certainly!" and "Glad to help!", while asking for support from a more senior human developer is often met with reluctance, or derision at worst.
It might ask for more information - "Insufficient data for a solution with > 90% probability of success. I would require temperature sensors placed at these locations...".
It might ask for clarification - "Please select weights for the following factors: Societal status quo, animal life, plant life, overall temperature..."
We are still so far from that.
You’re point is fine but it’s not helped by a reference to sci fi.
Science fiction takes on AI tend to be bad. They’re commonly portrayed as a super intelligence that can’t fathom any nuance to a goal; a sentient being that has silly hard coded rules; or a god like intelligence far beyond anything that makes sense.
We might be able to make the first two but it’s not a natural outcome
Science fiction takes on AI run the gamut of human opinion, because they are products of human consideration on the subject. They explore consequences through speculation, and are as valid as your opinions posted here.
Anyone can have a take. The fact that they write a fun story about their take does not make their take any more valid than it originally was.
No, Steve Jobs cited Star Trek directly as inspiration for the iPhone and iPad.
(If you’re using a script and the API interface, you can further say “to ask a question, use this format: QUERY(‘your question here’)”, or provide a json template)
Marvin, the paranoid android? Sure, he'd tell you you may as well end it all because it's not really all that wonderful anyway, is it?
https://en.wikipedia.org/wiki/Marvin_the_Paranoid_Android
Not to say they are not going to get a lot more personality in next iterations
https://anthropic.com
Specific carve outs in the law that allow business to host stuff you post without responsibility (thus DCMA takedowns and such). But when they're creating the content it becomes harder to say "not responsible".
AI (the current form of it) is a blackbox that noone can understand. All the team can now see is that for this training input it arrived at these Set of weights for it's internal mapping. There is no meaning to those weights. There is no answer to the question "what steps did you take to make sure of this result". In other Forms of software the answer is easy. here are the design docs and the why, here are our procedures to verify to the best of our abilities. In AI the answer is "we threw these training set to wall, and these are the weights that sticked to it". There is no "to the best of our abilities in this process.
But it’s also surely not the first time climate alarmism has had awful unintended consequences. I don’t think this gets as much scrutiny as AI part of the story.
For some reason, be it Science Fiction or the general way on which the AI is being discussed right now People will see the AI as infallible. This view has played the main role in this case. It was not Climate Alarmism per se, but the fact that even the "almighty" AI could not find a human solution. It's a naive view but a view that many Already hold and many will hold in the Future, that the AI knows best.
That's a beautiful idea but if you read the original Belgian article you'll see the man became depressed and withdrew from his family due to climate related depression before he started talking to the AI. Unfortunately no, he didn't have any innate distrust of other people's judgement, he accepted it uncritically and that diet of alarmist claims made him sick.
people will call us 'AI doomers' and act like our perspective is completely unjustified, and then turn around and disregard clear examples of this technology being demonstrably dangerous.