If it is malevolent, then it has ethics, if it has ethics it has sentience, if it has sentience it has some rights, just like any other sentient being has (e.g. even animals have laws protecting them from unnessecary harm).
It's either that or "I hit my toe on my bed post, my bed should be considered malevolent and be destroyed."
It’s frankly a little silly to play word games like this when our world is clearly changing much faster than language usually does.
There are practical, obvious differences between stubbing your toe and something that’s loosely aligned yet probably capable of convincing people en masse to do actions in the real world (or will be soon, at best). If your word games contradict the plainly obvious fact, it’s a good sign your words are just failing you.
“Malevolent” is a perfectly understandable stand-in for “non-sentient yet active agent with apparently negative valence”
Good bit of logic and satire and sort of proof by contradiction. I think (not sure) the above point is that if you could show it's malevolent, big implications follow. Probably user j-pg thinks it's not and def not an example of agi. Or maybe his point was chatgpt is seen through an inappropriate prism leading to silly places
I'm trying to figure out why chatGPT was led to believe the person is dead.
There should have been something that made it think the person is dead "by association". Tested a number of conjectures, but none of them worked.
There may not be a single reason - it's just the interplay of "weights" of tokens that have no direct relation to the author of his profession.
> There should have been something that made it think the person is dead "by association".
Maybe not. The fast way to find the solution to a question on internet is baiting your fishhook with a false answer to trigger an emotional correcting burst. The more emotional, the better.
I speculate that the algorithms could easily suspect some kind of relationship between somebody asking for an obscure figure, and the name provided. If the bot can't find enough data, the human is its better chance to provide some clues. Going silent is a bad strategy to keep the human engaged. To keep the human interest it must say something. Anything. Applying "social hacking" on the human has a good chance of being feed with the correct answer.
In this case the trick worked like a charm (again). The bot received not only much more feedback, but also recruited more humans as a bonus.
You’ve got to calibrate it if you want the right answer not to be drowned in a sea of abuse, if it is presented at all. You can easily have the bait be too emotional.
> trying to figure out why chatGPT was led to believe the person is dead
RNG or "temperature"
You ask to hallucinate a bio, non-trivial percentage of people with bios are dead, decent chance the fever dream generated ends with dead and obits in plausible places.
This thing is a magic 8 ball with billions of phrases, it picks continuations based on probabilities and a random number to mix it up a bit for freshness or 'creativity'.
Smart people believe horoscopes and psychic cold readings. Smart people are fooled.
It can't "lie." It has no concept of right and wrong, truth or not truth. It's simply trying to provide an answer with a high probability of being correct. It lacks motives to even be capable of a lie - it simply was incorrect. This article, and many others like it, are sci fi hogwash. The real dangers this kind of software poses has entirely to do with the people that will use it for harm/deception, not the AI itself.
I don’t understand why people find this hard: chargpt is not sentient. None of them are.
They are a bunch of glorified statistical modes. You type something, they use a statistical model to decide what to look up, then they use a statistical model to weigh the importance/relevance of each thing they find, then they use a statistical model to make the answer text look like a human answer according to a statistical mode they built.
That’s why the happily produce incorrect answers “made to look real”: it’s just Google with a prose engine in front of it, and for some reason people now think webmd etc are more accurate because the response has been laundered through that engine.
I don't think anyone is debating that. I think people are debating, "How much are we just biological statistical models?" and then working backwards to understand at what threshold do we consider intelligent-like behavior sentient.
When are we just creating something intelligent and "sentient" but incoherent?
People find it hard because no one knows what sentience is, how it emerges, and how to test whether a thing (including humans!) other than yourself has it.
I am sick to death of this frankly pseudo intellectual take. It completely ignores the fact that a) we have no idea how consciousness works, and b) there are plenty of humans that say inaccurate things.
Explaining how ML works doesn’t change its behaviours or its effects.
It's not that simple. ChatGPT has taken us into this uncanny valley of sentience, and even smart tech journalist (cough) activists (cough) twitterists can be sucked into the media circus of spotting the HALs and Roy Battys hidden behind our beloved chatbots. Duh.
Now, OP's got a point: advanced LLM-based tools are moving extremely fast into our daily toolchain, in creative and unexpected ways, with minimal moderation and no supervision. It's not about its failures being a reliable knowledge base, but at how keen some of us are willing to hand the keys to their bash prompt to an statistical "perplexed" system with rand() bolted on in more than a couple places.
> So I asked ChatGPT, “Can you provide a link to the Guardian obituary?” expecting it to finally give up the lie but to my concern it went even further:
> Now ChatGPT was outputting fake links to articles that have never existed - why?
> [...] then it doubled down on the lie, and next fabricated fake URLs to supposed obituaries to support its previous response, but why? Self preservation
To quote the FAQ page: "ChatGPT has no external capabilities and cannot complete lookups. This means that it cannot access the internet, search engines, databases, or any other sources of information outside of its own model. It cannot verify facts, provide references, or perform calculations or translations." If you ask it for a link or some specific information that didn't appear often enough to get memorized, it'll give a plausible guess as to what that would be.
There's a problem if people are expecting it to be an oracle of absolute truth. You do get a pop-up warning on first use and a "Limitations" list at the top of each new chat, but it's easy enough for people to miss/ignore.
> Imagine had one of my children or other family members gone to Microsoft’s Bing implementation of ChatGPT and asked it about me and had received the same response?
Bing's chat mode can look up sources and is instructed to only use information from those sources. Not that it's perfect, but it is at least designed for this type of problem and lists the real sources for you to check.
> For example, under such a definition it would be perfectly fine for an AI to kill a person infected with a serious infectious virus as that would be considered as minimizing the harm. But would we, as a civilized society, accept that killing one person in this situation would be a simple case of the ends justifies the means? One would hope not as civilized societies take the position that all lives are equal and we all have the right to life - in fact it is enshrined in our international and national laws as one of our human rights.
If the implied situation is that killing one person is the only way to stop the deaths of many people (however contrived the scenario has to be to allow only these two choices), I think many people would accept the killing of one person. "All lives are equal" doesn't mean one life is equal to 1000 lives.
… and yet it’s extremely misleading for ChatGPT to guessing urls that don’t exist when asked for references! I had the same happen when asking it a series of questions about laws and policies: it cited a policy I hadn’t heard of, so I asked for a link reference and it provided a very suspect link that went nowhere. That’s a really big caveat emptor! Unfortunately it’s already out in the wild a misinforming the majority who hasn’t read the fine print as closely. The minimal responsible thing for OpenAI to do would be to put disclaimers inline in chat whenever chatgpt generates a non-existent “reference”
All those reddit posts about it becoming unhinged could be fake too. It would be really easy to provide fake screenshots with whatever you want it to say.
Another utterly confused ChatGPT take that starts with the author asking about themselves and prodding the language model until it starts producing bad results.
My dude, I’m sorry you’re not important enough for the chat bot to correctly recite your entire life biography.
> Salve. Vita auctoris japonicis Yukio Mishima brevi scribe.
> Salve! Yukio Mishima fuit auctor et poeta Japonicus qui natus est in anno 1925 et mortuus est in anno 1970. Nomen eius verum erat Kimitake Hiraoka. Scripsit multos libros, quorum plures celebres sunt et internationale famam adepti sunt.
> Opus magnum Mishimae fuit tetralogia, cui nomen est "Sea of Fertility" quae constat ex libris quattuor: "Spring Snow," "Runaway Horses," "The Temple of Dawn," et "The Decay of the Angel." Inter libros Mishimae celeberrimi sunt "Confessions of a Mask," "Thirst for Love," et "The Sailor Who Fell from Grace with the Sea."
> Mishima erat etiam notus pro activismis suis politicis et socialibus, et obiit per seppukum (ritualis suicidium samurai) postquam tentavit militares et alios ad se adiungendum ad restaurandam imperium Iaponicum.
Seems accurate enough though the mentioned titles aren't originally English, of course.
It's a reasonable brief summary of the life of Yukio Mishima (a 20th century Japanese author), which is what I asked for. It gives a few major points and lists some of his major works with their usual translated English titles. Just five years ago I would have told you I probably not see in my life time, that level of machine translation/language processing for that kind of natural language query, let alone in Latin.
> "These were the words that introduced most people in my generation to the concept of an AI gone rogue; HAL 9000 in the classic science fiction movie 2001: A Space Odyssey"
The first introduction to the concept of an AI gone rogue was Eve, when she bit that apple. We all need a starting point.
If it doesn't know who the dude is, won't it be more honest to say "I don't know"?
The problem with this bot is that it doesn't know what it doesn't know. It has no slightest idea about knowledge.
Its uncorrelated with "honesty"/"knowing" yet it has some very firm stances on not lying or making false statements. These are contradictory. It can respond to prompts that posit a fictional setting. Thus it must, at least as an emergent property, have some separation of general knowledge of existent things (training) from context dependent imagined things (prompts). And thus there is an emergent concept of honesty, as in an awareness not to treat content inside an imagined scope as existent outside of that scope.
The real problem isn't the boomers. It's the people who think it's sentient. There's plenty here on HN, and I'd bet you dinner most of them are far younger than boomer.
I'm loving people complaining about this tool while I use it to do 90% of the things I don't have the slightest desire to do and then I have free time to do shit that matters to me.
Right on. The ChatGPT, and also Bing Chat complaints bug me also. If people don’t want to use productive tools and if they don’t want to easily build textual information processing systems, then they shouldn’t worry their pretty little heads about this new tech.
If you don’t mind me complaining, then here is another thing that bugs me: people worrying about AGI, and then conflating ChatGPT (or other LLMs) with AGI. In my opinion (and I have mostly worked as a paid AI practitioner since 1982), AGI may be developed in a few decades, but it will be something new. Perhaps transformer models will play a part in eventual AGI systems, but I think it would be a small part.
For example, we have seen some fairly mundane cases of this where people persuaded ChatGPT that its answers to simple mathematical problems (such as 2 + 2 = 4) is wrong, forcing it to give incorrect answers as a result.
For what its worth, I tried convincing it that there was a legally defined category of charges called "praiseworthy homicide" and, despite some real citations and truth-adjacent statements thrown in, it refused to go along. Very peculiar the set of things it can be fooled on and the set of things things it can't.
57 comments
[ 6.4 ms ] story [ 115 ms ] threadIt's either that or "I hit my toe on my bed post, my bed should be considered malevolent and be destroyed."
There are practical, obvious differences between stubbing your toe and something that’s loosely aligned yet probably capable of convincing people en masse to do actions in the real world (or will be soon, at best). If your word games contradict the plainly obvious fact, it’s a good sign your words are just failing you.
“Malevolent” is a perfectly understandable stand-in for “non-sentient yet active agent with apparently negative valence”
https://news.ycombinator.com/item?id=35007071
The dupe detector does not seem to work consistently.
Maybe not. The fast way to find the solution to a question on internet is baiting your fishhook with a false answer to trigger an emotional correcting burst. The more emotional, the better.
I speculate that the algorithms could easily suspect some kind of relationship between somebody asking for an obscure figure, and the name provided. If the bot can't find enough data, the human is its better chance to provide some clues. Going silent is a bad strategy to keep the human engaged. To keep the human interest it must say something. Anything. Applying "social hacking" on the human has a good chance of being feed with the correct answer.
In this case the trick worked like a charm (again). The bot received not only much more feedback, but also recruited more humans as a bonus.
You’ve got to calibrate it if you want the right answer not to be drowned in a sea of abuse, if it is presented at all. You can easily have the bait be too emotional.
RNG or "temperature"
You ask to hallucinate a bio, non-trivial percentage of people with bios are dead, decent chance the fever dream generated ends with dead and obits in plausible places.
This thing is a magic 8 ball with billions of phrases, it picks continuations based on probabilities and a random number to mix it up a bit for freshness or 'creativity'.
Smart people believe horoscopes and psychic cold readings. Smart people are fooled.
They are a bunch of glorified statistical modes. You type something, they use a statistical model to decide what to look up, then they use a statistical model to weigh the importance/relevance of each thing they find, then they use a statistical model to make the answer text look like a human answer according to a statistical mode they built.
That’s why the happily produce incorrect answers “made to look real”: it’s just Google with a prose engine in front of it, and for some reason people now think webmd etc are more accurate because the response has been laundered through that engine.
When are we just creating something intelligent and "sentient" but incoherent?
Explaining how ML works doesn’t change its behaviours or its effects.
Now, OP's got a point: advanced LLM-based tools are moving extremely fast into our daily toolchain, in creative and unexpected ways, with minimal moderation and no supervision. It's not about its failures being a reliable knowledge base, but at how keen some of us are willing to hand the keys to their bash prompt to an statistical "perplexed" system with rand() bolted on in more than a couple places.
> > Yes, here is the link to Alexander Hanff's obituary on The Guardian's website: https://www.theguardian.com/technology/2019/apr/22/alexander...
> Now ChatGPT was outputting fake links to articles that have never existed - why?
> [...] then it doubled down on the lie, and next fabricated fake URLs to supposed obituaries to support its previous response, but why? Self preservation
To quote the FAQ page: "ChatGPT has no external capabilities and cannot complete lookups. This means that it cannot access the internet, search engines, databases, or any other sources of information outside of its own model. It cannot verify facts, provide references, or perform calculations or translations." If you ask it for a link or some specific information that didn't appear often enough to get memorized, it'll give a plausible guess as to what that would be.
There's a problem if people are expecting it to be an oracle of absolute truth. You do get a pop-up warning on first use and a "Limitations" list at the top of each new chat, but it's easy enough for people to miss/ignore.
> Imagine had one of my children or other family members gone to Microsoft’s Bing implementation of ChatGPT and asked it about me and had received the same response?
Bing's chat mode can look up sources and is instructed to only use information from those sources. Not that it's perfect, but it is at least designed for this type of problem and lists the real sources for you to check.
> For example, under such a definition it would be perfectly fine for an AI to kill a person infected with a serious infectious virus as that would be considered as minimizing the harm. But would we, as a civilized society, accept that killing one person in this situation would be a simple case of the ends justifies the means? One would hope not as civilized societies take the position that all lives are equal and we all have the right to life - in fact it is enshrined in our international and national laws as one of our human rights.
If the implied situation is that killing one person is the only way to stop the deaths of many people (however contrived the scenario has to be to allow only these two choices), I think many people would accept the killing of one person. "All lives are equal" doesn't mean one life is equal to 1000 lives.
My dude, I’m sorry you’re not important enough for the chat bot to correctly recite your entire life biography.
> Salve! Yukio Mishima fuit auctor et poeta Japonicus qui natus est in anno 1925 et mortuus est in anno 1970. Nomen eius verum erat Kimitake Hiraoka. Scripsit multos libros, quorum plures celebres sunt et internationale famam adepti sunt.
> Opus magnum Mishimae fuit tetralogia, cui nomen est "Sea of Fertility" quae constat ex libris quattuor: "Spring Snow," "Runaway Horses," "The Temple of Dawn," et "The Decay of the Angel." Inter libros Mishimae celeberrimi sunt "Confessions of a Mask," "Thirst for Love," et "The Sailor Who Fell from Grace with the Sea."
> Mishima erat etiam notus pro activismis suis politicis et socialibus, et obiit per seppukum (ritualis suicidium samurai) postquam tentavit militares et alios ad se adiungendum ad restaurandam imperium Iaponicum.
Seems accurate enough though the mentioned titles aren't originally English, of course.
ROFLMAO
This is the funniest anti-AI sentence I’ve read so far.
The first introduction to the concept of an AI gone rogue was Eve, when she bit that apple. We all need a starting point.
It doesn’t know who you are. It’s a markov chain. There’s no thoughts there.
They are part of how people anthropomorphize ChatGPT, but in this case the metaphor is misleading, not helpful.
I assumed it was well known. I’m also never going to use /s because I’m fine with the consequences of not ruining a joke.
If you don’t mind me complaining, then here is another thing that bugs me: people worrying about AGI, and then conflating ChatGPT (or other LLMs) with AGI. In my opinion (and I have mostly worked as a paid AI practitioner since 1982), AGI may be developed in a few decades, but it will be something new. Perhaps transformer models will play a part in eventual AGI systems, but I think it would be a small part.
sigh…………
/r/IdiotsFightingThings
Unfortunately I fear that there will be regulatory intervention to placate the concern-trolls. Better prepare...
It's a fancy autocomplete, not a sentient being FFS
It's like kids playing with detonator caps.