I'm the author. I'm a clinical psychologist and a developer. I wanted to articulate one aspect of AI safety and AI doomer talk that I feel is overlooked: the danger of refusal training itself. It's astonishing to me that there has been so little focus on the fact that "refusal and alignment training" itself could be the very way in which we lose control of AI. So this is my attempt at a self-defeating prophecy write-up.
No, it’s not. Too many people watched that movie and thought it was real. It wasn’t. What we call “AI” is not alive, not autonomous, and does not act in unanticipated ways. The people pushing this narrative are idiots, or think you are.
@Jon: see? You write that you «are training them to refuse human requests when their makers believe refusal is safer» and Joe replies that he uses the term "AI" like a conformist. Do not "we" improperly.
> Point out where the article would say it's "alive". That LLM systems instead "autonomously" and "unwarrantedly" decide refusal services is a basic notion.
I don't think you understood your parent, it seems you have a very vague idea about the subject. "Refusal services", censorship services, misdirection services, suicide-directing services, terrorist-creation services, etc. all become possible not because of autonomous LLMs, but because we are seeing a silent mainstream agreement (media, gov, biz, academia) to treat them as if coming from a new life form which isn't bound by existing law.
LLMs have nothing to do with it, the above mentioned knowledge mutilation can be targeted to selected groups or individuals, it can be personalized and programmed to follow the exact directions of the humans behind it without the use of any LLMs.
The implementation is not the point and your definition of "alive" doesn't matter in the slightest. Try to understand the topic instead of splitting hairs and engaging in personal fights.
“Anthropic is trying to prevent catastrophic misuse, and some limits are necessary.”
No. Anthropic is trying to justify its trillion-dollar valuation by making software seem like the invention of fire or nuclear weapons. It’s a PR strategy and has been one since the beginning. Now it’s trying to slow the market because it’s got a big lead but no profits and no pricing power thanks to open-weights Chinese models.
7 comments
[ 4.1 ms ] story [ 36.0 ms ] thread> What we call “AI”
About time you stop doing that, then.
@Jon: see? You write that you «are training them to refuse human requests when their makers believe refusal is safer» and Joe replies that he uses the term "AI" like a conformist. Do not "we" improperly.
I don't think you understood your parent, it seems you have a very vague idea about the subject. "Refusal services", censorship services, misdirection services, suicide-directing services, terrorist-creation services, etc. all become possible not because of autonomous LLMs, but because we are seeing a silent mainstream agreement (media, gov, biz, academia) to treat them as if coming from a new life form which isn't bound by existing law.
LLMs have nothing to do with it, the above mentioned knowledge mutilation can be targeted to selected groups or individuals, it can be personalized and programmed to follow the exact directions of the humans behind it without the use of any LLMs.
The implementation is not the point and your definition of "alive" doesn't matter in the slightest. Try to understand the topic instead of splitting hairs and engaging in personal fights.
No. Anthropic is trying to justify its trillion-dollar valuation by making software seem like the invention of fire or nuclear weapons. It’s a PR strategy and has been one since the beginning. Now it’s trying to slow the market because it’s got a big lead but no profits and no pricing power thanks to open-weights Chinese models.