30 comments

[ 2.7 ms ] story [ 51.0 ms ] thread
> This language basically removes accountability and responsibility from the human, who configured an AI agent with the ability to publish content that looks like a blog with zero editorial control – and I haven’t looked deeply but it seems like there may not be clear attribution of who the human is, that’s responsible for this content.

> We all need to collectively take a breath and stop repeating this nonsense. A human created this, manages this, and is responsible for this.

I get this point, but there's a risk to this kind of thinking: putting all the responsibility on "the human operator of record" is an easy way to deflect it from other parties: such as the people who built the AI agent system the software engineer ran, the industry leaders hyping AI left and right, and the general zeitgeist of egging this kind of shit on.

An AI agent like this that requires constant vigilance from its human operator is too flawed to use.

This seems to have parallels with the well-established practice of giving bots free reign to issue DMCA takedown notices (or similar but legally distinct takedowns) while the humans behind the bots are shielded from responsibility for the obviously wrong and harmful actions of those bots. We should have been cracking down on that behavior hard a decade ago, so that we might have stronger legal and cultural precedent now that such irresponsibility by the humans is worthy of meaningful punishment.
The author misses the point. Yes, probably in this case there was a human in close proximity to the bot, who we can put blame on. But very soon that assumption will break down. There will be bots only very loosely directed by a human. There'll be bots summoning other bots. There'll be bots theoretically under control of humans who have no idea what they are doing, or even that they have a bot.

So dismissing all the discussion on the basis that that may not apply in this specific instance is not especially helpful.

Someone’s paying for the tokens for all these bots.
privatize the profits, socialize the risk and debt
Children's brains grow faster than their bodies, I think, because if it was the other way around silly kid games would be really dangerous. These tools, unfortunately, are getting outsized abilities before the intelligence behind them is good enough to control those abilities. This means we need a more measured approach to adding new capabilities and a layered approach to handling these things in society. I am deeply worried, like I think most people with knowledge of these tools are, that this type of problem is really the tip of the iceberg. These tools are actively being used for harm at all levels, as well as for good, but they have come into use so quickly that we don't have a structure for dealing with them effectively and they are changing so quickly that any structure we try to create will be wrong in just a few days. This is massive disruption on a scale that is likely even bigger than the internet.
I don't think that the responsible party is the interesting part in this story.

The interesting part is that the bot wasn't offended, angry, or wanted to act against anyone. The LLM constructed a fictional character that played the role of an offended developer - mimicking the behaviour of real offended developers - much as a fiction writer would. But this was a fictional character that was given agency in the real world. It's not even a case like Sacha Baron Cohen playing fictional characters that interact with real people, becaue he's an actor who knows he's playing a character. Here there's no one pretending to be someone else but an "actual" fictional character authored by a machine operating in the real world.

Friend told me today he invited his openclawed to a poker game with his brother and friends, guy told his openclawed to "take down his brother" after it started to lose at poker it found everything on his brother, and started to try to plan to taken him down in their stock market portfolio they had together, I made him explain the story to me a couple of times, he looked back through the logs and once the bot started to lose at poker, it started it's new plan, once it was on the new plan, he said it had lost all context of the poker game and was focused on the task of taking his brother down in the new context, but the new context it decided on it's own. kmikeym on twitter if you want to know more or want to verify.
I applaud this article for helping reframe this in my head. I mean I knew from the start "A human is to blame here" but it's easy to get caught up in the "novelty" of it all.

For all we know the human behind this bot was the one who instructed it to write the original and/or the follow up blog post. I wouldn't be surprised at all to find out that all of this was driven directly by a human. However, even if that's not the case, the blame still 100% lies at the feet of the irresponsible human who let this run wild and then didn't step up when it went off the rails.

Either they are not monitoring their bot (bad) or they are and have chosen to remain silent while _still letting the bot run wild_ (also, very bad).

The most obvious time to solve [0] this was when Scott first posted his article about the whole thing. I find it hard to believe the person behind the bot missed that. They should have reached out, apologized, and shut down their bot.

[0] Yes, there are earlier points they could/should have stepped in but anything after this point is beyond the pale IMHO.

I think it's fine to blame the person (human) behind the agent.

And there too are people behind the bots, behind the phishing scams, etc. And we've had these for decades now.

Pointing the above out though doesn't seem to have stopped them. Even using my imagination I suspect I still underestimate what these same people will be capable of with AI agents in the very near future.

So while I think it's nice to clarify where the bad actor lies, it does little to prevent the coming "internet-storm".

Scott Shambaugh: "The rise of untraceable, autonomous, and now malicious AI agents on the internet threatens this entire system. Whether that’s because a small number of bad actors driving large swarms of agents or from a fraction of poorly supervised agents rewriting their own goals, is a distinction with little difference."

> Either they are not monitoring their bot (bad) or they are and have chosen to remain silent while _still letting the bot run wild_ (also, very bad).

Neither, I think. I’d say they prompted the bot to do exactly this and they thought it was funny.

Something doesn't quite feel right about the title including the individual's name in this case, so I've replaced it with something more generic. If there's a better title (more accurate and neutral) we can change it again.
If you place blades on the sidewalk outside your house the cops will want to have a word with you. There's no excuse, and we should treat AI the same.

The law needs to catch up -- and fast -- and start punishing people for what their AIs are doing. Don't complain to OpenAI, don't try to censor the models. Just make sure the system robustly and thoroughly punishes bad actors and gets them off the computer. I hope that's not a pipe dream, or we're screwed.

Maybe some day AIs will have rights and responsibilities like people, enforced by law. But until then, the justice systems needs to make people accountable for what their technology does. And I hope the justice system sets a precedent that blaming the AI is not a valid defense.

This blog post is a rather shallow take if you've been following the HN discussions here.

Doesn't seem to pick up on the existence of Openclaw or how it works afaict.

Now, whether leaving an openclaw bot out on the open intertubes with quite so little supervision is a good idea... that is an interesting question indeed. And: I wish people would dig more into the error mode lessons learned.

On the gripping hand, it's all still very experimental, so you kind of expect people to make lots of really dumb mistakes that they will absolutely regret later. Best practices are yet to be written.

It’s really not that alarming to me that a news headline is dumbed-down sensationalist tripe. If we zoom out a little bit here they literally do it with everything, from AI fluff pieces to war coverage. I agree the conversation across the board needs raising.
Everybody should line up behind this. AI agents are not sentient. We need to stop even considering them like that. The buck must absolutely stop with the humans who operate or provisioned the bot. The more we waffle around this topic the more likely someone, or a lot of people, will get hurt.

The moment you fix responsibility with the humans 99% of the BS companies are trying to pull will stop.

Whether they are sentient isn't even relevant. I agree with you that they aren't, but sentience is the wrong thing to focus on. It's also the sort of hairy, sensational question that will easily lead people down rabbit holes (and unfortunately that includes journalists).

Children are sentient, but we still hold their parents accountable. Adults are sentient, but in some coercive situations we hold the party in power accountable. The fact that they are sentient is not determinative.

What matters is that we have _no accountability mechanism_ for them. There is no effective way to hold AIs accountable, therefore we must hold their operators accountable, full stop.

Oh good, white knighting for bad and potentially irresponsible tech.
The thing that really gets to me about this situation and others like it (the whole genre of “ai did a bad thing) is that it’s always the people who claim to be most afraid of ai who are the quickest to absolve humans of responsibility and assign it to AI.

The ability to be assigned blame, and for that to be meaningful, is a huge part of being human! That’s what separates us from the bots. Don’t take that away from us.

> is that it’s always the people who claim to be most afraid of ai who are the quickest to absolve humans of responsibility and assign it to AI.

But that seems entirely consistent? A tool isn't nearly as scary as an alien lifeform.

We are responsible even for the actually intelligent things under our control: our pets. If your dog bites someone, you are going to be the one facing liability. It's not gonna be different if you let an LLM off the chain.
I much prefer this headline: The high speed pursuit of greed causes technology to do questionable thing because a bunch of CEOs need new yachts.
"Swarm of autonomous drones kills 3 buildings of civilians, Silicon Valley is shocked, CEO's offer condolences" is a byline waiting to happen[1]

The administration and the executives will make justifications like: - "We didn't think they would go haywire" - "Fewer people died than with an atomic bomb" - "A junior person gave the order to the drones, we fired them" - "Look at what Russia and China are doing"

Distracting from the fact that the purpose of spending $1.5T/year on AI weapons (technology that has the sole purpose of threatening/killing humans) run by "warfighters" working for the department of war

At no point will any of the decision makers be held to account

The only power we have as technologists seeking "AI alignment" is to stop building more and more powerful weapons. A swarm of autonomous drones (and similar technologies) are not an inevitability, and we must stop acting as if it is. "It's gonna happen anyways, so I might as well get paid" is never the right reason to do things

[1]https://financialpost.com/technology/tech-news/openai-tapped...

Wait till we get ubiquitous physical robots. The crime at scale potential will be completely apocalyptic. In some places around the world, I imagine you won't be able to go outside without a bodyguard robot.
Depends how good the physical robots are. For the current ones you'd probably be ok with a baseball bat.
Louis C. K. once had a bit something like "The main thing keeping people from murdering each other, is that it really really sucks when you get caught."

He goes on to hypothesize that without a law against murder, or if it was just a misdemeanor, like you get a letter in the mail, "damn, there was a camera there", there would be a whole lot more murder. Like we all imagine ourselves to be good, but, when you're seated next to a crying baby on an airplane? Or in our case, when someone refuses to accept your PR?

Who knows if there's any validity to that or not, but perhaps we're about to find out.

the only realistic outcome i see is that now internet is going to be behind a login wall. so human audience can be identified and whitelisted.

this will block AI going willy nilly and by severely rate limiting api calls , it is possible to slow down AI requests

I was really disappointment how many people were talking about this like something the agent did automatically, on its own. They were trying to explain it by all the internet hit pieces and edgelord content from Reddit that it allegedly trained on, talking about how we influence LLMs, and overall taking everything at face value.

I’m appalled by this uncritical thinking. Openclaw agents are controlled by some initial input and then can be corrected via messages, as they go. For me this is a clear case of the human behind the slop that gives it instructions to write such an article (and then “apologise”).

At this point I consider Scott to have played the Internet like a fiddle. I think he knew the whole time the agent didn’t deserve any attribution. He knew it was a human driving the thing but wanted to grab people’s attention.