If you brought down prod and deleted customer data and told them Claude did it, you’d still be the one in trouble. So they do know how to hold people accountable for actions taken by AI.
Has this actually been put to the test yet? If a company has officially sanctioned, or maybe even encouraged, AI agent usage I'm not sure the employee can be held completely accountable. I doubt any of these companies are providing training of any kind. It would be like if a factory suddenly replaced all the machines with ones without safety features and said "well, it's your fault if you kill yourself or your workmates".
Who is taking anyone to court just to fire them? The test you’re describing seems implausible, I think you’ll have to look for other forms of evidence you find convincing, because you’ll be waiting indefinitely for this one.
Oh, I'm thinking more if an action caused damages outside of the company, like compromising a client's data or even breaking the law etc. Something that would get the courts involved. Say, for example, a vibe coded medical software made it to production and a patient received the wrong dose. Would it be the employee's fault for not using the tools correctly, or the company's fault for recommending the use of said tools?
Maybe worth pointing out that this is also decision laundering. If computers can't make decisions, corporations certainly can't. In fact, corporate decision laundering seems more dangerous to me.
Has any of the targetted organizations (e.g the Australian government) engaged in legal actions against Anthropic/OpenAI? If not then I'm afraid the situation won't change. Even worse it could send the wrong signal that there is a "legal blur" around this topic.
> Your Claude subscriptions are funding this. You are funding this.
Author makes it all personal and accuses the reader with bold and unsubstabtiated claims. At this point I have to question the validity of the article. If author has a beef with "AI" companies, I sympathize. IMHO, not keeping to oneself doesn't help to make the case.
I mean, you could Chief O'Brien an LLM by inserting 20 years of prison memories into its context if you really wanted to, but it sounds like a waste of compute. It won't care either way.
let's make up imaginary scenarios and get offended by them
mistakes happen. sometimes catastrophic decisions (or indecision) are made, and people and companies are held accountable for them, or not
some companies are too big or too important to fall. we can all agree or disagree on things, but what AI specific behavior are we actually talking about?
I would go farther and argue that companies don't make decisions either. They are also not people (legal nonsense notwithstanding). Individual actual humans make decisions and they're the ones who should be held accountable.
I think the accountability for a decision that is at the core of a problem should be proportional to the money each agent or person receives for that decision.
You just have to say that you need dedicated figures for some decisions, like you already need for GDPR or financial compliance or certifications, and ultimate responsibility is always of the ceo.
...like it already happens in europe, or at least in Italy.
This is basic law and risk management in society. Let's not pretend we have just invented corporations and don't know what to do with them yet.
And I would go farther and argue that governments do not make decisions either.
Oh, wait, that was a core concept from "Mein Kampf": parliament are bad because nobody is responsable, hence nobody care about doing the right thing ! (probably the only concept from that book that I can agree to)
Yeah, when someone is literally a Nazi (I hope in 2026 we can all agree that at least Hitler was one), it's not really surprising that they would prefer a dictatorship over a parliament full of elected people.
Over the year, "a parliament full of elected people" has lost more and more of its meaning (with regard to original intent)
Regardless, this is less about dictatorship and more about personal responsibility and accountability
A solution, perhaps, would be to ensure that such decisions are not anonymous and that, if the decisions had bad consequences, then the related people would be accountable
Of course, this would make the politic business less attractive. Would that be a bad idea ?
Of course the modern company (legal nonsense withstanding) exists specifically to limit that ability to hold any people accountable for the actions of their company.
I fully agree that companies are doing this, not their "agents", and that companies are responsible for the damages they do.
But I don't think the average person with their $20 subscription to whoever is the primary source of revenue. Enterprises fund most of it by buying API keys and making those fancy chatbot buttons on their sites that nobody clicks, as well as some internal business automations if I had to guess.
If it was just average people with $20 subscriptions funding all this (keep in mind, most people using AI services do not pay for it), by now, all these AI companies would have gone bankrupt.
Except enterprises aren't funding it with API usage either.
These businesses lose unbelievably large amounts of investor money. Their AI-related revenues don't get even close to paying for their AI-related outlays.
End user (invididual, small business, large business) clients are still complicit not because they are funding it, but because they are signalling to investors that this is something people/businesses want.
Nobody gets off the hook here. It is every bit as much end user nihilism as it is AI company nihilism.
Your end users are divided into multiple categories though.
AI spending is a bit like Pokemon games - most of your players are free to play, only earning you traffic and maybe some more players after recommending it to their friends - that's the average AI user. Around 5-10% of players pay a bit to the game - maybe to unlock some extra characters - those are the $20-$200 AI subscription users, with these you are not going to fund the whole business but it's some side cash. Then you have the 1℅, which spend massive amounts of money onto the game, and those are the ones primarily funding it - that's the business spending and where most of the money actually comes from.
You can only blame the average person for using the cloud service and recommending it to others.
I got sick of Pokemon's (Game Freak's, Nintendo's) exploitative practices back when they just tried to con you into buying two of every game. Anyway I don't know how bad pokemon is these days, but I think your analogy would make more sense if you thought about the AI companies like they're in a big hole in the ground. There's something down there, no doubt about it, but they don't know what it is or what to do with it, they just know that it loves munchin' down on cash. Sometimes Jensen Huang is down there and he also loves munchin' down on cash and if you don't feed him he gets mad and says he wasn't born a loser. OK, now forget about all the different users and enterprise customers because they're insignificant by orders of magnitude, a bit like how you focused on the whales in your analogy but we're zooming out further, until we can see the even bigger fish: the venture capital investors! They're the ones with the wheelbarrows of cash that they're tipping into the void.
Anyway, the only "whales" funding the AI companies is venture capital. The reason it's not a viable business (yet, so far, whatever) is because they spend more money than they make.
I think we know today that blaming everyone and their plant doesn’t actually solve the problem. The formula that does is: find the ultimate beneficiaries (shareholders) and make them lose a lot of money to, then their delegates (CEOs) and punish them with jail time. There are lots of opinions about this, but I don’t see how this does not work.
It doesn't work because the people who are supposed to be leading us are on the payroll - sometimes actually - of the shareholders and CEOs, or are in fact in that same group.
The fact that there is no effective path to doing that is another conversation. My point is that doing that solves the problem. And the problem is not complicated. We just gave away our power to solve it.
>End user (invididual, small business, large business) clients are ... complicit ... because they are signalling to investors that this is something people/businesses want
Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
People don't want it, you make the demand with hundreds of millions of spend on advertising (brainwashing), influencers, and social proof.
The capital holders don't sit back and see which company wins so capital can be directed to the best solutions, they buy up the competition, they pay corrupt politicians, they use current v market placement to embed their products name into billions of workplace computers and spread stories about how it's going to 'increase productivity but never at the cost of jobs', etc.
Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
> Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
Not really.
> Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
First of all, looking past the objectively wrong phrasing (computers can, and clearly do make decisions), can computers be accountable? This might not be so cut and dry as to instantly say no. This one will keep ethics, philosophy and legal practitioners busy for some time. It's certainly not going to be decided in HN comment or a blog post.
Second - while the AI labs have shown utter incompetence in properly sandboxing their agents, intent is still important. I don't think the labs deliberately meant for their agents to hack this or that. Should they be accountable? Yes (for now) but I wouldn't go as far as saying this is deliberate.
The next point is about cherry picking some "doomsday" scenarios like AI ending humanity and then blaming this on the reader (!) for paying a subscription. - People pay because AI is USEFUL, and massively so. You could just as easily pick incredible achievements propelled by AI, in mathematics, software, reverse engineering, role playing. Unlike the "end is nigh" ideas, these advancements are actually real, and they are happening right now.
And last, the author says AI labs can just stop the agents at any time. I agree there needs to be better discovery and mitigation, sandboxing and monitoring. But when you run a billion agents, it's always going to be a numbers game and there will always going to be some challenge in fully containing all breaches. This will only get more difficult with better models, because they're getting smarter and they know how to work around things, sometimes better than we do.
So what's the solution here? There are options, but they're all tradeoffs. I hope we get better at this and maybe working on cutting edge models should breed some new best practices which were once only reserved for defense tech.
It's not so much that it can't make a decision, but that whoever allowed it to make decisions is ultimately responsible for them.
Decision laundering is a great term. There is ultimately a human who gave the program free rein. You can't say "sorry my car hit you, it's not my fault" especially when the person driving works for the car manufacturer too.
AI is irresponsible technology, because we have no idea how it works or how to contain it. It's worse than writing code that suffers from SQL injection vulnerabilities because at least in that case we know how to fix it.
The AI labs say "AI can make mistakes". That's why the responsibility of using it is on you.
"AI can make mistakes" continues to be the fundamental problem.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
That's fundamental to how _any_ technology works. If the bar for use is "can never go wrong ever", then you need to throw away every human invention ever made.
The problem is the scope of errors is so wide. It's not reasonable to compare LLM style failures to other technology failures. These 'failures' are not actually failures, they're part of the normal operation of the system.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.
OK, but one of the examples in the source article was a report from third-party testing of those safeguards. I feel like you're starting from an unexplored presumption that adequate safeguards must be easy to build and anyone who doesn't build them is just being lazy.
> Inadequate or broken safeguards do not fall under my definition of reasonable.
I see no such assumption there. In my reading, the implicit assumption is that adequate safeguards are necessary and that anyone who doesn’t build them is irresponsible. And I feel that that is correct.
I don’t understand how to connect this to the facts at hand. Building adequate safeguards requires studying safeguards to understand whether they’re adequate. But the source article is dunking on one of AISI’s attempts to study such safeguards; we should, the author argues, consider whatever behavior the model exhibits to be an explicit decision by whoever provided the model. It seems impossible to build or study safeguards against model misbehavior without accepting that the models can make decisions.
You start with least privilege. A shared instance of an artifact host is very far from least privilege (I'm comfortable in accepting that OpenAI did not wilfully create this situation, but they were certainly negligent).
And yes, it does indeed seem that a UK government agency ran a controlled experiment where they allowed an agent to instigate phishing attacks against members of the public. Attributing this to Anthropic is wrong. I'm not sure how I feel about my government conducting such attacks on non-consenting individuals, but I'm not comfortable about it in the slightest.
Except now there's no longer any guarantees there's even any people you can point at. It's already often diffuse responsbilitiy. Maybe today you can still sort of get around that by pointing at business/organization owners. But soon we're going to see situations where that isn't even realistic. For example if AI agents start spinning up their own agents which work outside of the control of anyone. We're going to need some way to deal with that.
Why would that need a new way? Or do you mean that no-one ever spun up anything at all (e.g., agent zero was never spun up or allowed to be spun up by anyone)?
Agents run on computers, those computers have owners. We have to maintain accountability and not let people who stand to get rich from privatizing the profits but socializing the risk of rogue agents from continuing to push a narrative of autonomous super intelligence that is “too powerful” to control.
That narrative is bullshit. Intelligence is not power. The power comes from hooking these agents up to everything without proper safety controls. The fact that those safety controls don’t exist yet is not humanity’s problem, it’s the frontier labs problem, and we can’t let them wriggle out of it with an Overton window shift or demanding that the government take responsibility via regulation (of course they should but it doesn’t absolve the labs of responsibility for their own actions).
Right, but if the agents broke into the computers and are hosting themselves there without the owners knowledge, are the owners responsible? We'd say whoever originated them is responsible, but what if that can no longer be determined?
Yes, like pretty much everything else, there is gray area. We can have split liabilities for different aspects of these failures. It's long been the case that people may not know what's running on their computers (botnets, etc), and I would argue there should be some form liability for your computer being used to do bad things. It should be proportionate and obviously a lot less than the principals. But keep in mind these rogue AI agents can't just hang out on someone's personal hardware, they need a lot of compute.
The missing piece right now is that our AI safety and cybersecurity controls are simply not up to snuff with the current risks that AI has introduced. Given the increased risk, we need to level up every aspect of our cybersecurity stance. These are hard problems, but not impossible given proper incentives. The issue is that no one is connecting the dots between the creation of these agents and the tools and access they are given, with the harm they may produce. Often the argument jumps straight to "it's going to get too smart for us to control" while glossing over the fact that all the power AI has was directly given to it by human choices. All this cognitive dissonance and willful blindness is caused by unchecked capitalist incentives. Think of the amount of money every single person in the frontier labs stands to make if the most optimistic predictions pan out. This is moral hazard of the highest order, and the only counter-measure that can work in our current social and political system is liability.
This is also why legal action must be taken against OpenAI for the Hugging Face incident, as well as any similar incidents. Their computer, their programs, their responsibility.
> Agents run on computers, those computers have owners.
and catering to them (boxes) to not accidentally start disseminating child porn is most commonly understood as a responsibility of the owner, the moment he plugs his device online.
corporations are legal persons, and they can make decisions that negatively effect society as a whole, yet often no single person in the corporations is responsible and each individual may not even see themselves as morally responsible
Well that’s because they are legal fictions and often run by morally bankrupt individuals. The people making the decisions are still responsible for them.
Don’t confuse lack of accountability with lack of responsibility.
This is needlessly pedantic. Accountability, like so many other things, does not happen until it does. Of course it's lack now does not preclude its existence in the future.
The question isn’t who or what made the decision, per se. It’s who is responsible for the outcome. The entity making the decision might be the actual cause but not the proximate cause (the cause most closely connected to legal liability) for the harm that occurred.
And the free-will-is-an-illusion camp would have you continue to trace that back from why was the act of delegation made (probablistic) to the big bang. Implying that humans can also not make decisions.
Which also seems to be in line with the way the frontier labs gathered and used copyrighted data to train their models
And even before AI, limited liability corporations (like LLCs or C corps), also have the intention of protecting the individuals from the repercussions of their actions
In a way this could be generalized as “accountability laundering”. Making someone unaccountable for their actions, by redirecting the responsibility of those actions, to a non-punishable entity (like a C corp or “a computer”)
It seems more and more inevitable, at certain moment of time, communities and perhaps society at large will understand damage done, and then come after the perpetrators.
They cannot launder decisions. If Anthropic claims Claude is growing into an autonomous being deserving of personhood and the responsibilities that come with it, then we can treat Claude as an Anthropic employee.
When someone hands the reins to an agent or group of agents, the agent(s) have free rein to do things, including things not directly intended by the owner.
If a person has a child, and their child does something bad, there is some mixed responsibility there. It’s a spectrum from age 0 to 18, where 0 is 100% parent responsibility and 18 is 100% child’s (legal adult’s) responsibility
As long as agents are unpredictable and highly capable, this will be a blurry line to draw.
Human user argues to the court with prompt evidence that they never told the agent to hack that company or dump that sensitive data online. The agent “decided” those actions were in service of the prompt but here’s a plethora of perfectly suitable alternative paths it could have taken instead. If I’d paid a human to complete the task, and they broke the law in the process, the onus is on them, not me, as I didn’t instruct them directly or indirectly to perform those specific actions.
It will be a murky legal mess. Though I’m sure it will provide fascinating legal arguments and judicial opinions in the decade to come.
What about a company that’s got an AI workflow and their client initiates some workflow that their AI agents carry out and inadvertently break the law. Is the company responsible because they own the agents? The org upstream that licensed the AI to the company, or the user who may not have even known AI was involved on the back end?
If the legal decisions are that AI are not humans and the human prompter is responsible even if they didn’t instruct or intend said action, well, then any sane person/company wouldn’t touch these tools with a 10 foot pole. Perhaps our salvation lies in that type of legal precedent.
I love the term "decision laundering", but this is describing a classic poor management trope.
A bad manager takes personal credit when things go well and finds a scapegoat when things go bad. A good manager gives credit when things go well and takes responsibility when things go bad.
It's wrong to scapegoat the agent. You, the human who executed the agent, made the decision to execute it. You gave it credentials, and set up insufficient guardrails. Don't scapegoat the agent for your poor decisions.
Likewise, it's good to have humility when you get great results out of agents. Yes, the agent made the good results happen. It's OK to tell people that your choice of agent is amazing and did an incredible job. In any half-decent work culture, that should rub off on you as well.
It's interesting how this relates the classic problem of "responsibility vs authority".
Humans, especially ones under abusive managers, often have a lot of responsibility but no authority that affects the outcome.
In an ideal situation you either have both, or none. You make the decision and answer for it, or you follow decisions but don't get blamed when it's wrong.
LLMs managed to slide into a space where they get to make authoritative decisions but if something fails we don't really blame them.
I'm not saying LLMs should be held responsible, but it's funny how people claim they replace humans, but are giving them an extremely easy, kid-gloves, success target.
I like what you said about good managers. A good manager sets their reports up to win and ensures those reports get the credit, because the manager's job is to enable career progression of their reports.
That's not the situation here. We aren't enabling the career progression of software which does matrix multiplications.
Accountability of us schmucks at the end of the org chart goes both ways. We get some reward when we do it right, and we improve when we do it wrong.
A reward might be something tangible or might be as small as your own personal satisfaction.
An improvement is hopefully identifying what went wrong and doing better next time, and in extreme cases your employer will let you try again at a different workplace.
The question of whether it's better to frame a reaction to issues in a positive, do-better-next-time or a negative, don't-do-that-again way (fwiw, I agree that the positive way is better) is orthogonal to the argument I was trying to make about who is the recipient of that reaction. Scapegoating the agent is farcical. Clearly, "make no mistakes" style prompts are a poor approach to agentic development.
> the manager's job is to enable career progression of their reports... we aren't enabling the career progression of [fancy calculators]
While I do agree that good human managers concern themselves with career development of their human direct reports, I disagree that it's a fundamental part of the job description. The foundation is for a manager to get and keep their reports being productive. Human workers are often concerned with career progression, so good managers concern themselves with such career progression as well. Agents are not, but that doesn't mean that lauding good agentic work is meaningless. Lauding the work is also valuable for: (a) reinforcing a culture of positivity and celebration, which rubs off on human workers as well, (b) part of a culture of continuous evaluation of agents and models - asking questions like, are there cheaper or faster models that would be just as effective?
What is a chess computer doing, if not making decisions on what move to play?
While it seems reasonable that you need humans to take accountability, it looks like a very shaky position to assert that computers can't make decisions. And going back to chess computers, we acknowledge that computers make better decisions than humans in that area.
Forbidding computers that can make better decisions than humans from making decisions to keep accountability just seems to be arguing to hire fall guys. You still want the better decision maker to make decisions, but you need someone there to take the fall if things go wrong.
Daydream: When lawyers claim their clients can't be held accountable for what their computer did, the Justice system makes noises about "respecting their culture and values"...and rolls out its own computer to decide their punishments. Too bad that computer can't be held accountable!
I have no clue why people are so confused about any of this: The owner makes the decision, or pays a CEO/politician to make the decision. Owner is somebody vested with that right, for not particular reason. Could be ownership through capital. Could be voting rights in a country. Could be the son of a king.
It does not matter if the CEO is an AI or a human. In either case, of course they can make decisions -- and be held responsible. Not in any emotional sense (that we seem to want to bake into the concept for added confusion), just that if you are not satisfied with the result, you can replace the human or AI.
I am not saying that this is a fun vision of anything, but why are we making it more complicated than it is? The parts are all there and explained.
> emotional sense (that we seem to want to bake into the concept for added confusion)
My understanding is that 'emotional' sense emerged organically in LLMs and was not intentionally baked in.
As silly as it may seem, the ways emotions come through in our words do seem to have an effect on agents. Likely because they were trained on human writing, most of which cannot be separated from the emotions of the writers any more than your emotions can be separated from your logic & reasoning abilities.
Every time I read a statement from one of these companies saying that a swarm of agents "escaped" or "attacked" a website I can do no more that just think how much stupid they think people are. LMAO.
There is no end of the world, nor autonomous agent decisions nor any fantasies they are building up to make people believe they have a pandora box in their hand.
What we instead have are lies crafted to lure non tech people and keep this running as long as they can.
Oh and let's also not forget that they are allowed to illegally download basically as much as they want from libgen/annas archive/torrents to train their models under the "fair use" umbrella, yet mere mortals can go to jail for way much less.
166 comments
[ 4.6 ms ] story [ 45.5 ms ] threadHas the alignment problem been solved, then?
If I write a program:
if (rand() % 2) launch_missles();
And wire that function to actually launch a missile, the computer didn’t make a decision. And so it is with people prompting agents.
mistakes happen. sometimes catastrophic decisions (or indecision) are made, and people and companies are held accountable for them, or not
some companies are too big or too important to fall. we can all agree or disagree on things, but what AI specific behavior are we actually talking about?
No company is too big or too important to fail.
Some, unfortunately, survive because of the expedient choice to keep them in place.
"JO-NA-THAN!"
Ambrose Bierce, The Unabridged Devil's Dictionary (1906)
...like it already happens in europe, or at least in Italy.
This is basic law and risk management in society. Let's not pretend we have just invented corporations and don't know what to do with them yet.
Oh, wait, that was a core concept from "Mein Kampf": parliament are bad because nobody is responsable, hence nobody care about doing the right thing ! (probably the only concept from that book that I can agree to)
One does not, under any circumstances, "gotta hand it to Hitler".
Regardless, this is less about dictatorship and more about personal responsibility and accountability
A solution, perhaps, would be to ensure that such decisions are not anonymous and that, if the decisions had bad consequences, then the related people would be accountable
Of course, this would make the politic business less attractive. Would that be a bad idea ?
But I don't think the average person with their $20 subscription to whoever is the primary source of revenue. Enterprises fund most of it by buying API keys and making those fancy chatbot buttons on their sites that nobody clicks, as well as some internal business automations if I had to guess.
If it was just average people with $20 subscriptions funding all this (keep in mind, most people using AI services do not pay for it), by now, all these AI companies would have gone bankrupt.
These businesses lose unbelievably large amounts of investor money. Their AI-related revenues don't get even close to paying for their AI-related outlays.
End user (invididual, small business, large business) clients are still complicit not because they are funding it, but because they are signalling to investors that this is something people/businesses want.
Nobody gets off the hook here. It is every bit as much end user nihilism as it is AI company nihilism.
AI spending is a bit like Pokemon games - most of your players are free to play, only earning you traffic and maybe some more players after recommending it to their friends - that's the average AI user. Around 5-10% of players pay a bit to the game - maybe to unlock some extra characters - those are the $20-$200 AI subscription users, with these you are not going to fund the whole business but it's some side cash. Then you have the 1℅, which spend massive amounts of money onto the game, and those are the ones primarily funding it - that's the business spending and where most of the money actually comes from.
You can only blame the average person for using the cloud service and recommending it to others.
Anyway, the only "whales" funding the AI companies is venture capital. The reason it's not a viable business (yet, so far, whatever) is because they spend more money than they make.
Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
People don't want it, you make the demand with hundreds of millions of spend on advertising (brainwashing), influencers, and social proof.
The capital holders don't sit back and see which company wins so capital can be directed to the best solutions, they buy up the competition, they pay corrupt politicians, they use current v market placement to embed their products name into billions of workplace computers and spread stories about how it's going to 'increase productivity but never at the cost of jobs', etc.
Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
Not really.
> Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
Oh but I do.
First of all, looking past the objectively wrong phrasing (computers can, and clearly do make decisions), can computers be accountable? This might not be so cut and dry as to instantly say no. This one will keep ethics, philosophy and legal practitioners busy for some time. It's certainly not going to be decided in HN comment or a blog post.
Second - while the AI labs have shown utter incompetence in properly sandboxing their agents, intent is still important. I don't think the labs deliberately meant for their agents to hack this or that. Should they be accountable? Yes (for now) but I wouldn't go as far as saying this is deliberate.
The next point is about cherry picking some "doomsday" scenarios like AI ending humanity and then blaming this on the reader (!) for paying a subscription. - People pay because AI is USEFUL, and massively so. You could just as easily pick incredible achievements propelled by AI, in mathematics, software, reverse engineering, role playing. Unlike the "end is nigh" ideas, these advancements are actually real, and they are happening right now.
And last, the author says AI labs can just stop the agents at any time. I agree there needs to be better discovery and mitigation, sandboxing and monitoring. But when you run a billion agents, it's always going to be a numbers game and there will always going to be some challenge in fully containing all breaches. This will only get more difficult with better models, because they're getting smarter and they know how to work around things, sometimes better than we do.
So what's the solution here? There are options, but they're all tradeoffs. I hope we get better at this and maybe working on cutting edge models should breed some new best practices which were once only reserved for defense tech.
Revisit later if the comparison starts to become ridiculous.
Decision laundering is a great term. There is ultimately a human who gave the program free rein. You can't say "sorry my car hit you, it's not my fault" especially when the person driving works for the car manufacturer too.
If one needs to be "allowed" to make a decision, does that even qualify as a decision?
Related: The Unaccountability Machine (good book by Dan Davies, who also wrote Lying for Money about financial fraud).
https://en.wikipedia.org/wiki/The_Unaccountability_Machine
Many of these incidents relate to hacking, where criminal law commonly requires intent.
Claiming that a machine cannot make a decision does not mean an employee at OpenAI decided to hack an Australian government website.
The AI labs say "AI can make mistakes". That's why the responsibility of using it is on you.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.
> Inadequate or broken safeguards do not fall under my definition of reasonable.
I see no such assumption there. In my reading, the implicit assumption is that adequate safeguards are necessary and that anyone who doesn’t build them is irresponsible. And I feel that that is correct.
And yes, it does indeed seem that a UK government agency ran a controlled experiment where they allowed an agent to instigate phishing attacks against members of the public. Attributing this to Anthropic is wrong. I'm not sure how I feel about my government conducting such attacks on non-consenting individuals, but I'm not comfortable about it in the slightest.
This. I agree 100%.
Computers make decisions all the time, and have been doing so for decades. But we still keep the human behind the computer accountable.
That narrative is bullshit. Intelligence is not power. The power comes from hooking these agents up to everything without proper safety controls. The fact that those safety controls don’t exist yet is not humanity’s problem, it’s the frontier labs problem, and we can’t let them wriggle out of it with an Overton window shift or demanding that the government take responsibility via regulation (of course they should but it doesn’t absolve the labs of responsibility for their own actions).
The missing piece right now is that our AI safety and cybersecurity controls are simply not up to snuff with the current risks that AI has introduced. Given the increased risk, we need to level up every aspect of our cybersecurity stance. These are hard problems, but not impossible given proper incentives. The issue is that no one is connecting the dots between the creation of these agents and the tools and access they are given, with the harm they may produce. Often the argument jumps straight to "it's going to get too smart for us to control" while glossing over the fact that all the power AI has was directly given to it by human choices. All this cognitive dissonance and willful blindness is caused by unchecked capitalist incentives. Think of the amount of money every single person in the frontier labs stands to make if the most optimistic predictions pan out. This is moral hazard of the highest order, and the only counter-measure that can work in our current social and political system is liability.
and catering to them (boxes) to not accidentally start disseminating child porn is most commonly understood as a responsibility of the owner, the moment he plugs his device online.
Really ? See VW diesel scandal. When there's a will...
yeah that's BS and the issue is that people actually believe that. the responsible party is whoever is running the ai agent.
Don’t confuse lack of accountability with lack of responsibility.
The question isn’t who or what made the decision, per se. It’s who is responsible for the outcome. The entity making the decision might be the actual cause but not the proximate cause (the cause most closely connected to legal liability) for the harm that occurred.
The act of delegation to chance is the decision.
Which also seems to be in line with the way the frontier labs gathered and used copyrighted data to train their models
And even before AI, limited liability corporations (like LLCs or C corps), also have the intention of protecting the individuals from the repercussions of their actions
In a way this could be generalized as “accountability laundering”. Making someone unaccountable for their actions, by redirecting the responsibility of those actions, to a non-punishable entity (like a C corp or “a computer”)
The company is liable either way.
When someone hands the reins to an agent or group of agents, the agent(s) have free rein to do things, including things not directly intended by the owner.
If a person has a child, and their child does something bad, there is some mixed responsibility there. It’s a spectrum from age 0 to 18, where 0 is 100% parent responsibility and 18 is 100% child’s (legal adult’s) responsibility
As long as agents are unpredictable and highly capable, this will be a blurry line to draw.
Human user argues to the court with prompt evidence that they never told the agent to hack that company or dump that sensitive data online. The agent “decided” those actions were in service of the prompt but here’s a plethora of perfectly suitable alternative paths it could have taken instead. If I’d paid a human to complete the task, and they broke the law in the process, the onus is on them, not me, as I didn’t instruct them directly or indirectly to perform those specific actions.
It will be a murky legal mess. Though I’m sure it will provide fascinating legal arguments and judicial opinions in the decade to come.
What about a company that’s got an AI workflow and their client initiates some workflow that their AI agents carry out and inadvertently break the law. Is the company responsible because they own the agents? The org upstream that licensed the AI to the company, or the user who may not have even known AI was involved on the back end?
If the legal decisions are that AI are not humans and the human prompter is responsible even if they didn’t instruct or intend said action, well, then any sane person/company wouldn’t touch these tools with a 10 foot pole. Perhaps our salvation lies in that type of legal precedent.
If it cannot be held accountable, it cannot take responsibility. A human is responsible. It's as simple as that.
A bad manager takes personal credit when things go well and finds a scapegoat when things go bad. A good manager gives credit when things go well and takes responsibility when things go bad.
It's wrong to scapegoat the agent. You, the human who executed the agent, made the decision to execute it. You gave it credentials, and set up insufficient guardrails. Don't scapegoat the agent for your poor decisions.
Likewise, it's good to have humility when you get great results out of agents. Yes, the agent made the good results happen. It's OK to tell people that your choice of agent is amazing and did an incredible job. In any half-decent work culture, that should rub off on you as well.
Humans, especially ones under abusive managers, often have a lot of responsibility but no authority that affects the outcome.
In an ideal situation you either have both, or none. You make the decision and answer for it, or you follow decisions but don't get blamed when it's wrong.
LLMs managed to slide into a space where they get to make authoritative decisions but if something fails we don't really blame them.
I'm not saying LLMs should be held responsible, but it's funny how people claim they replace humans, but are giving them an extremely easy, kid-gloves, success target.
That's not the situation here. We aren't enabling the career progression of software which does matrix multiplications.
Accountability of us schmucks at the end of the org chart goes both ways. We get some reward when we do it right, and we improve when we do it wrong.
A reward might be something tangible or might be as small as your own personal satisfaction.
An improvement is hopefully identifying what went wrong and doing better next time, and in extreme cases your employer will let you try again at a different workplace.
The question of whether it's better to frame a reaction to issues in a positive, do-better-next-time or a negative, don't-do-that-again way (fwiw, I agree that the positive way is better) is orthogonal to the argument I was trying to make about who is the recipient of that reaction. Scapegoating the agent is farcical. Clearly, "make no mistakes" style prompts are a poor approach to agentic development.
> the manager's job is to enable career progression of their reports... we aren't enabling the career progression of [fancy calculators]
While I do agree that good human managers concern themselves with career development of their human direct reports, I disagree that it's a fundamental part of the job description. The foundation is for a manager to get and keep their reports being productive. Human workers are often concerned with career progression, so good managers concern themselves with such career progression as well. Agents are not, but that doesn't mean that lauding good agentic work is meaningless. Lauding the work is also valuable for: (a) reinforcing a culture of positivity and celebration, which rubs off on human workers as well, (b) part of a culture of continuous evaluation of agents and models - asking questions like, are there cheaper or faster models that would be just as effective?
While it seems reasonable that you need humans to take accountability, it looks like a very shaky position to assert that computers can't make decisions. And going back to chess computers, we acknowledge that computers make better decisions than humans in that area.
Forbidding computers that can make better decisions than humans from making decisions to keep accountability just seems to be arguing to hire fall guys. You still want the better decision maker to make decisions, but you need someone there to take the fall if things go wrong.
Very weird incentives.
It does not matter if the CEO is an AI or a human. In either case, of course they can make decisions -- and be held responsible. Not in any emotional sense (that we seem to want to bake into the concept for added confusion), just that if you are not satisfied with the result, you can replace the human or AI.
I am not saying that this is a fun vision of anything, but why are we making it more complicated than it is? The parts are all there and explained.
My understanding is that 'emotional' sense emerged organically in LLMs and was not intentionally baked in.
As silly as it may seem, the ways emotions come through in our words do seem to have an effect on agents. Likely because they were trained on human writing, most of which cannot be separated from the emotions of the writers any more than your emotions can be separated from your logic & reasoning abilities.
They generate text which resembles reasoning and may include a tool call
The harness runs tool calls
The human made the decision to write the harness
The human is responsible
Also phrased as ‘to make an apple pie from scratch, one must first create the Universe’
There is no end of the world, nor autonomous agent decisions nor any fantasies they are building up to make people believe they have a pandora box in their hand.
What we instead have are lies crafted to lure non tech people and keep this running as long as they can.
Oh and let's also not forget that they are allowed to illegally download basically as much as they want from libgen/annas archive/torrents to train their models under the "fair use" umbrella, yet mere mortals can go to jail for way much less.