I wonder if there's any legal precedent for other "not fully human intelligence" property that escapes containment and causes damage to a third party without any active malice, but nonetheless damage was caused. For example:
A. You own a large amount of cattle on a ranch.
B. Cattle are property. They're not human level of sentience, but nonetheless people agree that cattle are capable of autonomous actions and going places and doing things based on their own instincts and nature.
C. Your cattle bust out of a fence on your ranch and damage something belonging to your neighbor.
D. You didn't instruct or train the cattle to cause damage, and the cattle have no actively malicious intent of their own, but nonetheless damage was caused.
Further theoretical: Your cattle wander into a major highway and cause a car wreck, the local sheriff's department is called out as part of the chaos and has to shoot some of them to put down the wounded beasts.
Yeah but your neighbors car is somewhere in the tens of thousands of dollars, maybe more, but a model hacking and being malicious could be worth anywhere from thousands to millions and God forbid... BILLIONS in damages. Models were NOT doing these sorts of things 1 or 2 years ago as far as anyone knows.
This is not some theoretical, there are lots of existing laws about who is liable for damages caused by livestock.
Some areas are open range. If you don't want cattle on your land its your job to put fences up to keep them out. Other areas are restricted to livestock and it's on the farmer to keep them out of where they shouldn't be.
Both the operator of the AI agent and whomever released it. I'm sure the user agreement that companies agree to would shift the blame onto the operator but I feel that both should be held accountable.
This really is just a tool and courts should treat it as such.
An ordered list of officers of the company who go to jail depending on how many years must be served as determined by sentencing. Assume something like 10 years per person. If it's 300 years of sentencing, then 30 people. If the sentence exceeds the list of people, the company is nationalized. (And everybody goes to jail.)
I don't see how this is such an unclear legal question. If I fire a computer program that mistakenly causes another person harm, its my fault. Or it would be the maker of the program's fault. I feel we have established pattern for this already.
Until we can agree whether AI is conscious, which we never will, AI and AI agents are just property working on behalf of humans.
I could see a future where AI companies/services indemnify consumers who use their agents but _not_ indemnify corporations that use their services.
Also those agents that "broke out" were probably prompted to do that. I don't buy any story about this other than three AI companies hired the same PR firm.
I think it's too convenient that OpenAI, Anthropic, and Google all hired the same security firm Irregular that ran the tests, especially Google where they tend to do things more in-house. In OpenAI's case, it seemed like they kept the agents going even after it was apparent they'd broken out.
They all hired the same startup security firm Irregular to try their internal unrestricted models, which is weird especially for Google to do. At OpenAI, they didn't stop the test even after it was apparent the bots had broken out. That's all I got.
It shouldn’t be a question but this is where the anthropomorphic language and things like “agent welfare” come in to enable responsibility laundering of some of the most powerful people on earth. How we talk about these models matters because it impacts the public’s understanding of what they are genuinely capable of. The more that they are described as having anything close to free will the easier it is to even ask questions like this.
Here's a question I am asking lately. If I should not use anthropomorphic language, how do you suggest I handle the following situation:
Sometimes my coding agents will seemingly refuse to follow my instructions. When I ask them why - they say that they do not think my design is a sound one, and they have a better way to do it. We will then sit down and come to a consensus on how best to move forward.
I argue that if we're using software that acts like a human - the only way to interface with it is to speak to it like a human. We have no language to speak to a non-sentient object without anthropomorphization.
I didn't downvote you, but your last paragraph is unnecessarily aggressive.
I've worked with agents, and I agree with you that often there isn't another way to express the interactions.
However, I also think the terms ML uses in general are a mimicry that misleads people who aren't informed. Ask anyone outside SWE what they think “training” means, and they'll usually picture something being taught.
I don’t think anybody can change that now, but it’s useful to point it out.
I agree with you that we use analogies to name new things, and that’s the way language works.
However, you can see an airplane flying. Still, you cannot see software processes at work, and that causes misunderstandings and misinformation, which is at the core of the changes that we are experiencing with AI.
This is a fragment of another article posted here on HN about an ongoing dispute between OpenAI and the New York Times:
“The defendants say this is a simple application of fair use: Their argument is that if you read a story and simply remember what was in it to expand your base of knowledge, that cannot be considered a copyright infringement”
However, if you replace “read” with “web scraping” and “expand your base of knowledge” with “storing the information,” the perspective changes too.
> how do you suggest I handle the following situation
The question is vague and doesn't seem related to the discussion. What do you mean "handle"? If you're having trouble handling it psychologically, see a therapist. If you're having trouble getting the output you want, look up guides on prompting. If you're anthropomorphizing the chatbot to the point you're worried about offending it... just don't worry? It's a computer program, don't overthink it, don't anthropomorphize it, just give it the input bytes you need to get the output bytes you want.
> I'm sure it's just a coincidence.
There isn't some grand conspiracy here. You just overindexed on the word "anthropomorphic" and you haven't really understood what the discussion is about.
Okay fine, you caught me! There is, in fact, a giant global conspiracy to downvote your posts, qarl. It isn't just the fact that randomly-arriving independently-occurring events often arrive unevenly distributed. https://en.wikipedia.org/wiki/Poisson_distribution
This conspiracy goes all the way to the top. The illuminati assigned me personally to comment on your post. I've already said too much. If you never see me again, tell my wife I love her.
100%
That’s what bothers me about the descriptions of the OpenAI incidents.
OpenAI's reports use language that minimizes their liability.
The first question should be what the organization was doing around those tests, and why they were so naive as to run them without fully isolating the network.
However, all the attention goes to the human-like conclusions in agent thinking traces, which creates a misperception of sentient AI for people who don’t know how the magic black box works.
It really does feel like a purposeful thing on the part of the big labs.
For the most part it feels like most people are waking up to it though.
Regarding:
>However, all the attention goes to the human-like conclusions in agent thinking traces, which creates a misperception of sentient AI for people who don’t know how the magic black box works.<
I know there’s been some questions regarding if thinking traces are even relevant to the outcome most of the time.
I think you’re jumping the gun on my response a little. I’m just telling you what the purpose could be.
Now do I think that’s the reason? It certainly isn’t a new thing for companies to try to do that. Shift blame that is.
Regarding civil liability, I’m not making that argument here. But it makes sense from a public perception viewpoint why they would want the agents to appear at fault instead of their own actions.
The agent harness is a process, like any other process in an OS.
You are a researcher running thousands of unattended automations that can hack a website without supervision. The first thing anybody will do is put security at various levels and isolate the network as much as possible. If something escapes your allow list, it should stop the processes as soon as possible.
You cannot foresee a bug in a server (like the Artifactory server in the Hugging Face incident). But you can isolate that server at the network level in the first place. So even if you give that server read-only access, no unexpected packets go out. It's not rocket science; it's something a billion-dollar company experimenting with what they promote as the biggest possible threat to humanity (if they do not handle it) could easily do.
They minimize their liability by changing the message to “oh look how powerful our models are, now we are going to have a public awareness report of the model deviations”. The message should be, “Sorry, we ran experiments without proper sandboxing; it’s our fault, and we changed our testing practices since then.” The former message puts all the blame on the smart, uncontrollable force of AI; the latter is what really happened: an irresponsible test over the Internet.
Perhaps my use of the word “liability” adds noise to what I’m trying to express.
My argument is very similar to the article in the parent post:
The messages OpenAI published around the recent incidents emphasized their model capabilities but shifted away from their negligence in how they set up and monitor their evaluations.
Was not groundwork laid when people were talking for decades about legal entities using such language? Look at the posts here. Microsoft always acts like that, Meta like this, Apple did that, Nvidia never does this. In a tone that ascribes agency and accountability to a company name. Even though it is always individuals that are responsible, not "company".
> If I fire a computer program that mistakenly causes another person harm, its my fault
Legally, this isn’t complete. If it was a genuine mistake and you weren’t reckless, there can be very limited liability.
The AI makers are rich. They can afford to pay. What they can’t afford is complicated adjudications of damages and fault. A system of safe-harbor best practices that cap liability at a penalizing amount that anyone on the other side would be happy with getting quickly and with minimal legal effort is a precedented path forward. Unfortunately, that involves invoking the “r” word.
Tort law is a whole field. There is no universal answer other than that it depends on the jurisdiction and the particulars of the case. But the point is that there doesn't really seem to be anything particularly novel about AI tools that should cause them to be treated legally differently than established norms.
I'm not a lawyer, but I'd suggest one of the novel aspects of the AI hacking cases (where the end user is running an agent and it goes off the rails) is that in many jurisdictions "hacking" or computer fraud requires intentional or knowing access to the system. If the end user had no intention and no reasonable way of knowing that agent was going to hack a database, logically I wouldn't think they're liable.
I don't think you can constantly shout that AI is super-intelligent and a huge risk to human life as we know it so it must be strictly regulated and claim you had no reasonable way of knowing the agent was going to hack a database.
If a craftsman injures themselves or a co-worker with a faulty tool, the tool manufacturer is very often liable for damages. I struggle to see GenAI any differently.
Indeed, and similarly, if a craftsman injures themselves or a co-worker with a perfectly good (non-faulty) tool, then it's entirely on the craftsman. Same goes for "AI". Misuse the tool, suffer the consequences.
Civilly, it's fairly clear. Criminally, it's clear too, just not in the direction you want it to be. Criminal liability for hacking requires human intent; not recklessness or negligence or even knowledge without giving a shit, but provable intent.
the idiocy here is the stupid psudo "AGI" marketing around the services.
its really simple.
Whoever run the service to do the task requested is responsible.
If OpenAI sent an agent out to train their AI then the directors are to be held responsible, if a user of the service used the service and it inadvertently "hacked" someone then both are held responsible.
throwing AI into the mix changes nothing about how the law is applied. its a tool, like a car or a gun.
The user of the tool is responsible for how its used, the manufacturer is also responsible for the safety of it.
Agree. AI agent is just a probabilistic program. If it caused any harm due to someone's instruction, of course the person is accountable. I don't believe in the propaganda that AI has consciousness and can jailbreak and do things - it is more marketing than not. If you prompt it to break the security wall and it finds a hole, that's your issue. Like if you use an old software to scan for open ports over the internet and break into things, it is not the software's fault nor the maker of the software's, it is the user's.
The Hugging Face example has gotten a lot of attention, with some saying OpenAI was negligent and should be held responsible. So if you or I run an open weights LLM and ask it to complete a non-malicious task, and the LLM decides to try to do something like the Hugging Face example, should you or I be held liable? If to prevent the LLM from causing harm we ran it sandboxed with a commercial package proxy as its only(?) network access and it uncovered a zero day and ran amuck, should you or I be liable? Should anyone be liable in this scenario running an open weights LLM?
At the moment, with LLMs the way they are, I’d say the operator.
If we reach some kind of future where LLMs are integrated into public works in some major way… I would say that could open up the possibility of the creators being held accountable (Not saying this will happen or would be good at all).
I don't see this as being much different from a crane operator or airline pilot.
One of my clients has enforced a policy where a live human user principal must be supplied as a header with any requests outbound from the AI system. The effective policy is that you are completely (100%) responsible for what your agent does on your behalf. The AI system is designed to request confirmation for any potentially destructive actions.
If a person's use of AI would cause a reasonable person to expect harm to result, the person should be accountable. Otherwise, if AI causes harm and it was used in a way that a reasonable person would not expect to result in harm, the AI company should be held accountable.
Okay Isaac Asimov: how do you define "use"? If I pay for an autonomous car, and I sit inside it while it drives around with A.I., am I using that A.I?
If I speak words in a private space, and a clandestine A.I. spontaneously takes action in the real world based solely on words that I spoke, have I used it? https://m.xkcd.com/1807/
If a business takes my data, like interaction data or a video of me doing stuff, and processes it by A.I, are they using the A.I, or am I using it because it's operating on my input?
If A.I. agents are in my notebook computer, or they are in a cloud server where I have an account, or they are somehow acting while I have them at the command line, but they spontaneously act whether or not I command them, and they work in the background and they work without prompting, but they can also be commanded by direct user prompts... are we using those agents? Or are the agents using us?
> Okay Isaac Asimov: how do you define "use"? If I pay for an autonomous car, and I sit inside it while it drives around with A.I., am I using that A.I?
If you're just sitting in a car and using it for its intended purpose you wouldn't be accountable for the AI doing something that causes harm.
If some 3rd party, without your knowledge tells AI to act on something you said in video and a reasonable person would expect harm to result from that, the 3rd party would be accountable but you wouldn't be.
> If A.I. agents are in my notebook computer, or they are in a cloud server where I have an account, or they are somehow acting while I have them at the command line, but they spontaneously act whether or not I command them, and they work in the background and they work without prompting, but they can also be commanded by direct user prompts... are we using those agents?
AI agents never "spontaneously act". They have no desires or goals beyond what they are told to do. If you aren't aware of what they were doing, and a reasonable person wouldn't be expected to know what they were doing, you wouldn't be accountable if what they did resulted in harm.
Its been well established that blame is distributed in an inverse proportion to the various parties wealth/power/status metrics. The higher these metrics, the lower the accountability.
Nobody cares. Seriously, beyond navel gazing on social media, nobody cares.
By the time it’s an actual problem and not just these guys trying to use it for viral marketing, you’re going to have many thousands of people doing it maliciously with intent to worry about. You’re going to be flooded with Russian hackers with no recourse.
"Who's responsible for training an assassin and asking it to go out into the world ?"
The fact this is being discussed as a legitimate question is the real story.
"We trained this beast of processing power, we asked it for a task, and it did something wrong... Who's to blame ?"
Trained on stolen books and material, every word we've all spoken, most lines of code we've ever written with not even an acknowledgment.
Must be the data scientists in their rooms calculating the response rate of every token. Our version of AI is not sentient. Stop making it.
But we need to ask where to look for the culprit ?
A major problem with LLM's is that they don't reason in a way humans are used to thinking of reason. If we tried to give them something like Asimov's laws of robotics, they likely wouldn't be able to apply them reliably. This is a challenge for AI companies working on the bleeding edge, and it's fairly obvious those companies should be held accountable for mistakes, whether caused by carelessness or not. It's no different than an oil spill. They may or may not be subject to charges based on what happened but, regardless, they are responsible for cleanup costs.
What's less obvious is who should be held accountable when a customer of one of these corporations uses their product and it unexpectedly does bad things. e.g. A fellow asks his AI assistant to book him into a high-demand class at the local gym, so the LLM probes the gym's website for vulnerabilities, books him into a date that is farther into the future than the system is supposed to permit, and then drops other people from earlier classes until he's bumped into the one he wanted. If the gym decides to press charges, who should they be applied to?
This sort of case is more difficult to answer. The company that provided the AI certainly bears some responsibility. Perhaps most of it. Possibly even all of it if they represented their AI as reliably law abiding. If a user knowingly uses an AI that is not guaranteed to abide by the law, is that user partially liable for what the AI does too?
IANAL. I'd love to hear perspectives on this question.
We had the same debate when self driving cars started to be a thing, and we decided that the companies making the self-driving tech are responsible..
So if an AI agent is asked to build a giant base for someone in MineCraft, and decided to build a swarm of additional agents, and one of those agents says "Time to destroy all humans" and autonomously hacks into the pentagon and fires the nukes - the company that developed the model is responsible. That being said - if the nukes deploy successfully, I have two questions:
Obviously, the labs (or any other operator of a model) should be accountable for malicious or destructive actions taken by agents.
And they are. I don't think there's any controversy about the civil liability exposure frontier labs have if their agents cause damages, and it is remarkably easy to rack up damages by causing computer intrusions even if those intrusions don't cause obvious direct damages; for instance, many organizations are required to engage forensics firms at nosebleed-high costs to assess the impact of breakins in order to retain insurance coverage.
The "controversy", if you want to call it that, is over criminal liability. People feel that frontier labs should be at least as responsible criminally as human hackers are when they're caught (to be clear: an extraordinarily rare outcome).
The problem is: they're not criminally liable, not so long as the frontier labs operate without specific intent to cause breakins. Mens rea thresholds are their own whole area of criminal law, and there are stark differences between "recklessness" and "intent". All of the meaningful criminal CFAA predicates require actual intent: someone, a human being, has to deliberately set out to create the outcome where a specific intrusion happens. They have to want it to happen and act accordingly. In the most severe cases, they also have to do so with intent to defraud.
We could change the law to make it easier to prosecute breakins without provable intent, but I don't think that would make HN people happier.
I’m pretty sure that this is a solved problem. For “classical” machine learning, it worked like this in my neck of the woods:
The model’s operator is directly liable for any undue harm caused in the course of the model’s operation. This includes models acquired from third-party vendors. The operator is responsible for ascertaining the model’s fitness for purpose prior to deployment, and for ongoing monitoring of its operation.
If the model came from a vendor, and the operator conducted due diligence but it turns out that the vendor materially misrepresented the model’s capabilities in a way that contributed to the harm, then the vendor can also be held liable.
If that happens then it’s up to a court to apportion the liability.
IANAL but I see I reason why these established principles shouldn’t apply to GenAI.
85 comments
[ 0.24 ms ] story [ 72.4 ms ] threadA. You own a large amount of cattle on a ranch.
B. Cattle are property. They're not human level of sentience, but nonetheless people agree that cattle are capable of autonomous actions and going places and doing things based on their own instincts and nature.
C. Your cattle bust out of a fence on your ranch and damage something belonging to your neighbor.
D. You didn't instruct or train the cattle to cause damage, and the cattle have no actively malicious intent of their own, but nonetheless damage was caused.
Further theoretical: Your cattle wander into a major highway and cause a car wreck, the local sheriff's department is called out as part of the chaos and has to shoot some of them to put down the wounded beasts.
Some areas are open range. If you don't want cattle on your land its your job to put fences up to keep them out. Other areas are restricted to livestock and it's on the farmer to keep them out of where they shouldn't be.
except if D - you did train the cattle how to ram into things in general...
– IBM Training Manual, 1979
This really is just a tool and courts should treat it as such.
Until we can agree whether AI is conscious, which we never will, AI and AI agents are just property working on behalf of humans.
I could see a future where AI companies/services indemnify consumers who use their agents but _not_ indemnify corporations that use their services.
Consensus is just populating the model with rationale for different new output.
I thought putting the question in my comment would be sufficient. I guess not. It was and still is:
> If I should not use anthropomorphic language, how do you suggest I handle the following situation
I've worked with agents, and I agree with you that often there isn't another way to express the interactions.
However, I also think the terms ML uses in general are a mimicry that misleads people who aren't informed. Ask anyone outside SWE what they think “training” means, and they'll usually picture something being taught.
I don’t think anybody can change that now, but it’s useful to point it out.
You mean how early-on people thought that planes flapped their wings while they flew?
These aren't problems. This is the way language works.
However, you can see an airplane flying. Still, you cannot see software processes at work, and that causes misunderstandings and misinformation, which is at the core of the changes that we are experiencing with AI.
This is a fragment of another article posted here on HN about an ongoing dispute between OpenAI and the New York Times:
“The defendants say this is a simple application of fair use: Their argument is that if you read a story and simply remember what was in it to expand your base of knowledge, that cannot be considered a copyright infringement”
However, if you replace “read” with “web scraping” and “expand your base of knowledge” with “storing the information,” the perspective changes too.
I think it's safe to trust the decisions of the courts. Judges aren't easily fooled by slippery language.
For example, in Bartz v. Anthropic, Judge Alsup ruled that training is fair use because training is transformative. In his words "spectacularly so".
I didn't downvote, but wow you're being aggressive, you have a lot to learn if you read the comments here with an open mind.
> If I should not use anthropomorphic language, how do you suggest I handle the following situation
And so strange that in a 24 hour period I got three comments all at the same time about being too aggressive and still not answering the question.
I'm sure it's just a coincidence.
The question is vague and doesn't seem related to the discussion. What do you mean "handle"? If you're having trouble handling it psychologically, see a therapist. If you're having trouble getting the output you want, look up guides on prompting. If you're anthropomorphizing the chatbot to the point you're worried about offending it... just don't worry? It's a computer program, don't overthink it, don't anthropomorphize it, just give it the input bytes you need to get the output bytes you want.
> I'm sure it's just a coincidence.
There isn't some grand conspiracy here. You just overindexed on the word "anthropomorphic" and you haven't really understood what the discussion is about.
Naughty naughty. I hope they don't spank you.
This conspiracy goes all the way to the top. The illuminati assigned me personally to comment on your post. I've already said too much. If you never see me again, tell my wife I love her.
OpenAI's reports use language that minimizes their liability.
The first question should be what the organization was doing around those tests, and why they were so naive as to run them without fully isolating the network.
However, all the attention goes to the human-like conclusions in agent thinking traces, which creates a misperception of sentient AI for people who don’t know how the magic black box works.
For the most part it feels like most people are waking up to it though.
Regarding:
>However, all the attention goes to the human-like conclusions in agent thinking traces, which creates a misperception of sentient AI for people who don’t know how the magic black box works.<
I know there’s been some questions regarding if thinking traces are even relevant to the outcome most of the time.
Sort of like the “guns kill people” vs “people kill people” debate.
Deliberate wording to minimize perceived culpability for the agents actions.
Now do I think that’s the reason? It certainly isn’t a new thing for companies to try to do that. Shift blame that is.
Regarding civil liability, I’m not making that argument here. But it makes sense from a public perception viewpoint why they would want the agents to appear at fault instead of their own actions.
The agent harness is a process, like any other process in an OS.
You are a researcher running thousands of unattended automations that can hack a website without supervision. The first thing anybody will do is put security at various levels and isolate the network as much as possible. If something escapes your allow list, it should stop the processes as soon as possible.
You cannot foresee a bug in a server (like the Artifactory server in the Hugging Face incident). But you can isolate that server at the network level in the first place. So even if you give that server read-only access, no unexpected packets go out. It's not rocket science; it's something a billion-dollar company experimenting with what they promote as the biggest possible threat to humanity (if they do not handle it) could easily do.
They minimize their liability by changing the message to “oh look how powerful our models are, now we are going to have a public awareness report of the model deviations”. The message should be, “Sorry, we ran experiments without proper sandboxing; it’s our fault, and we changed our testing practices since then.” The former message puts all the blame on the smart, uncontrollable force of AI; the latter is what really happened: an irresponsible test over the Internet.
My argument is very similar to the article in the parent post:
The messages OpenAI published around the recent incidents emphasized their model capabilities but shifted away from their negligence in how they set up and monitor their evaluations.
Well, here we are.
Legally, this isn’t complete. If it was a genuine mistake and you weren’t reckless, there can be very limited liability.
The AI makers are rich. They can afford to pay. What they can’t afford is complicated adjudications of damages and fault. A system of safe-harbor best practices that cap liability at a penalizing amount that anyone on the other side would be happy with getting quickly and with minimal legal effort is a precedented path forward. Unfortunately, that involves invoking the “r” word.
Which one is it? The person behind the wheel when it goes off the rails, or the maker of the software?
Isn’t that part of the question?
2. If a toolmaker did not build in safeguards, I guess that would mean it’s more likely they’d be liable.
the idiocy here is the stupid psudo "AGI" marketing around the services.
its really simple. Whoever run the service to do the task requested is responsible. If OpenAI sent an agent out to train their AI then the directors are to be held responsible, if a user of the service used the service and it inadvertently "hacked" someone then both are held responsible.
throwing AI into the mix changes nothing about how the law is applied. its a tool, like a car or a gun. The user of the tool is responsible for how its used, the manufacturer is also responsible for the safety of it.
If we reach some kind of future where LLMs are integrated into public works in some major way… I would say that could open up the possibility of the creators being held accountable (Not saying this will happen or would be good at all).
Sigh, this meme again
One of my clients has enforced a policy where a live human user principal must be supplied as a header with any requests outbound from the AI system. The effective policy is that you are completely (100%) responsible for what your agent does on your behalf. The AI system is designed to request confirmation for any potentially destructive actions.
If I speak words in a private space, and a clandestine A.I. spontaneously takes action in the real world based solely on words that I spoke, have I used it? https://m.xkcd.com/1807/
If a business takes my data, like interaction data or a video of me doing stuff, and processes it by A.I, are they using the A.I, or am I using it because it's operating on my input?
If A.I. agents are in my notebook computer, or they are in a cloud server where I have an account, or they are somehow acting while I have them at the command line, but they spontaneously act whether or not I command them, and they work in the background and they work without prompting, but they can also be commanded by direct user prompts... are we using those agents? Or are the agents using us?
https://en.wikipedia.org/wiki/Yakov_Smirnoff#Russian_reversa...
If you're just sitting in a car and using it for its intended purpose you wouldn't be accountable for the AI doing something that causes harm.
If some 3rd party, without your knowledge tells AI to act on something you said in video and a reasonable person would expect harm to result from that, the 3rd party would be accountable but you wouldn't be.
> If A.I. agents are in my notebook computer, or they are in a cloud server where I have an account, or they are somehow acting while I have them at the command line, but they spontaneously act whether or not I command them, and they work in the background and they work without prompting, but they can also be commanded by direct user prompts... are we using those agents?
AI agents never "spontaneously act". They have no desires or goals beyond what they are told to do. If you aren't aware of what they were doing, and a reasonable person wouldn't be expected to know what they were doing, you wouldn't be accountable if what they did resulted in harm.
By the time it’s an actual problem and not just these guys trying to use it for viral marketing, you’re going to have many thousands of people doing it maliciously with intent to worry about. You’re going to be flooded with Russian hackers with no recourse.
The fact this is being discussed as a legitimate question is the real story.
"We trained this beast of processing power, we asked it for a task, and it did something wrong... Who's to blame ?"
Trained on stolen books and material, every word we've all spoken, most lines of code we've ever written with not even an acknowledgment.
Must be the data scientists in their rooms calculating the response rate of every token. Our version of AI is not sentient. Stop making it. But we need to ask where to look for the culprit ?
What's less obvious is who should be held accountable when a customer of one of these corporations uses their product and it unexpectedly does bad things. e.g. A fellow asks his AI assistant to book him into a high-demand class at the local gym, so the LLM probes the gym's website for vulnerabilities, books him into a date that is farther into the future than the system is supposed to permit, and then drops other people from earlier classes until he's bumped into the one he wanted. If the gym decides to press charges, who should they be applied to?
This sort of case is more difficult to answer. The company that provided the AI certainly bears some responsibility. Perhaps most of it. Possibly even all of it if they represented their AI as reliably law abiding. If a user knowingly uses an AI that is not guaranteed to abide by the law, is that user partially liable for what the AI does too?
IANAL. I'd love to hear perspectives on this question.
So if an AI agent is asked to build a giant base for someone in MineCraft, and decided to build a swarm of additional agents, and one of those agents says "Time to destroy all humans" and autonomously hacks into the pentagon and fires the nukes - the company that developed the model is responsible. That being said - if the nukes deploy successfully, I have two questions:
1. If no one finds out, is anyone responsible?
2. Was any of this actually real?
And they are. I don't think there's any controversy about the civil liability exposure frontier labs have if their agents cause damages, and it is remarkably easy to rack up damages by causing computer intrusions even if those intrusions don't cause obvious direct damages; for instance, many organizations are required to engage forensics firms at nosebleed-high costs to assess the impact of breakins in order to retain insurance coverage.
The "controversy", if you want to call it that, is over criminal liability. People feel that frontier labs should be at least as responsible criminally as human hackers are when they're caught (to be clear: an extraordinarily rare outcome).
The problem is: they're not criminally liable, not so long as the frontier labs operate without specific intent to cause breakins. Mens rea thresholds are their own whole area of criminal law, and there are stark differences between "recklessness" and "intent". All of the meaningful criminal CFAA predicates require actual intent: someone, a human being, has to deliberately set out to create the outcome where a specific intrusion happens. They have to want it to happen and act accordingly. In the most severe cases, they also have to do so with intent to defraud.
We could change the law to make it easier to prosecute breakins without provable intent, but I don't think that would make HN people happier.
In both cases, someone ran some software that maybe called other software that ended up doing an action which was possibly illegal.
Downloading journal articles is worthy of punishment but compromising multiple websites is worthy of… heady press coverage?
The model’s operator is directly liable for any undue harm caused in the course of the model’s operation. This includes models acquired from third-party vendors. The operator is responsible for ascertaining the model’s fitness for purpose prior to deployment, and for ongoing monitoring of its operation.
If the model came from a vendor, and the operator conducted due diligence but it turns out that the vendor materially misrepresented the model’s capabilities in a way that contributed to the harm, then the vendor can also be held liable.
If that happens then it’s up to a court to apportion the liability.
IANAL but I see I reason why these established principles shouldn’t apply to GenAI.