> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Thought crimes are real when you're sharing your thoughts with Claude
It is unlawful for any person to send, post, or transmit, or procure the sending, posting, or transmission of, a writing or other record, including an electronic record, in any manner in which it may be viewed by another person, when in such writing or record the person makes a threat to: (a) Kill or to do bodily harm to another person; or (b) Conduct a mass shooting or an act of terrorism.
To me, that is the more interesting legal question. Does a LLM-based safety net that sends content to a human, when the original use case would not have sent it to a human, count as "may be viewed by another person". It certainly wasn't intended to be, and that isn't the norm. At the same time, because no security is perfect, we could say that any digital record, stored in any way "may be viewed by another person."
The usual thing that makes laws against criminal conspiracy pass First Amendment muster is that the words have to be combined with some concrete acts furthering the criminal conspiracy. That might just be something as otherwise innocuous as looking up the blueprints of the bank you talked about robbing but it has to be something other than just talk.
Lotsa words in that law... except they don't define what a "threat" may be.
If you enter my house I will k!ll you. <- is this a threat? noone knows. The interpretation of the word "threat" is unknown. Besides, is a conditional a statement? who knows. But sure, blah blah... "in any manner in which it may..." these words are putting me to sleep.
I have some sympathy for Anthropic here because I've seen the headlines after OpenAI failed to report a shooter in a similar situation. So from their perspective, it's damned-if-you-don't, damned-if-you-do.
However, people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech. Before LLMs, Big Tech had no way to scrutinize the bulk of what was going on within their services, so you could have a secret hate diary in Google Docs. Now, everything you say or write can be automatically screened for red flags on a planetary scale, and probably will be because that's what the regulators and "concerned citizens" will demand. In a couple of years, you'll be biting your tongue a lot more often in private chats.
And such was the case for a man who snapped a photo of his own child to send to the doctor which got uploaded to his Google photos resulting in his Google account of over a decade getting shutdown for CSAM.
As another commenter said, you're not chatting with a friend; you're chatting with Big Tech.
Yes, Anthropic has to follow state and federal laws, even if they're stupid, plainly unconstitutional laws. But they don't help themselves in the way that they lean into the "Sorry Dave, I can't do that" aspect of "AI safety" that has made thoughtcrime tangible. Deity forbid that you mis-phrase your innocent query in a way that triggers whatever internal signals are "problematic" to these providers.
Every AI bot does this idiotic stuff, but Anthropic is the worst, and so it makes it one doubt their...let's call it "alignment"...with free-speech principles.
The main thing I fear from AI is not the made-up chance of the extermination of humanity, but rather, the increasing reach of the judgmental woke surveillance state. Remember when you could ask Google anything, and not worry about some bot criticizing you for wrongthink?
> people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech
Yup. And with zero privacy protections in statute for AI chat, there is nothing to prevent an AI CEO looking to curry political favour from e.g. handing over the private correspondence of an opponent or an entire district’s residents.
Don't know why this got downvoted? This is a problem with epistemics.
It's more efficient to have one central "verifier" for everything, but the "who watches the watchers"? question basically says: Either constrain by construction, have everyone verify (which are two sides of the same coin, btw, when looking at a "global" thing), or centralize explicitly.
Customers who what privacy protections for AI chats are welcome to negotiate this in enterprise contracts. The major LLM vendors do offer that as an option. Customers can then enforce any violations in civil court (although this obviously wouldn't apply if the customer used the LLM for criminal purposes).
The more sensible thing would be to link to any example of any lab anywhere offering enterprise solutions—with the opportunity to negotiate—to individuals.
People are confused at your take because it’s based on a reality that so far you’ve only described as the better/obvious option for individuals. Companies treat corp customers and individual customers differently. This isn’t new.
My company did negotiate the data policy amongst other things for our enterprise plan, I was involved in the process. I can say with certainty that they’re not picking up the phone for _one dude_ who wants the same thing.
Individuals can sign up for enterprise agreements. An enterprise can consist of one person. Or run a local model.
It seems like some people seem to think that a chatbot should guarantee total privacy like a confession to a Catholic priest or something. That is ludicrous and unrealistic. It's a commercial service subject to terms and conditions. If you don't like it then don't use it.
> Individuals can sign up for enterprise agreements. An enterprise can consist of one person.
Custom negotiation is a big difficulty for individuals and the pricing will probably have fixed overhead that becomes pretty ridiculous when it's not spread over a hundred or thousand users.
> Or run a local model.
Much better advice. But still an obnoxious amount of effort and restriction to avoid spying.
> It seems like some people seem to think that a chatbot should guarantee total privacy like a confession to a Catholic priest or something. That is ludicrous and unrealistic. It's a commercial service subject to terms and conditions. If you don't like it then don't use it.
I want it to have the same privacy as using a local program. Does it still sound ludicrous if I put it that way? I'm not talking to a person that would have to suppress their own conversations later, and there's no need for any data from this chat to escape into the outside world. It's like wanting a phone line where the phone company won't listen in; even if doing that is standard procedure and I can walk away it's still gross.
In the case that the company running this program for me is served a warrant, I accept that saved data would be shared. But proactively screening my data for a reason that isn't to help me is bad. We should fight to make that easier to avoid.
Also you're talking out both sides of your mouth when you say it's "ludicrous and unrealistic" right after telling me I can negotiate that exact feature in an enterprise contract.
Yes, it still sounds ludicrous if you put it that way. If you want a contractual agreement for privacy then pay for it. Everything is available — for a price. But a civil agreement can't bind a counterparty to conceal evidence of a crime.
Privacy should be an offer from the start if it's not mandatory. It shouldn't need special negotiation.
> But a civil agreement can't bind a counterparty to conceal evidence of a crime.
Are you talking about a different scenario? In a chatbot situation, the crime of "threat" requires snooping to even be possible. If the data is private there is no crime to begin with.
I suspect it will go further: imagine giving an mp3 to LLM to clean up some noise. It detects it was illegally downloaded from youtube, deletes it and automatically fines you via attached credit card.
Or worse: downloading a picture of pirate ship and without any concern for the copyright asking the LLM to make a coloring page for your kid. BTW Chatgpt does that way better than Claude
They could be happy to assist you with a task that they also then flag and forward to authorities, especially if their assistance doesn't break the law but has evidence of you doing so.
Ooo, have you read any articles on that happening? I missed them! Given that this is specifically a comment chain in a discussion about them working with authorities, that they'd work with authorities hardly seems like science fiction.
I had to move some games from C:\Games\RandomWarezGroup\GameName to my Steam directory, because Claude refused to reverse engineer an asset format due to piracy being incredibly immoral and illegal and blah blah.
Hey but you’ll be allowed to file a response that will also go to an LLM and deny you automatically. And you will be charge a NSE fee (no sufficient explanation).
It's a byproduct of the nannyism safety marketing from the AI companies. I'm glad these cases were caught, but disagree with how they were disposed of. If the automated flagging is good, enforce it by default. If it's noisy, refine the tech then enforce it by default. This middleground where everything going through the platforms is subject to training and arbitrary human inspection in the midst of an acrid cloud of marketing-driven fearmongering is unacceptable, and it reinforces the idea the fearmongering is legitimate.
Somehow humanity survived the past 40 years without Microsoft Word and Excel phoning home and shopping users to the feds at random, I don't see why the standard should be any different for this new class of tooling.
They're actively rummaging through your inputs so the damned-if-you-don't case doesn't really exist; no one expects Anthropic to not notify law enforcement once they learn of something like this. What you might have expected was some privacy in the first place though, where Anthropic would never have learned of this in the first place and where the damned-if-you-do case wasn't a thing.
You should probably get a head start on waiting a couple years to bite your tongue and assume everything you type into a computer is summarized and sent to your boss, government, advertisers, political actors, insurance companies, worst enemy, etc. With phones, Alexas, and little AI tamagotchis, you probably shouldn't say much in person either.
They have some responsibility for people's expectations of the product, at least. They want the personal assistant personas to be able to help you with anything, and don't point out that they'll be judging your thoughts along the way.
For anyone technically inclined it should be obvious, but it isn't part of the zeitgeist or how they pitch it. People see it as being different than talking to a human, and behave as if there won't be a human in the mix.
We really need a class, probably in high school, that works through how LLMs work at the high level (don't need to get too far into the deep math, but give people a taste) and then how they're trained, used, and deployed.
I feel like if people understood what these things actually are there'd be way less of this AI psychosis and similar stuff.
There'd also be fewer people falling for apocalyptic Rationalist delusions.
Also: people need to understand "not your computer, not your data." (Unless it's stored in the cloud but encrypted locally with keys only you possess.) Same goes for storing things unencrypted in OneDrive, Google Drive, etc. There is nothing to stop these companies from bulk scanning, data mining, or reporting people based on whatever request a government gives them. Don't count on them to resist, because they often can't, especially if the request is from a sovereign state where they do business.
There'd also be fewer people falling for apocalyptic Rationalist delusions
Assuming you consider it a "delusion" to have a p(doom) of more than 5% or so, that's not uncommon among frontier lab employees who have a pretty good idea of how LLMs work.
Indeed, I don't see how knowing the details of how LLMs work (which I know btw) would change anything about how intelligent they are. I only need to know that it's a computer program that writes stories, solves math problems and seduces people.
You can make a reasonable case for adding a lot more classes in high school: statistics, nutrition, personal finance, etc. But ultimately it's a zero-sum game and to add a new class means removing an existing class. So what do we cut?
My kids' high school covers those topics. And I think that is not a new thing, I specifically remember taking the required personal finance class 35 years ago (great teacher -- crotchety old man who wrote "compound interest" on the blackboard every single day before class and repeatedly proclaimed that if we forgot everything else he taught, please try to remember how compound interest works).
>We really need a class, probably in high school, that works through how LLMs work at the high level (don't need to get too far into the deep math, but give people a taste) and then how they're trained, used, and deployed
Not applicable.
If the woman was chatting to a local LLM, there would be no way to charge her for a goddamn thought crime.
Which is what the government did here, with Anthropic's assistance.
Thinking about committing a crime isn't a crime.
As for treating the "threat" as "public".. something tells me if the court ordered to share something publicly, telling it to a chatbot would not count.
>Also: people need to understand "not your computer, not your data"
Rrrright, because it's so easy for people to know when stuff from their computer is transmitted to not their computer.
There's nothing physically stopping those companies from scanning data on your computer either.
And legally, surely the state that treats convo with a chatbot as public when it suits them would prosecute mega corps for overly thorough telemetry when it also suits them, right?
Why people here accept this as normal is beyond me.
A disclaimer on the top of page every time would have been a better approach helping Anthropic and the end user.
A bot talking to you directly as if its some one real caters to your thoughts and can take you in a certain direction without you realizing it. I have heard first hand experience from people that they feel more comfortable talking to chatgpt or claude cause it gives a feeling of being on their side and listening to them.
This is one of the reasons why AI companies are looking for explicit regulations, it can help to reduce the risk of possible liability and maybe make the legal way forward more tractable. With a regulatory framework in place much of the burden of identifying risks falls on the regulatory authority.
Then the AI companies have more confidence that they can move forward in a certain way, and issue investor guidance that is maybe closer to reality.
I think they're mostly looking for regulations because they've spent close to $2T, all to realized they have no technical moat, so they're trying to build a regulatory one.
After all the claims of "the product we are developing could cause the extinction of humanity" I'm honestly surprised Florida is the only place so far to seek an injunction preventing further development of the product.
It's really hard to take these companies seriously or at face value about anything they say.
It doesn't have to be an existential threat. As long as the lobbying is cheaper than the possible legal costs, it will look reasonable to lobby for rules.
As restrictive as possible towards their competitors, while being as permissive as possible for them. Getting control of the regulation making process means they can make sure they don't end up too constrained anywhere that would hurt their bottom line.
> people need to get it in their heads that they're not chatting with their secret BFF
I see constant ads on video platform (particularly youtube/tiktok) about llm chat apps, from friends, dating, romance and everythkng inbetween; that's personal.
People need to be reminded constantly if they use such apps that they are participating in easier mass surveillance, profiling and AI training.
Problem is that even not using those apps, the apps you currently use might have turned more hostile.
It wasn't technically feasible to scan personal chats easily, other than grepping keywords which must have had a bajillion false positives. Now you can get everything autoscanned at scale.
How do you get that through to someone who doesn't even understand that mass surveillance and profiling is a problem? Or to young people who have only lived in a society of mass surveillance?
I'm still puzzled that people's default assumption isn't that somebody is reading all of their internet communications. Ever since the Snowden revelations of 2013 I've more or less assumed that everything I type into a computer is stored in a government database somewhere. Not that I actually believe it is 100% of the time; there's a spectrum of trust, so I'm more confident that local apps on my Linux desktop are secure, somewhat confident in the end-to-end encryption of certain apps on my iPhone, but all bets are off for non-E2E encrypted data going across the internet.
For technical people, this is incredibly old news.
For non-technical people, it isn't really news, because they already forgot about it after reading it. Maybe they'll be a little more monitored in their own typing for like... a day or two.
One of the hardest lessons to internalize, and keep internalized, as someone who works on and writes software, is the vast, vast, vast majority of the Public doesn't understand even the most basic shit about software. It just does stuff. Hopefully the stuff is good. That's it, beginning, middle, and end.
"Why would you think x would y" is a poor framing. They didn't think about x or y because they don't care. The phone works, that's the beginning and end of their interest in the subject.
You're completely right. I like to think I've internalized this on the technical side, but the reason why I still find it puzzling is this has been a political issue for decades. We've had warrantless wiretapping since the invention of the telephone, there was COINTELPRO, Watergate, the Patriot Act, and then Snowden. I understand the argument that most people don't care, but if you're planning a crime it seems like you would have a strong incentive to care.
Honestly the older I get and the more I learn about criminals, of all stripes really be they petty thieves all the way up to state actors behind the most atrocious crimes against humanity we know of, they are all a bit on the dumb side. The ones that get caught are, anyway. Like so very often the perpetrator of a given crime just did one incredibly bone-headed thing that brought the cops directly to their door.
I think in part it's selection bias? Like if you're smart enough to get by honestly, you're probably also smart enough to realize getting by honestly is just a way more comfortable way to live. The only reason you'd probably cross that line is because your principals, whatever they may be, conflict with those laws, or your life circumstances are so bad that you have no choice BUT to turn to crime.
And that cuts the other way too: if you're dumb enough to think you'd NEVER get caught for a burglary, for example, you'd probably be way more down to plan and execute one, failing to consider that most thieves aren't caught when they steal the shit, they're caught when they try and sell it later.
> Honestly the older I get and the more I learn about criminals, of all stripes really be they petty thieves all the way up to state actors behind the most atrocious crimes against humanity we know of, they are all a bit on the dumb side. The ones that get caught are, anyway.
...but the last sentence if kind of important, right?
We only know about the criminals who got caught. Which probably means they did something dumb.
It's conceivable to me that there could be many—maybe even a majority—of criminals we don't know about because no one ever caught them. Maybe they committed fraud once, decided not to push their luck, and lived the rest of their life quietly. Maybe they killed someone and made it look like an accident, and no one ever suspected anything. How would we know?
Zodiac killer was never caught and it seems he already died a few years back - it was established some time ago through some teen's DNA whose grandfather was ex-cop or something. Don't quote me on that but it's on the net somewhere and was there before AI slop happened.
Basically a lot more evil goes unpunished than most realize, because carma is a thing only when it's a thing - sometimes it just isn't.
You can look up crime clearance rates. It's a little depressing.
Here's an FBI chart for 2019 [1]
Though some states like Japan and Germany have really good clearance rates for homicide. [2]
But more heartening, it seems like it doesn't really matter how many people the cops catch. Crime rates are still generally on a long downward trend (with some bumps along the way). [3]
I actually completely agree with that. I guess it depends how you want to slice that in terms of categorization.
Which is to say: If a criminal gets away with a crime and faces no punishment, not even public scorn, are they really a criminal? Or did they just work a situation, even of their making, to their advantage?
I don't particularly know where we should draw that line. But yes, my statement does deserve a big caveat there.
Those in power are often dumb criminals themselves. The fact that the system will break its back for an excuse to not prosecute them doesn’t make them skilled operators.
It isn’t that black and white. I have relatives that are highly non-technical but still care a lot about privacy and data protection in their computer use. They hate having to deal with technical things and have very little technical understanding, but still ask me whether using X software in Y way might pose privacy issues or not.
TV is a human in the box. A phone is internet in the box. It's all implicitly local, because you clutch this internet in the box right in your hand. Synchronization sends data directly from one internet in the box to another internet in the box.
People don’t want to believe the Stasi is monitoring them. They want to believe the world is a nice place just like where they grew up.
Also, unlike the bad old days when 1 in 3 was an informant, now ordinary people aren’t in “informant loop” of providing information on others, so they aren’t thinking about being informed on either.
Our devices, even Anthropic in this case, set an expectation that doesn't match reality. The software and devices we use are marketed and sold to us to be used for our private documents, photos, and conversations. I can't blame people for using them for their intended purpose. They shouldn't assume that someone is carefully watching everything they do and judging their every word and action looking for evidence of crimes. If we want people to think of the devices and services they bought and use as belonging to someone else and without any expectation for privacy we should really start with forcing companies to change how they market their products and services to reflect that reality.
> They shouldn't assume that someone is carefully watching everything they do and judging their every word and action looking for evidence of crimes.
This is silly. Nevermind the government; you should always assume some Redditor will pore through OSINT data and leaks looking for anything they can use to dox you and wreck your personal and/or professional life.
> I'm more confident that local apps on my Linux desktop are secure
Why should you be? Linux gives us software we can reasonably say we own, but not hardware. Everything you type into your local linux apps can be being monitored by your processor and whatever is sitting in the CSME/PSP subsystem which you don't have permission to access, but which is always running even when your computer is off. We need fully open, documented, auditable hardware if allowing any third party access to our devices means our private documents will make us all into criminal suspects.
Most of the hardware isn't made in the US, so either the spooks would need agreements with every offshore business, intercept every hardware shipment, or convince every vendor operating in the US, even if they are not locally owned, to do this for all hardware sold here, and convince all of these offshore vendors not to ever leak the fact that it is happening.
On top of that, the data has to pass through several layers of routers and access points we control, which makes at least the fact of exfiltration visible. Plenty of people have a very strong vested interest in preventing data exfiltration from company hardware and so the government would have to somehow get in touch with every one of them to tell them to stay quiet before they leak any hints of it happening.
It's possible for targeted shipments, but the prospect of it being done for all consumer hardware sold in the US is a bit far fetched.
Companies like Google and Microsoft pay top dollar to build special workstations to interact with production systems. These are not just special because of the software they run, but also the chips these machines are built with.
Sure, I'll take your comment at face value, not all consumer hardware has spyware, but it cannot be ruled out now, in the future, or at least now for targeted high value customers.
For these kinds of operations, open source hardware won't save you unless you build it yourself, much like open source software. At the end of the day, you're hoping that the pre built thing matches what it claims to be.
The hardware case of TAO is still targeted and didn't rely on the manufacturer's compliance and secrecy for their entire line of products, though. The scale of that to my mind is quite the qualitative difference.
That’s… really not true or at least not as dramatic as that statement makes it sound.
Those “special workstations” are mostly about lack of features they view as potential attack vectors (external or internal motherboard ports, replaceable/non-soldered components, thunderbolt, anything extensible or “repairable”) and mandate certain hardware features (hardware tpm, locked secure boot, locked UEFI). Other than that, those workstations are Lenovo, Samsung, or Dell machines running typical chips. They may pick certain CPU models or chips but that’s mostly about avoiding new or “cool” features that may not have matured enough yet. Most of them are moving to thin clients though. Where you use that “special workstation” to remote into an VM that can access production systems. And that part is partially about controlling the software running in the VM and making sure updates can be forced “offline” on the VM even if you haven’t connected to it in few weeks.
We know for a fact the government does intercept hardware shipments already on some scale. It's a lot easier when there are only a few chip makers being used for most devices. Intel and AMD alone cover the vast majority of the CPU market for desktops and laptops. It's basically the same situation for the wireless chipsets in every mobile device. No need to worry about dealing with about every cell phone manufacturer directly when you only need to hit up the extremely small number of companies every manufacturer has to get their chips from.
> Plenty of people have a very strong vested interest in preventing data exfiltration from company hardware and so the government would have to somehow get in touch with every one of them to tell them to stay quiet before they leak any hints of it happening.
Whistleblowers are extraordinarily rare. Edward Snowden worked with many many others. Any of them could have come forward at any time, but they didn't. Only one AT&T employee came forward to tell the public about Room 641A (https://en.wikipedia.org/wiki/Room_641A). The government marched into AT&T's building, took over part of their offices, rerouted their network and the whole AT&T backbone into their offices and through hardware. You can bet that more than one person noticed that. Government has no problems at all with going to a business with guns and gag orders and telling people to keep their mouths shut. Companies have no problems keeping quiet about what's happening either.
No one who isn't directly in the know is going to notice if someone's CPU (which is running it's own network stack) sends a few extra encrypted packets going to the servers of major internet companies (microsoft, cloudflare, apple, google) to either be collected at those end points or just picked up as it passes across the internet backbone.
Maybe if we had a ton more competition in things like chip makers, ISPs, operating systems, etc. it might be more difficult, but the way things are it'd be easy.
Yes, I mentioned the Tailored Access Operations NSA unit in a sibling comment. The thing is, open source hardware won't save you from any of this unless you have the manufacturing skills and equipment to build it yourself. Just like with open source software, if you aren't building it yourself, with your own hands, you can't be sure that what end result you have actually matches the code that you've read, unless there's a published checksum that you've also verified is real. Quite a bit harder to do with hardware than software though, especially since every single piece of the stack, including monitor cables, have been targeted by TAO.
It's true that you'd have a hard time knowing a chip wasn't swapped someone on its way to you. There are a few things that would help but nothing easy. Getting the hardware into retail locations so that anyone could walk in off the street and buy a system with cash. Random spot checks, chip by chip. Packing the system in colored beads/rice to prevent tampering after it's in a box.
At this point, even if I could never be sure my system was secure, just knowing that systems were purchased, tested, and verified would make me feel a lot better. It'd be nice to have confirmation that it's even possible to make and sell a system that isn't backdoored in this country.
and this assumption is wrong how exactly when viewed through the lens of real-world events?
I'm not talking about the lens of morality nor even the law, because it's obvious this shouldn't be the case but in actuality, it is - everyday more true than it was yesterday.
People not reading it is unfortunately pretty close to the truth.
You can bet that everything sent across the Internet is stored somewhere. But read? Yeah right. People don't seems to realize that there are 6B+ people online and just how big a number 6B is. If you have a staff of 30-40K people (like the FBI or NSA) and they spend 40 hours/week doing nothing but reading Internet posts and the average person spends 4 hours/week writing Internet posts (likely a huge underestimate) and the FBI agent can read 5x as fast as the person can post, then the collective manual surveillance capacity is about 2M people. That gives a 1 in 3000 chance that a random Internet user would be surveilled (though it is decidedly not random). With more realistic numbers of maybe 5000 analysts reviewing potential threats, spending 10 hours/week on it (so they have time to actually follow up on threats, attend meetings, communicate with their boss & coworkers, etc.), and the average person spending more like 25 hours/week posting on the Internet, that's a surveillance capacity of 10K people and your chances are more like 1 in 600K.
Sorry mate. The one thing llms effectively guaranteed is that short of wonks like me, no one is actually reading the source. Everything is a summary of summary of a summary.
FWIW, I feel you; I too try to leave internet a mildly more amusing place to be.
I don’t think Google is reading all of your email in this way, or your phone text messages being read in this way, or say iCloud or OneDrive files being monitored in this way.
Certainly it shows that agents like Muse are a complete nonstarter for anyone doing anything that needs to be private. Even if you never talk about it with the bot, merely having sensitive information on your hard drive means it can be sent to outside servers where, if it hits some safety filter, some probably low paid employee is going to read it. Say you’re working for a public company and have files on your computer that could be material nonpublic information, now potentially some content monitor is going to be seeing that and trading based on it. It’s completely untenable to have everything you put on your computer be subject to human review at some tech company.
All your communications which are available are being swept up by efforts like prism and stored.
Now that the technology is available to do sentiment analysis on that text, why wouldn’t government agencies do so? The information could be very useful from a few different angles, so I’d suspect they are doing it.
That's definitely not the good word to use. It is most likely made with the intent to be privacy-friendly, but they are unfortunately anything but secure (including the whole Linux userspace), and especially in the age of agents it would be the best if everyone understood it that you are a single bash/npm install/malicious PDF away from everything bad happening with your data. But rest assured, your video driver won't get updated!
> I'm still puzzled that people's default assumption isn't that somebody is reading all of their internet communications.
I'm also puzzled by the phenomenon of using ones real name on the Internet (outside a professional context). I believe this began to occur around the time facebook became popular.
I've always used my real name, since I first got on to the internet in the early 90s. It's never been a problem. Big Tech and your own government know who you really are anyway, as do the big online advertisers. Pseudonyms are no defence against that. There is a potential risk from griefers, but they have ways and means as well.
I wouldn't be surprised if a dedicated enough IT at a company is not monitoring your enrolled devices microphones. Just listening in the background for shit-talking the company.
You know, at my previous job the company had a semi-strict policies with non-enrolled devices. If we didn't enroll our devices we were locked out of some company systems, but we could still use email and teams (which is like 99% of my use-cases for my phone).
So I obviously opted to not enroll my phone, but that came to bite me in the ass one day when I needed to get a train ticket from my company email (it was a business trip) and the policy prevented non-enrolled devices from attaching or downloading attachments...
Felt like a fool grabbing my laptop to let the train attendant scan the QR code.
funnily enough, afterwards I would just forward it to my personal email. IT never bothered me about sending .pdf attachments from my company email to my personal email...
It is funny that you mention that, because at home I do/hear/watch all sorts of things, so I suspect transcripts look fun and may make management feel a little icky ( makes me wonder ). Honestly, one would think reasonable would not want to know too much:D
Ditto. Everything you do online is tracked and stored as much as possible. Treat it as public. It doesn't matter what the privacy policy says.
Your ISP tracks you, your traffic is logged at every hop and that's even before the sites themselves track you. Text you type into a box and then delete is usually tracked. How fast you scroll past that porn ad is tracked.
Right, the difficulty is partly that they can get a negative headline from any choice of behavior.
"Anthropic failed to report murderer's threats to authorities"
(or "Chatbot knew man was planning murder, yet company did nothing")
"Anthropic reported private chats to authorities"
(or "Arrested for chatbot fantasy")
To be fair to the journalists in these cases, there's also no society-wide agreed Schelling point about the correct outcome or correct rules. I have strong beliefs and intuitions about what should happen, but other people also have strong beliefs and intuitions, and many of those are probably opposite of mine. Even if my intuitions are the best and most justified, a journalist is unlikely to think "I'm just not going to mention that some people are mad at this company over this outcome, because a hypothetically better norm or principle would support the company's actions here". Hopefully the journalism can at least contextualize the lack of legal or social consensus and the difficult incentive problems, rather than jumping to "obviously companies are sociopaths staffed by supervillains".
Including any chats anywhere where someone might have a phone in their pocket, or if there's a "camera" attached to a utility pole or a nearby tree. The only real private chats might be whispered lying down in the bathtub together, with a mattress covering it like you're both hiding from a hurricane.
> they're chatting with Big Tech
They're chatting with any powerful person who wants to hear it. She thought she was chatting with Anthropic, who doesn't give a shit about her. But after being threatened (and immediately backing down because, of course, they don't give a shit about her) Anthropic has become an arm of the government. So she was chatting with the Bonita Springs, FL Sheriff's office, or anybody else. If I paid enough, Anthropic would tell me about what she was doing so I could sell her laundry detergent.
Absolutely no sympathy. Anthropic is every bit as slimy as any other big corp. And like other big corps, they must open the vault to whatever governments they intent to do biz with.
For the woman, whose entire crime was using a chatbot that runs on Anthropic's server, and wouldn't be charged or guilty of any crime if she used a similar chatbot on her own machine to write the same exact thing?
The woman whose only crime was using the cloud to store private information (diary entry, stream of consciousness, thought experiment, etc) assuming it will stay private, as is normally the case?
The woman who was jailed on a technicality that allows the government to treat the 1:1 conversation with a robot as a public threat because the fine print says that the corporate has the right to snoop on the conversation?
I hope the the woman gets some sympathy from whoever reads this news, because it's a travesty and perversion of justice that this took place and Anthropic assisted in.
> people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech
It's the other way around, big techs need to properly disclose in their platform, during interaction that they aren't in a private and safe environment
Not just bad headlines; OpenAI is actively being sued [1] for this:
OpenAI’s stated rationale was a concern for the shooter’s privacy, but its own interests
better explain its silence. Upon information and belief, OpenAI was seeking to avoid implementation of a
hard line rule to refer planning of real-world violence to authorities, perhaps due to how frequently its
product is implicated in threats to human life. Requiring such disclosures would be incompatible with the
company’s public position that ChatGPT is safe. It could also threaten the valuation underlying OpenAI’s
anticipated initial public offering. Rather than expose those risks, OpenAI accepted the consequences of
its silence. A mass murder in the only secondary school in Tumbler Ridge followed.
Accordingly, the Crown, led by Attorney General Sharma, and SD59 jointly bring this
action to hold OpenAI and Sam Altman accountable for designing a dangerous product, distributing it to
every home with internet access, ignoring the warnings of their own safety team,
refusing to notify authorities when they knew the shooter was planning gun violence, inviting the shooter back onto the
platform after deactivating the shooter’s account, and choosing corporate self-interest over the lives of
children. They seek compensation for the not just foreseeable but known harm OpenAI inflicted, the
damages that they incurred and are incurring, and injunctive relief to ensure that this tragedy does not
happen again.
That's what the suit alleges. Also that it had been automatically flagged to an internal human review team who had concluded that there was a credible, specific risk of harm to others and recommended notifying police, and that it was OpenAI leadership that rejected the recommendation.
You can see it all in GP's link. It's very readable.
So mentioning killing in passing is fine, but planning with the tool is not? How should LLM providers draw the line? I'd prefer if they didn't scrutinize any of my inputs to see if I am committing a felony or not in the first place - but that ship has sailed.
They draw the line using a team of humans who evaluate whether the threat is real and give a recommendation to management.
If you operate a gun store and someone reveals to you that they intend to use your product to shoot up a school you have a similar moral obligation, I think. This isn't a special case.
You're really stretching here for some reason I can't fathom, but yes - if you write a credible threat to shoot up your school on a crumpled up paper and someone finds it, they should report it. I'm not sure that complicating this with which kind of app it's written in is all that useful ...
Is this a point you're arguing seriously? If you find someone's plans to, for example, shoot up a school crumpled in a note on the ground that should be...ignored? Or what? I struggle to comprehend what you're actually arguing for here.
I think I pretty much agree with you that if someone finds evidence of danger to someone else's life then it is a fair expectation that they do something with that evidence. And despite that I still believe that is not a good enough reason to eliminate someone's right to privacy, and yet still that right doesn't mean that we should be able to demand privacy in all products. I think it is fairly clear that these big LLMs are not private products and we shouldn't expect them to be, but I'm still in support of offering private alternatives that people could use without having every input scrutinized.
Innocent until proven guilty. Every citizen has rights until there is reasonable suspicion, probable cause, etc that they are commiting a crime.
The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated...
But consumer perception matters. When a person engages with a product a social contract is formed. If it’s broken as the case here, I think it’s reasonable that they fight back. They have an expectation is privacy and it wasn’t upheld regardless of what the TOS says or how the tech works. This probably ends with a warning in new sessions like the McDonald’s hot coffee warning.
Note that I’m not arguing if it’s right or wrong idk on that front
There's a big, big difference between "finding by happenstance" and "automated monitoring and analysis of everything written with invisible alerting of authorities" that you are trying to obscure. It's a step towards a digital panopticon and there are more than a few objections people would have to living under a digital panopticon.
> Now, everything you say or write can be automatically screened for red flags
Considering that many people will share their deepest thought with LLM, it might start looking more and more like attempts of precog agents from Minority Report...
This isn't normal, this isn't how technology has worked.
Google doesn't report people to the police for the private (non-CSAM) contents of searches or emails.
You could definitely go through people's shit before, but Big Tech generally had serious prohibitions and a stronger presumption of privacy about this sort of thing.
CSAM was an exception where they theoretically had the evidence of the actual crime, rather than writings alluding to them. And even that has been problematic.
Frankly, the charge here is total bullshit and should be thrown out. The law is meant to prevent people sending threats to others, not keeping notes. If this charge sticks, the law should be changed.
This is something of a HN trope, "I'm sympathetic to X 'cause they tried to do their thing one way and people complained so now it's OK if they do it the opposite way, even if that's kind of F'd up too".
The problem with that is, "no, they don't have to do their thing. They have no given right to do it. If they do it, they should, they have an ethical duty to, do it right..."
I'm hoping and actually expecting that eventually there will be a simple way to locally host a small model that is good enough for simple things. For a lot of use cases, the capability of free chatgpt one year ago was more than enough.
Maybe there already is! But last I checked it was a little bit of a mess of manually installing things from multiple websites with little documentation.
I don’t think so under historical English common law. If you lift a finger to help, i think so. If you see everything with no aid whatsoever, I don’t think so
Curiously predates US law, it’s old English common law still operating as a principle
I'd expect someone's BFF to call the police if their friend is making credible talk about shooting up a school or whatever. I sometimes feel like privacy talk online can be seriously divorced from how human relationships actually work.
> Before LLMs, Big Tech had no way to scrutinize the bulk of what was going on within their services, so you could have a secret hate diary in Google Docs.
I think this is the only business model that makes sense, monetarily, for current LLM CapEx and OpEx. The value is not in what it can do for consumers or how much money it makes, but how it can shape consumers through knowledge control. Think of how much a totalitarian government would pay to keep citizens from thinking certain things.
I have told llms all kinds of stories to find out what its answers would be. I always make it sound like it is the truth to make sure the AI answers in a way that it would if somebody actually said this. I also tested internal flagging systems of the ai company I work at with the most evil things a person can ever say to find out if it would flag them.
This is exactly the typical use I make of the llm.
Adding:
- I typically ask questions in the I form, regardless for whom or why I ask for.
- Gemini chats quite often end when it starts recommending psychological council or a suicide line, to talk about my problems. It apparently detects a persistent tendency to not agree with the party line. So it makes sense I must be suicidal ;-
But sure, as llm's start to babysit us, and know our inner dialog better than anyone else, we'll soon be debugging their opinion/behavior/co-existence/authority, when it comes to reporting people to the authorities, or taking on tasks in society in general. We'll hire doctors to cure our psychological profile from our record (Total Recall).
A Minority Report like this shouldn't cause a referral to the police.
I was sure I couldn't be the only one curious to push LLMs to their limits. Though these days it's much tougher, mostly impossible to get them to react in unforeseen ways to horrendous scenarios.
Over in Europe they want to read all ofd our private messages, yet these chatbots, pretending to be our friends, will snitch on us just for our thoughts.
In Europe they should provide the citizens with the service of their messages not reaching US servers. So basically that there is only one party reading along with them, not half the world.
This kind of thing will get worse with humanoid robots because they have cameras and microphones. Will they be programmed to tell on you if you break any kind of rule in front of them because their owners are terrified of being sued?
this feels very thought-crimey to me, but I think I'd have to actually see that chats to really decide. Like is she making plans/asking for advice? is she just talking about her feelings exactly as if it's a diary?
She is not being charged with planning, just making threats in a place a person could read - the person being employees of the company.
Basically, under this interpretation, any personal note you store in the servers of a company could qualify, even if you didn't ever imagine someone would read and as such you couldn't have thought about it as a threat
This news article [1] had a quote supposedly from the chat session:
> The arrest report states that on Sept. 26 at approximately 5:10 a.m., Heller reportedly wrote, "I'm going to shoot up the sheriff's right the [expletive] now." The following day, at approximately 1:07 a.m., she was accused of posting, "This is 100% last chance I'm done. I got a new [expletive] gun today. [Expletive] you."
Lol, the expletives were clearly the most troubling part of these texts. Good thing the reporter spared us some [language]. /s
I suppose a "reasonable person" should now conclude that expletives are only part of speech with a reasonable expectation of privacy. /s
[fwiw if I were a human flagged to review these texts, I'd likely report them. My problem is with building huge systems marketed as almost a tool for thought, and then exposing these almost-thoughts for unlimited copyright case retention, for human threat review, and god knows what else.]
LLM is not a person but T&C probably states that a human moderator may view any of it. Her lawyer could probably argue a moderator filtering usage isn't the intent of the law, it was more about publishing / sending message for other humans and it was never clear to her that a human was reviewing her private diary. Shouldn't LLM disclose that at some point when people are feeding really personal stuff?
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.
She didn't threaten anything, she wrote down that she was going to do it. A "threat" is more than a mere statement, especially when written in what is described as a "diary".
> A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office.
Obviously I don't want anyone to shoot up anything, but this seems like a weak case legally speaking.
Yeah.. if you write a personal note and it's backed up by the operating system, it appears to be in violation of this law as well (since the company could theoretically read it)
People assume there won't be another human in the mix, but there is. She was judged for what she probably assumed was a private thought when it was actually not private.
Sharing them certainly can be under certain circumstances. I wouldn't be too surprised if this case gets tossed, but it's not inconceivable that prosecutors could win on the grounds that AI chats do not have a reasonable expectation of privacy and are therefor meet the requirements in the Florida statutes, or even more likely, they find some lesser charges for a plea bargain.
How could it fail to qualify for that? Another person did view it, which clearly establishes that another person could have viewed it.
The clause in the law is pointless, since if you do something that nobody else can see, you can never be punished. But many, many, many laws are written without regard to whether they make any sense.
In general, you have to intend to commit the crime you're being charged with (referred to as "mens rea"). Though, it's important to note that "intend" is extremely ill-defined in the US and it varies with the crime (eg for theft you must take the item on purpose whereas something like manslaughter requires only that you were negligent).
What this means here is, of course, equally spongy, but it is interesting as there might be an argument here that she did not intend for it to be viewed by anyone as, regardless of what the T&C say, most people do not expect their "private" chat logs between them and a machine to be seen by anyone at all.
Too broad. Unless you're transferring ink from a typewriter ribbon onto paper in a hut with no electricity, your words, or my words as I type this, are being grammar checked by something partly in the cloud. If I delete my words, are you saying I've transmitted them nevertheless?
There was no intent for a human to read the message. By your logic, if she wrote a threat in a diary and a burglar broke in and read it, it would be a crime on her part.
Chats with a company's computer aren't private the way a diary is. A better example would be she intended to write it in a word document and instead accidentally sent it in an email to a random person.
If she had intended to write it in a word document on her computer but instead accidentally wrote it into her email client and sent it to a random person, I'm not convinced she would escape getting charged then either.
It's not any different than telling an automated phone voice tree system that you plan on killing someone and then being surprised that your words were later heard by a human. She absolutely told a company's computer. She sent the message.
The law may have been intended for more direct threats to a person as a means of intimidation, but that's a separate conversation.
Yes but the law usually evaluates the application of a statute within the context of someone's mental state. This is why you are not guilty of battery when you trip and accidentally bump into someone. https://en.wikipedia.org/wiki/Mens_rea
That depends on the crime; several related crimes are only distinguished by intent. Negligence is itself a crime if it is the cause of a preventable death when the person has a reasonable obligation, such as when driving a vehicle.
I'm not really sure that this can be likened to a diary when it is called a "chat" but that's for the legal system to determine, not me sitting on my couch.
The Florida statute requires that it be transmitted in a manner that can be viewed by another person. If you have no idea that someone could view your communications with a chatbot, did you really intend to break that specific law? Technically, that threat was communicated to another person but not through her own intentions.
A sibling comment includes an important rider to the provision: "...in any manner in which it may be viewed by another person." If you wrote this in a google doc, it almost certainly would not qualify as a threat under this statute. Even though google docs, like LLM chats, have administrative override and you could look at their contents - you would not expect either to be "viewed by another person."
IMO I do not think this is a grey area and it's legal to tell a LLM you want to kill someone. It's certainly not a "threat" like you might send to another person, though it may end up being evidence of conspiracy or premeditation. I suspect we would be well served to, after a few years of experience, put together some laws governing when LLM chats must be made available to authorities.
It is very interesting that the LLM responses to these lines - the context around what she is saying - is not in the article. I suspect, as is the case in many instances where LLMs are involved in violent planning, that the LLM was urging this behavior on. Basically entrapment - you are encouraged by a robot to become more violent and vindictive and then when you do you are handed over to police.
I am not sure what point you think you are taking. I have no idea if what I said based on the text of this law applies to the text of a different law. I am having trouble following your argument (especially the "legalese notwithstanding" as my point is based on the legal text and I don't understand the purpose of a conversation around 'is this legal' outside of legal text) - could you say more?
Claude's terms of service makes it very clear that their employees will read messages[0] to determine that they don't contain the things that these messages contained[1].
> Review is needed to enforce our Usage Policy... designated members of our Trust & Safety team may access this data on a need-to-know basis as a part of their evaluation process.
I doubt it too, but they're legally binding, and extremely permissive in favor of the company far past the point of technical necessity...
...But since they exist, the company acted more responsibly than if they had simply ignored the data they chose to observe. Which is both the bare minimum given the circumstances, but also strangely absent almost all the time in the industry (including within Anthropic).
I am not a lawyer, but I have the feeling that reporting a discovered terrorist manifesto or something similar is in the spirit of the law.
Nobody is saying that Anthropic didn't have the right to read it.
I am suggesting that the criminal case against her lacks the mens rea requirement because inside of her own mind she did not expect that anyone would read it.
Doesn't "mens rea" mean "criminal intent"? I can't imagine a better example of this than intent than writing it down as a conversation. Never mind she uploaded it to a service that she explicitly agreed could be checked by employees.
I would love to sue every SaaS company for hard-to-understand terms too, as you suggest, but at what point would Anthropic actually need to tell someone about a terrorist manifesto that their customer gave them to read... Never?
Nobody is debating whether or not she is bound by the terms of the contract. I'm suggesting the prosecutor lacks mens rea which is absolutely required to prove that a criminal threat took place.
A reasonable person would not expect humans to review the millions of messages passing through the LLMs, or their own threats to ever be transmitted to a human without their authorization.
Reporting the danger is by itself a good deed. But there should be a better way of restricting firearms from the probably irresponsible lady than using inappropriate charges to punish the thoughtcrime, OR waiting for them to commit violence.
For the sake of argument what would happen if she had kept the diary locally and claude code scanned the file?
What would happen if it had scanned a file it didn’t have permission to look at and found this threat?
I honestly don’t know how I feel about this. On the one hand if you’re using claude as a diary you have no expectation of privacy and she was talking about committing a very serious crime.
> For the sake of argument what would happen if she had kept the diary locally and claude code scanned the file?
For the sake of even more argument, imagine if she was writing her thoughts with a pencil, on a good old fashioned paper diary, and she had a phone nearby and the phone took a picture of her diary, OCR'd the words, and reported it to the police?
Anthropic commits literal felonies by stealing millions of books and violating Copyright like it doesn't exist: no charge.
One unfortunate woman who happened to write the wrong thing in the wrong place is now having her life turned upside-down for perceived thought-crime.
To Anthropic, and all employees working there, your company's product and the result of your work is cruelty. You are enabling it and pushing it down everyone's throat. You can never again claim that you are the "ethical" AI company, for no such thing exists.
After people getting pilloried for social media posts from twenty years ago, I really hope (sensible) people will have the wherewithal to think twice about what they hand over to their chatbots.
Is this an invasion of their privacy (reporting to police)? Yes, but possibly warranted?
Should a social worker have contacted them rather than the police? Probably, if for no other reason than to ask if they were serious about harming someone.
Difficult questions, I'm still undecided on whether it's OK to always ignore someone's rants, even if it may be (or they think it may be) a private diary.
Actually charging them with a felony seems pretty quick to accuse. (Maybe I missed a hint about how long the investigation took before the felongy charge?)
It is my very european belief that the problem here is not that the woman was reported, but that what likely is a mental episode was made public in a way that reduces the chances of recovery.
I haven't seen anybody claiming this for consumer/free accounts anywhere.
You get privacy if you're a big corporation that needs to make sure OpenAI/Google/Anthropic can't read your trade secrets etc.
But those contractual privacy protections have been in place for a long time. It doesn't have anything to do with AI, it's been the same with Office365, Google Docs, etc.
I looked into ZDR, it's basically a vapid claim with little to no due diligence or auditing. I expect most providers to cave with only a minimal amount of legal pressure.
They still read every single message you send and train on them.
I’ve had them email me before because I was testing it as a filter for abusive messages and they detected some no-no and wrongthink in those test messages.
They run anti-abuse filters with ZDR but to my knowledge they don't have the specific prompts that triggered them. I have never received any emails from OpenRouter or otherwise.
535 comments
[ 0.26 ms ] story [ 29.9 ms ] threadThought crimes are real when you're sharing your thoughts with Claude
— via https://www.leg.state.fl.us/Statutes/index.cfm?App_mode=Disp...
The DA is serious about "in any matter."
To me, that is the more interesting legal question. Does a LLM-based safety net that sends content to a human, when the original use case would not have sent it to a human, count as "may be viewed by another person". It certainly wasn't intended to be, and that isn't the norm. At the same time, because no security is perfect, we could say that any digital record, stored in any way "may be viewed by another person."
Something for the courts to sort out, of course.
If you enter my house I will k!ll you. <- is this a threat? noone knows. The interpretation of the word "threat" is unknown. Besides, is a conditional a statement? who knows. But sure, blah blah... "in any manner in which it may..." these words are putting me to sleep.
However, people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech. Before LLMs, Big Tech had no way to scrutinize the bulk of what was going on within their services, so you could have a secret hate diary in Google Docs. Now, everything you say or write can be automatically screened for red flags on a planetary scale, and probably will be because that's what the regulators and "concerned citizens" will demand. In a couple of years, you'll be biting your tongue a lot more often in private chats.
As another commenter said, you're not chatting with a friend; you're chatting with Big Tech.
How much do you love Big Brother?
Every AI bot does this idiotic stuff, but Anthropic is the worst, and so it makes it one doubt their...let's call it "alignment"...with free-speech principles.
The main thing I fear from AI is not the made-up chance of the extermination of humanity, but rather, the increasing reach of the judgmental woke surveillance state. Remember when you could ask Google anything, and not worry about some bot criticizing you for wrongthink?
Yup. And with zero privacy protections in statute for AI chat, there is nothing to prevent an AI CEO looking to curry political favour from e.g. handing over the private correspondence of an opponent or an entire district’s residents.
It's more efficient to have one central "verifier" for everything, but the "who watches the watchers"? question basically says: Either constrain by construction, have everyone verify (which are two sides of the same coin, btw, when looking at a "global" thing), or centralize explicitly.
I really hope you mean that in an insulting way to the industry and current legal situation, not as an actual solution.
People are confused at your take because it’s based on a reality that so far you’ve only described as the better/obvious option for individuals. Companies treat corp customers and individual customers differently. This isn’t new.
My company did negotiate the data policy amongst other things for our enterprise plan, I was involved in the process. I can say with certainty that they’re not picking up the phone for _one dude_ who wants the same thing.
It seems like some people seem to think that a chatbot should guarantee total privacy like a confession to a Catholic priest or something. That is ludicrous and unrealistic. It's a commercial service subject to terms and conditions. If you don't like it then don't use it.
Custom negotiation is a big difficulty for individuals and the pricing will probably have fixed overhead that becomes pretty ridiculous when it's not spread over a hundred or thousand users.
> Or run a local model.
Much better advice. But still an obnoxious amount of effort and restriction to avoid spying.
> It seems like some people seem to think that a chatbot should guarantee total privacy like a confession to a Catholic priest or something. That is ludicrous and unrealistic. It's a commercial service subject to terms and conditions. If you don't like it then don't use it.
I want it to have the same privacy as using a local program. Does it still sound ludicrous if I put it that way? I'm not talking to a person that would have to suppress their own conversations later, and there's no need for any data from this chat to escape into the outside world. It's like wanting a phone line where the phone company won't listen in; even if doing that is standard procedure and I can walk away it's still gross.
In the case that the company running this program for me is served a warrant, I accept that saved data would be shared. But proactively screening my data for a reason that isn't to help me is bad. We should fight to make that easier to avoid.
Also you're talking out both sides of your mouth when you say it's "ludicrous and unrealistic" right after telling me I can negotiate that exact feature in an enterprise contract.
> But a civil agreement can't bind a counterparty to conceal evidence of a crime.
Are you talking about a different scenario? In a chatbot situation, the crime of "threat" requires snooping to even be possible. If the data is private there is no crime to begin with.
I had to move some games from C:\Games\RandomWarezGroup\GameName to my Steam directory, because Claude refused to reverse engineer an asset format due to piracy being incredibly immoral and illegal and blah blah.
But it sure is annoying
The issue is when Anthropic flips a switch and instead of telling me "no" it calls the report-super-dangerous-crime tool.
Somehow humanity survived the past 40 years without Microsoft Word and Excel phoning home and shopping users to the feds at random, I don't see why the standard should be any different for this new class of tooling.
On the other hand, they did put themselves into this position deliberately.
Show me any product, no matter how simple, that has no safety vs utility tradeoff.
For anyone technically inclined it should be obvious, but it isn't part of the zeitgeist or how they pitch it. People see it as being different than talking to a human, and behave as if there won't be a human in the mix.
Yes.
>Show me any product, no matter how simple, that has no safety vs utility tradeoff.
Show me a product that was not offered willingly.
Observe that Claude isn't one.
I feel like if people understood what these things actually are there'd be way less of this AI psychosis and similar stuff.
There'd also be fewer people falling for apocalyptic Rationalist delusions.
Also: people need to understand "not your computer, not your data." (Unless it's stored in the cloud but encrypted locally with keys only you possess.) Same goes for storing things unencrypted in OneDrive, Google Drive, etc. There is nothing to stop these companies from bulk scanning, data mining, or reporting people based on whatever request a government gives them. Don't count on them to resist, because they often can't, especially if the request is from a sovereign state where they do business.
Assuming you consider it a "delusion" to have a p(doom) of more than 5% or so, that's not uncommon among frontier lab employees who have a pretty good idea of how LLMs work.
There are people with education here that don't fare much better, so I don't know.
Um, this is coming from the researchers building these models. So, good luck?
Not applicable.
If the woman was chatting to a local LLM, there would be no way to charge her for a goddamn thought crime.
Which is what the government did here, with Anthropic's assistance.
Thinking about committing a crime isn't a crime.
As for treating the "threat" as "public".. something tells me if the court ordered to share something publicly, telling it to a chatbot would not count.
>Also: people need to understand "not your computer, not your data"
Rrrright, because it's so easy for people to know when stuff from their computer is transmitted to not their computer.
There's nothing physically stopping those companies from scanning data on your computer either.
And legally, surely the state that treats convo with a chatbot as public when it suits them would prosecute mega corps for overly thorough telemetry when it also suits them, right?
Why people here accept this as normal is beyond me.
A bot talking to you directly as if its some one real caters to your thoughts and can take you in a certain direction without you realizing it. I have heard first hand experience from people that they feel more comfortable talking to chatgpt or claude cause it gives a feeling of being on their side and listening to them.
Then the AI companies have more confidence that they can move forward in a certain way, and issue investor guidance that is maybe closer to reality.
They want stable rules they can follow, which will limit their liability as long as they stay within them.
They also would like those rules to be as restrictive as possible towards their competitors.
I'm quite sure that if there wasn't the existential threat of a lack of a moat, they would not be pushing for regulations at all.
It's really hard to take these companies seriously or at face value about anything they say.
I see constant ads on video platform (particularly youtube/tiktok) about llm chat apps, from friends, dating, romance and everythkng inbetween; that's personal.
People need to be reminded constantly if they use such apps that they are participating in easier mass surveillance, profiling and AI training.
It wasn't technically feasible to scan personal chats easily, other than grepping keywords which must have had a bajillion false positives. Now you can get everything autoscanned at scale.
For non-technical people, it isn't really news, because they already forgot about it after reading it. Maybe they'll be a little more monitored in their own typing for like... a day or two.
One of the hardest lessons to internalize, and keep internalized, as someone who works on and writes software, is the vast, vast, vast majority of the Public doesn't understand even the most basic shit about software. It just does stuff. Hopefully the stuff is good. That's it, beginning, middle, and end.
"Why would you think x would y" is a poor framing. They didn't think about x or y because they don't care. The phone works, that's the beginning and end of their interest in the subject.
I think in part it's selection bias? Like if you're smart enough to get by honestly, you're probably also smart enough to realize getting by honestly is just a way more comfortable way to live. The only reason you'd probably cross that line is because your principals, whatever they may be, conflict with those laws, or your life circumstances are so bad that you have no choice BUT to turn to crime.
And that cuts the other way too: if you're dumb enough to think you'd NEVER get caught for a burglary, for example, you'd probably be way more down to plan and execute one, failing to consider that most thieves aren't caught when they steal the shit, they're caught when they try and sell it later.
...but the last sentence if kind of important, right?
We only know about the criminals who got caught. Which probably means they did something dumb.
It's conceivable to me that there could be many—maybe even a majority—of criminals we don't know about because no one ever caught them. Maybe they committed fraud once, decided not to push their luck, and lived the rest of their life quietly. Maybe they killed someone and made it look like an accident, and no one ever suspected anything. How would we know?
Basically a lot more evil goes unpunished than most realize, because carma is a thing only when it's a thing - sometimes it just isn't.
Here's an FBI chart for 2019 [1]
Though some states like Japan and Germany have really good clearance rates for homicide. [2]
But more heartening, it seems like it doesn't really matter how many people the cops catch. Crime rates are still generally on a long downward trend (with some bumps along the way). [3]
[1] https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...
[2] https://en.wikipedia.org/wiki/Crime_clearance_rate
[3] https://ourworldindata.org/grapher/types-violent-crime-rate-...
Which is to say: If a criminal gets away with a crime and faces no punishment, not even public scorn, are they really a criminal? Or did they just work a situation, even of their making, to their advantage?
I don't particularly know where we should draw that line. But yes, my statement does deserve a big caveat there.
I think it's also much about self-esteem -- am I proud of myself? And criminals probably think highly of themselves when they succeed in their crimes.
Also, unlike the bad old days when 1 in 3 was an informant, now ordinary people aren’t in “informant loop” of providing information on others, so they aren’t thinking about being informed on either.
This is silly. Nevermind the government; you should always assume some Redditor will pore through OSINT data and leaks looking for anything they can use to dox you and wreck your personal and/or professional life.
Why should you be? Linux gives us software we can reasonably say we own, but not hardware. Everything you type into your local linux apps can be being monitored by your processor and whatever is sitting in the CSME/PSP subsystem which you don't have permission to access, but which is always running even when your computer is off. We need fully open, documented, auditable hardware if allowing any third party access to our devices means our private documents will make us all into criminal suspects.
On top of that, the data has to pass through several layers of routers and access points we control, which makes at least the fact of exfiltration visible. Plenty of people have a very strong vested interest in preventing data exfiltration from company hardware and so the government would have to somehow get in touch with every one of them to tell them to stay quiet before they leak any hints of it happening.
It's possible for targeted shipments, but the prospect of it being done for all consumer hardware sold in the US is a bit far fetched.
Sure, I'll take your comment at face value, not all consumer hardware has spyware, but it cannot be ruled out now, in the future, or at least now for targeted high value customers.
Well, sure, there was NSA's Tailored Access Operations unit which interfered with all kinds of hardware, even computer monitor cables:
https://www.nbcnews.com/tech/tech-news/report-nsa-intercepts...
For these kinds of operations, open source hardware won't save you unless you build it yourself, much like open source software. At the end of the day, you're hoping that the pre built thing matches what it claims to be.
The hardware case of TAO is still targeted and didn't rely on the manufacturer's compliance and secrecy for their entire line of products, though. The scale of that to my mind is quite the qualitative difference.
Those “special workstations” are mostly about lack of features they view as potential attack vectors (external or internal motherboard ports, replaceable/non-soldered components, thunderbolt, anything extensible or “repairable”) and mandate certain hardware features (hardware tpm, locked secure boot, locked UEFI). Other than that, those workstations are Lenovo, Samsung, or Dell machines running typical chips. They may pick certain CPU models or chips but that’s mostly about avoiding new or “cool” features that may not have matured enough yet. Most of them are moving to thin clients though. Where you use that “special workstation” to remote into an VM that can access production systems. And that part is partially about controlling the software running in the VM and making sure updates can be forced “offline” on the VM even if you haven’t connected to it in few weeks.
We know for a fact the government does intercept hardware shipments already on some scale. It's a lot easier when there are only a few chip makers being used for most devices. Intel and AMD alone cover the vast majority of the CPU market for desktops and laptops. It's basically the same situation for the wireless chipsets in every mobile device. No need to worry about dealing with about every cell phone manufacturer directly when you only need to hit up the extremely small number of companies every manufacturer has to get their chips from.
> Plenty of people have a very strong vested interest in preventing data exfiltration from company hardware and so the government would have to somehow get in touch with every one of them to tell them to stay quiet before they leak any hints of it happening.
Whistleblowers are extraordinarily rare. Edward Snowden worked with many many others. Any of them could have come forward at any time, but they didn't. Only one AT&T employee came forward to tell the public about Room 641A (https://en.wikipedia.org/wiki/Room_641A). The government marched into AT&T's building, took over part of their offices, rerouted their network and the whole AT&T backbone into their offices and through hardware. You can bet that more than one person noticed that. Government has no problems at all with going to a business with guns and gag orders and telling people to keep their mouths shut. Companies have no problems keeping quiet about what's happening either.
No one who isn't directly in the know is going to notice if someone's CPU (which is running it's own network stack) sends a few extra encrypted packets going to the servers of major internet companies (microsoft, cloudflare, apple, google) to either be collected at those end points or just picked up as it passes across the internet backbone.
Maybe if we had a ton more competition in things like chip makers, ISPs, operating systems, etc. it might be more difficult, but the way things are it'd be easy.
The US government's own digital security people disable Intel's management engine on sensitive servers. What does that tell you?
NOBUS.
You can have open and auditable hardware, but how can you be sure that the hardware you buy is exactly the same hardware as what's in the Github repo?
At this point, even if I could never be sure my system was secure, just knowing that systems were purchased, tested, and verified would make me feel a lot better. It'd be nice to have confirmation that it's even possible to make and sell a system that isn't backdoored in this country.
Wow, so we're assuming police state plus it's your fault if you didn't know...
I'm not talking about the lens of morality nor even the law, because it's obvious this shouldn't be the case but in actuality, it is - everyday more true than it was yesterday.
Of course, if I don't want anyone reading it, I sure as heck don't put it online.
You can bet that everything sent across the Internet is stored somewhere. But read? Yeah right. People don't seems to realize that there are 6B+ people online and just how big a number 6B is. If you have a staff of 30-40K people (like the FBI or NSA) and they spend 40 hours/week doing nothing but reading Internet posts and the average person spends 4 hours/week writing Internet posts (likely a huge underestimate) and the FBI agent can read 5x as fast as the person can post, then the collective manual surveillance capacity is about 2M people. That gives a 1 in 3000 chance that a random Internet user would be surveilled (though it is decidedly not random). With more realistic numbers of maybe 5000 analysts reviewing potential threats, spending 10 hours/week on it (so they have time to actually follow up on threats, attend meetings, communicate with their boss & coworkers, etc.), and the average person spending more like 25 hours/week posting on the Internet, that's a surveillance capacity of 10K people and your chances are more like 1 in 600K.
FWIW, I feel you; I too try to leave internet a mildly more amusing place to be.
Certainly it shows that agents like Muse are a complete nonstarter for anyone doing anything that needs to be private. Even if you never talk about it with the bot, merely having sensitive information on your hard drive means it can be sent to outside servers where, if it hits some safety filter, some probably low paid employee is going to read it. Say you’re working for a public company and have files on your computer that could be material nonpublic information, now potentially some content monitor is going to be seeing that and trading based on it. It’s completely untenable to have everything you put on your computer be subject to human review at some tech company.
Now that the technology is available to do sentiment analysis on that text, why wouldn’t government agencies do so? The information could be very useful from a few different angles, so I’d suspect they are doing it.
Anytime the strong encryption debate comes up it's actually about maintaining the effectiveness of unencrypted surveillance in the AI era.
That's definitely not the good word to use. It is most likely made with the intent to be privacy-friendly, but they are unfortunately anything but secure (including the whole Linux userspace), and especially in the age of agents it would be the best if everyone understood it that you are a single bash/npm install/malicious PDF away from everything bad happening with your data. But rest assured, your video driver won't get updated!
https://xkcd.com/1200/
I'm also puzzled by the phenomenon of using ones real name on the Internet (outside a professional context). I believe this began to occur around the time facebook became popular.
no outside device comes in - no inside device goes out.
So I obviously opted to not enroll my phone, but that came to bite me in the ass one day when I needed to get a train ticket from my company email (it was a business trip) and the policy prevented non-enrolled devices from attaching or downloading attachments...
Felt like a fool grabbing my laptop to let the train attendant scan the QR code.
Your ISP tracks you, your traffic is logged at every hop and that's even before the sites themselves track you. Text you type into a box and then delete is usually tracked. How fast you scroll past that porn ad is tracked.
"Anthropic failed to report murderer's threats to authorities"
(or "Chatbot knew man was planning murder, yet company did nothing")
"Anthropic reported private chats to authorities"
(or "Arrested for chatbot fantasy")
To be fair to the journalists in these cases, there's also no society-wide agreed Schelling point about the correct outcome or correct rules. I have strong beliefs and intuitions about what should happen, but other people also have strong beliefs and intuitions, and many of those are probably opposite of mine. Even if my intuitions are the best and most justified, a journalist is unlikely to think "I'm just not going to mention that some people are mad at this company over this outcome, because a hypothetically better norm or principle would support the company's actions here". Hopefully the journalism can at least contextualize the lack of legal or social consensus and the difficult incentive problems, rather than jumping to "obviously companies are sociopaths staffed by supervillains".
Including any chats anywhere where someone might have a phone in their pocket, or if there's a "camera" attached to a utility pole or a nearby tree. The only real private chats might be whispered lying down in the bathtub together, with a mattress covering it like you're both hiding from a hurricane.
> they're chatting with Big Tech
They're chatting with any powerful person who wants to hear it. She thought she was chatting with Anthropic, who doesn't give a shit about her. But after being threatened (and immediately backing down because, of course, they don't give a shit about her) Anthropic has become an arm of the government. So she was chatting with the Bonita Springs, FL Sheriff's office, or anybody else. If I paid enough, Anthropic would tell me about what she was doing so I could sell her laundry detergent.
For Anthropic? They don't need it.
For the woman, whose entire crime was using a chatbot that runs on Anthropic's server, and wouldn't be charged or guilty of any crime if she used a similar chatbot on her own machine to write the same exact thing?
The woman whose only crime was using the cloud to store private information (diary entry, stream of consciousness, thought experiment, etc) assuming it will stay private, as is normally the case?
The woman who was jailed on a technicality that allows the government to treat the 1:1 conversation with a robot as a public threat because the fine print says that the corporate has the right to snoop on the conversation?
I hope the the woman gets some sympathy from whoever reads this news, because it's a travesty and perversion of justice that this took place and Anthropic assisted in.
This crap makes KGB look good.
It's the other way around, big techs need to properly disclose in their platform, during interaction that they aren't in a private and safe environment
That could be a meaningful difference
You can see it all in GP's link. It's very readable.
I figured it just didn't get flagged properly.
They draw the line using a team of humans who evaluate whether the threat is real and give a recommendation to management.
If you operate a gun store and someone reveals to you that they intend to use your product to shoot up a school you have a similar moral obligation, I think. This isn't a special case.
There are tons of hosted app services where you can write private expressions. Why are only LLMs subject to monitoring?
I don't think that most (any?) of these other cloud services monitor for this stuff, or that government thinks they have an obligation to do so.
The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated...
The false positive rate of the test that you suggest is frankly atrocious.
Note that I’m not arguing if it’s right or wrong idk on that front
Considering that many people will share their deepest thought with LLM, it might start looking more and more like attempts of precog agents from Minority Report...
Google doesn't report people to the police for the private (non-CSAM) contents of searches or emails.
You could definitely go through people's shit before, but Big Tech generally had serious prohibitions and a stronger presumption of privacy about this sort of thing.
CSAM was an exception where they theoretically had the evidence of the actual crime, rather than writings alluding to them. And even that has been problematic.
Frankly, the charge here is total bullshit and should be thrown out. The law is meant to prevent people sending threats to others, not keeping notes. If this charge sticks, the law should be changed.
The problem with that is, "no, they don't have to do their thing. They have no given right to do it. If they do it, they should, they have an ethical duty to, do it right..."
Maybe there already is! But last I checked it was a little bit of a mess of manually installing things from multiple websites with little documentation.
If a person told their real BFF they were planning a shooting and went through with it, couldn’t the BFF be charged with accessory?
Curiously predates US law, it’s old English common law still operating as a principle
IANAL blahblah
They did and they do
https://www.bbc.co.uk/news/technology-44699263
I never considered online services safe. What I object to is active scanning of content on devices. This should not be allowed.
Then again, I wish this worked in a way that would give users more privacy and agency, instead of less.
I guess its a miracle I am not in jail yet.
Adding: - I typically ask questions in the I form, regardless for whom or why I ask for. - Gemini chats quite often end when it starts recommending psychological council or a suicide line, to talk about my problems. It apparently detects a persistent tendency to not agree with the party line. So it makes sense I must be suicidal ;-
But sure, as llm's start to babysit us, and know our inner dialog better than anyone else, we'll soon be debugging their opinion/behavior/co-existence/authority, when it comes to reporting people to the authorities, or taking on tasks in society in general. We'll hire doctors to cure our psychological profile from our record (Total Recall).
A Minority Report like this shouldn't cause a referral to the police.
Over in Europe they want to read all ofd our private messages, yet these chatbots, pretending to be our friends, will snitch on us just for our thoughts.
It's getting pretty orwellian out there.
Two teens riding in a Waymo were arrested because the AI detected them talking about having a gun.
Basically, under this interpretation, any personal note you store in the servers of a company could qualify, even if you didn't ever imagine someone would read and as such you couldn't have thought about it as a threat
> The arrest report states that on Sept. 26 at approximately 5:10 a.m., Heller reportedly wrote, "I'm going to shoot up the sheriff's right the [expletive] now." The following day, at approximately 1:07 a.m., she was accused of posting, "This is 100% last chance I'm done. I got a new [expletive] gun today. [Expletive] you."
[1] https://www.gulfcoastnewsnow.com/article/florida-woman-arres...
I suppose a "reasonable person" should now conclude that expletives are only part of speech with a reasonable expectation of privacy. /s
[fwiw if I were a human flagged to review these texts, I'd likely report them. My problem is with building huge systems marketed as almost a tool for thought, and then exposing these almost-thoughts for unlimited copyright case retention, for human threat review, and god knows what else.]
How does a LLM prompt satisfy this? I guess it'll be an easy win for her.
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.
She didn't threaten anything, she wrote down that she was going to do it. A "threat" is more than a mere statement, especially when written in what is described as a "diary".
> A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office.
Obviously I don't want anyone to shoot up anything, but this seems like a weak case legally speaking.
> The communication must be made in a manner in which another person may view it.
Even 'transmitted' is too broad if you also consider iCloud backup to be a means.
The clause in the law is pointless, since if you do something that nobody else can see, you can never be punished. But many, many, many laws are written without regard to whether they make any sense.
What this means here is, of course, equally spongy, but it is interesting as there might be an argument here that she did not intend for it to be viewed by anyone as, regardless of what the T&C say, most people do not expect their "private" chat logs between them and a machine to be seen by anyone at all.
It's not any different than telling an automated phone voice tree system that you plan on killing someone and then being surprised that your words were later heard by a human. She absolutely told a company's computer. She sent the message.
The law may have been intended for more direct threats to a person as a means of intimidation, but that's a separate conversation.
I'm not really sure that this can be likened to a diary when it is called a "chat" but that's for the legal system to determine, not me sitting on my couch.
She wouldn't have intended her housemate to open it, so it wasn't a threat.
And her housemate didn't intend to open it, so it wasn't Obstruction of Correspondence. https://www.law.cornell.edu/uscode/text/18/1702
IMO I do not think this is a grey area and it's legal to tell a LLM you want to kill someone. It's certainly not a "threat" like you might send to another person, though it may end up being evidence of conspiracy or premeditation. I suspect we would be well served to, after a few years of experience, put together some laws governing when LLM chats must be made available to authorities.
It is very interesting that the LLM responses to these lines - the context around what she is saying - is not in the article. I suspect, as is the case in many instances where LLMs are involved in violent planning, that the LLM was urging this behavior on. Basically entrapment - you are encouraged by a robot to become more violent and vindictive and then when you do you are handed over to police.
> Review is needed to enforce our Usage Policy... designated members of our Trust & Safety team may access this data on a need-to-know basis as a part of their evaluation process.
[0]: https://privacy.claude.com/en/articles/10458704-how-does-ant...
[1]: https://www.anthropic.com/legal/aup
The critical part of a "threat" is that the perpetrator takes some intentional method to deliver it.
...But since they exist, the company acted more responsibly than if they had simply ignored the data they chose to observe. Which is both the bare minimum given the circumstances, but also strangely absent almost all the time in the industry (including within Anthropic).
I am not a lawyer, but I have the feeling that reporting a discovered terrorist manifesto or something similar is in the spirit of the law.
I am suggesting that the criminal case against her lacks the mens rea requirement because inside of her own mind she did not expect that anyone would read it.
I would love to sue every SaaS company for hard-to-understand terms too, as you suggest, but at what point would Anthropic actually need to tell someone about a terrorist manifesto that their customer gave them to read... Never?
https://www.criminaldefenselawyer.com/crime-penalties/federa...
Whether she intended to act upon the idea might be relevant to some other crime but it doesn't really have anything to do with whether it is a threat.
Reporting the danger is by itself a good deed. But there should be a better way of restricting firearms from the probably irresponsible lady than using inappropriate charges to punish the thoughtcrime, OR waiting for them to commit violence.
A diary constitutes making a threat?
Oh boy the roleplaying part of LLM world is in for a bad time
What would happen if it had scanned a file it didn’t have permission to look at and found this threat?
I honestly don’t know how I feel about this. On the one hand if you’re using claude as a diary you have no expectation of privacy and she was talking about committing a very serious crime.
This still makes me feel queasy though.
For the sake of even more argument, imagine if she was writing her thoughts with a pencil, on a good old fashioned paper diary, and she had a phone nearby and the phone took a picture of her diary, OCR'd the words, and reported it to the police?
One unfortunate woman who happened to write the wrong thing in the wrong place is now having her life turned upside-down for perceived thought-crime.
To Anthropic, and all employees working there, your company's product and the result of your work is cruelty. You are enabling it and pushing it down everyone's throat. You can never again claim that you are the "ethical" AI company, for no such thing exists.
Didn't they pay a huge settlement for the books thing as well? My friend got a few hundred bucks for a book that barely even sold any copies
Is this an invasion of their privacy (reporting to police)? Yes, but possibly warranted?
Should a social worker have contacted them rather than the police? Probably, if for no other reason than to ask if they were serious about harming someone.
Difficult questions, I'm still undecided on whether it's OK to always ignore someone's rants, even if it may be (or they think it may be) a private diary.
Actually charging them with a felony seems pretty quick to accuse. (Maybe I missed a hint about how long the investigation took before the felongy charge?)
At best I think your comment is attempting to make a determination without all the facts.
At worst your comment reads like thinly veiled sexism against women.
I don’t know what your intentions are but you might want to consider reevaluating your conclusions here.
I am going to flag you though.
https://www.politico.com/news/2023/08/30/desantis-warns-hurr...
Every single lie of yours is exposed in the past few months.
You get privacy if you're a big corporation that needs to make sure OpenAI/Google/Anthropic can't read your trade secrets etc.
But those contractual privacy protections have been in place for a long time. It doesn't have anything to do with AI, it's been the same with Office365, Google Docs, etc.
LocalLLM enthusiasts exist for a good reason.
I’ve had them email me before because I was testing it as a filter for abusive messages and they detected some no-no and wrongthink in those test messages.