Or maybe it should be illegal to have a court order that the privacy of millions of people should be infringed? I’m with OpenAI on this one, regardless of their less than pure reasons. You don’t get to wiretap all of the US population, and that’s essentially what they are doing here.
They are preserving evidence in a lawsuit. If you are concerned, you can try petitioning the court to keep your data private. I don't know how that would go.
You can use that same argument for wiretapping the US, because surely someone did something wrong. So we should just collect evidence on everyone on the off chance we need it.
> The organisation might need you to prove your identity. However, they should only ask you for just enough information to be sure you are the right person. If they do this, then the one-month time period to respond to your request begins from when they receive this additional information.
In my domain, our set of services only authorizes Customer Centre system to do so. I guess I'd need to ask them for details, but I always assumed they have checks in place
Probably most just ignore backups. But there were some good proposals where you encrypt every users data with their own key. So a full delete is just deleting the users encryption key, rendering all data everywhere including backups inaccessible.
A set of encryption keys is a lot smaller than the set of all user data, so it's much more viable to have both more redundant hot storage and more frequently rotated cold storage of just the keys.
Deletion via encryption only works if every user’s data is completely separate from every other user’s data in the storage layer. This is rarely the case in databases, indexes, etc. It also is often infeasible if the number of users is very large (key schedule state alone will blow up your CPU cache).
Databases with data from multiple users largely can’t work this way unless you are comfortable with a several order of magnitude loss of performance. It has been built many times but performance is so poor that it is deemed unusable.
Some of these issues could perhaps be addressed by having fixed retention of PII in the online systems, and encryption at rest in the offline systems. If a user wants to access data of theirs which has gone offline, they take the decryption hit. Of course it helps to be critical about how much data should be retained in the first place.
It is true that protecting the user's privacy costs more than not protecting it, but some organizations feel a moral obligation or have a legal duty to do so. And some users value their own privacy enough that they are willing to deal with the decreased convenience.
As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it. I welcome government regulations that encourage more research and development in this area, since from my perspective that aligns actually-interesting technical work with the public good.
> As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it.
Unfortunately, this is a deeply hard problem in theory. It is not as though it has not been thoroughly studied in computer science. When GDPR first came out I was actually doing core research on “delete-optimized” databases. It is a problem in other domains. Regulations don’t have the power to dictate mathematics.
I know of several examples in multiple countries where data deletion laws are flatly ignored by the government because it is literally impossible to comply even though they want to. Often this data supports a critical public good, so simply not collecting it would have adverse consequences to their citizens.
tl;dr: delete-optimized architectures are so profoundly pathological to query performance, and a lesser extent insert performance, that no one can use them for most practical applications. It is fundamental to the computer science of the problem. Denial of this reality leads to issues like the above where non-compliance is required because the law didn’t concern itself with the physics of computation.
If the database is too slow to load the data then it doesn’t matter how fast your deterministic hard deletion is because there is no data to delete in the system.
Any improvements in the situation are solving minor problems in narrow cases. The core theory problems are what they are. No amount of wishful thinking will change this situation.
It would be interesting to hear more about your experience with systems where deletion has been deemed "literally impossible".
Every database I have come across in my career has a delete function. Often it is slow. In many places I worked, deleting or expiring data cost almost as much as or sometimes more than inserting it... but we still expired the data because that's a fundamental requirement of the system. So everything costs 2x, so what? The interesting thing is how to make it cost less than 2x.
Instantaneous deletes might be impossible, but I really doubt that it’s physically impossible to eventually delete user data. If you soft delete first to hide user data, and then maybe it takes hours, weeks, months to eventually purge from all systems, that’s fine. Regulators aren’t expecting you to edit old backups, only that they eventually get cleared in reasonable time.
Seems that companies are capable of moving mountains when the task is tracking the user and bypassing privacy protections. But when the task is deleting the users data it’s “literally impossible”
The entire mess isn't with data in databases, but on laptops for offline analysis, in log files, backups, etc.
It's easy enough to have a SQL query to delete a users' data from the production database for real.
It's all the other places the data goes that's a mess, and a robust system of deletion via encryption could work fine in most of those places, at least in the abstract with the proper tooling.
You can use row based encryption and store the encrypted encryption key alongside each row. You use a master key to decrypt the row encryption key and then decrypt the row each time you need to access it. This is the standard way of implementing it.
You can instead switch to a password-based key derivation function for the row encryption key if you want the row to be encrypted by a user provided password
This has been tried many times. The performance is so poor as to be unusable for most applications. The technical reasons are well-understood.
The issue is that, at a minimum, you have added 32 bytes to a row just for the key. That is extremely expensive and in many cases will be a large percentage of the entire row; many years ago PostgreSQL went to heroic efforts to reduce 2 bytes per row for performance reasons. It also limits you to row storage, which means query performance will be poor.
That aside, you overlooked the fact that you'll have to compute a key schedule for each row. None of the setup costs of the encryption can be amortized, which makes processing a row extremely expensive computationally.
There is no obvious solution that actually works. This has been studied and implemented extensively. The reason no one does it isn't because no one has thought of it before.
You’re not wrong about the downsides. However you’re wrong about the costs being prohibitive on general. I’ve personally worked on quite a few applications that do this and the additional cost has never been an issue.
Obviously context matters and there are some applications where the cost does not outweigh the benefit
Depends on the processes in place at the company. Presumably if a backup is restored, some kind of replay has to happen after that, otherwise all the other users are going to lose data that arrived in the interim. A catastrophic failure where both two weeks of user data and all the related events get irretrievably blackholed could still happen, sure, but any company where that is a regular occurrence likely has much bigger problems than complying with GDPR.
The point is that none of these problems are insurmountable - they are all processes and practices that have been in place since long before GDPR and long before I started in this industry 25+ years ago. Even if deletion is only eventually consistent, even if a few pieces of data slip through the cracks, it is not hard to have policies in place that at least provide a best effort at upholding users' privacy and complying with the regulations.
Organizations who choose not to bother, claiming that it's all too difficult, or that because deletion cannot be done 100% perfectly it should not even be attempted at all, are making weak excuses. The cynical take would be that they are just covering for the fact that they really do not respect their users' privacy and simply do not want to give up even the slightest chance of extracting value from that data they illegally and immorally choose to retain.
"When data subjects exercise one of their rights, the controller must respond within one month. If the request is too complex and more time is needed to answer, then your organisation may extend the time limit by two further months, provided that the data subject is informed within one month after receiving the request."
Backup retention policy 60 days, respond within a week or two telling someone that you have purged their data from the main database but that these backups exist and cannot be changed, but that they will be automatically deleted in 60 days.
The only real difficulty is if those backups are actually restored, then the user deletion needs to be replayed, which is something that would be easy to forget.
With how modern systems, languages, databases and file systems are designed, deletion often means "mark this as deleted" or "erase the location of this data". This is true on all possible levels of the stack, from hardware to high-level application frameworks.
Changing this would slow computers down massively. Just to give a few examples, backups would be prohibited, so would be garbage collection and all existing SSD drives. File systems would have to wipe data on unlink(), which would increase drive wear and turn operations which everybody assumed were O(1) for years into O(n), and existing software isn't prepared for that. Same with zeroing out memory pages, OSes would have to be redesigned to do it all at once when a process terminates, and we just don't know what the performance impact of that would be.
You just do it the way fast storage wipes do it. Encrypt everything, and to delete you delete the decryption key. If a user wants to clear their personal data, you delete their decryption key and all of their data is burned without having to physically modify it.
That only works if you have a single key at the block level, like an encryption key per file. It essentially doesn’t work for data that is finely mixed with different keys such as in a database. Encryption works on byte blocks, 16-bytes in the case of AES. Modern data representations interleave data at the bit level for performance and efficiency reasons. How do you encrypt a block with several users data in it? Separating these out into individual blocks is extremely expensive in several dimensions.
There have been several attempts to build e.g. databases that worked this way. The performance and scalability was so poor compared to normal databases that they were essentially unusable.
Consequently all your "deleted chats" might one day become public if someone manages to dump some tables off OpenAI's databases.
Maybe not today on its heyday, but who knows what happens in 20 years once OpenAI becomes Yahoo of AI, or loses much of its value, gets scrapped for parts and bought by less sophisticated owners.
It's better to regard that data as already public.
The concept of “deleted” is not black and white, it is a continuum (though I agree that this is a very soft delete). As a technical matter, it is surprisingly difficult and expensive to unrecoverably delete something with high assurance. Most deletes in real systems are much softer than people assume because it dramatically improves performance, scalability, and cost.
There have been many attempts to build e.g. databases that support deterministic hard deletes. Unfortunately, that feature is sufficiently ruinous to efficient software architecture that performance is extremely poor such that no one uses them.
The original submission was a link to a post on Mastodon. The post itself was too long to fit in the title, so I trimmed it, and put the full post here in a comment.
But with the URL in the submission being changed, this doesn't really make sense any more! In the future I'll make sure I include in the comment the original link with the original text so it makes sense even if (when?) the submission URL gets changed.
That’ll be the day. But even if it does happen, major AI players have the resources to move to a more ‘flexible’ country, if there isn’t a loophole that involves them closing their eyes really really tight while outsourced webscrapers collect totally legit and not illegally obtained data
You're being generous to even grant an "even if it does" proposition. Considering the people musing about "reform" of copyright at the moment -- Jack Dorsey's little flip "delete all IP law" comes to mind -- the clear direction we're headed is toward artistic and cultural serfdom.
In all fairness, the essence of it doesn't have to do anything with copyright. "Pro-copyright" is old news. Everyone knows these companies shit on copyright, but so do users, and the only reason why users sometimes support the "evil pro-copyright shills" narrative is because we are bitter that Facebook and OpenAI can get away with that, while common peasants are constantly under the risk of being fucked up for life. The news is big news only because of "anti-ChatGPT" part, and everyone is a user of ChatGPT now (even though 50% of them hate it). Moreover, it's only big news because the users are directly concerned: if OpenAI would have to pay big fine and continue business as usual, the comments would largely be schadenfreude.
And the fact that the litigation was over copyright is an insignificant detail. It could have been anything. Literally anything, like a murder investigation, for example. It only helps OpenAI here, because it's easy to say "nobody cares about copyright", and "nobody cares about murder" sounds less defendable.
Anyway, the issue here is not copyright, nor "AI", it's the venerated legal system, which very much by design allows for a single woman to decide on a whim, that a company with millions of users must start collecting user data, while users very much don't want that, and the company claims it doesn't want that too (mostly, because it knows how much users don't want that: otherwise it'd be happy to). Everything else is just accidental details, it really has nothing to do neither with copyright, nor with "AI".
It's surprising to me, because you'd think a site like Ars would attract a generally more knowledgable audience, but reading through their comment section feels like looking at Twitter or YouTube comments -- various incendiary and unsubstantial hot takes.
I'm "pro-copyright" in that I want the corporations that setup this structure to suffer under it the way we did for 25+ years. They can't just ignore the rules they spent millions lobbying for when they feel it's convinient.
On the other end: while copyright has been perverted over the centuries, the goal is still overall to protect small inventors. They have no leverage otherwise and this gives them some ability to fight if they aren't properly compensated. I definitely do not want it abolished outright. Just reviewed and reworked for modern times.
It's not like Sam Altman has been particularly hostile to the present administration. He's probably already handing them over behind closed doors and doesn't want to take the PR hit.
Why don't you want me to install a camera in your house watching you sleep and get dressed? Just because it would be embarrassing to be watched all the time?
There are a whole host of things people could (and probably are) using LLMs for. Think about what the tool fundamentally is. An enabler. A helper. A teaching assistant. A more ergonomic search tool.
Now think about all of the dumb, violent, ideological, illegal things that folks do.
No one wants their bowel/fertility/pregnancy/prostate/breast concerns made public or traded between companies. If for no other reason than health insurance fuckery.
To add a new point, what about the scope of trust here?
For the lifetime (which is?!) of the data do you trust:
- all the executives who make decisions on the data? IE direct policies but also indirect decisions like what budget exists for securing it.
- all the executive of companies who might buy the data or buy the company that has the data.
- all the technical admins of the originating company AND all the companies that might later have the data.
- all the governments that have legal leverage over the above companies
"requires anonymity" is already invalid assumption / invalid question territory.
You do not have to justify, and no one has the right to even ask for a justification, let alone judge the answer, of why ones thoughts should be allowed to be private by default.
What about "illegal" things? You could live in a country that punishes gay people and want to ask questions about being gay. Maybe you live under a tyrannical government and want to ask questions about the government or politics without raising any red flags on your name or IP. Maybe you overstayed your visa in the US and are worried about ICE putting out honeypot sites to find your location when you are trying to research options on how to get a work visa or how to escape the US without getting arrested at the border and sent to prison.
Is it being overly paranoid? Maybe in most cases yes. But we know the capability and desire exists for certain governments and agencies to locate and punish people and we know it does happen, even if it isn't super common.
That’s the wrong way to think about it. Everything should be anonymous and private by default against up to nation state actors, so that dissidents and journalists can hide easily among the masses without giving up their anonymity
Businesses submitting data that may be of interest to their competitors is one concern. Assuming that a business trusts how OpenAI handles the data, more parties will have access to it. That increases the probability of it being leaked. (The trust doesn't have to be intrinsic. There will be contracts in place, so those businesses can be held accountable via the courts.) Something that they should not be doing, in my opinion, but probably are: submitting confidential data of other parties. That adds another layer that may face legal repercussions if data is leaked.
On the level of individuals, "embarassment" may be more than emotional discomfort. It may lead to discrimination. Given some of the things happening in the US at present, it may lead to prosecution (or worse). It is possible that none of the repercussions will be legal, but they will happen nevertheless.
Like, theres so many good reasons to want privacy by default it isnt funny.
But just consider that the dumbest country on the planet, elected its greatest moron ever devised, and he is already screening people crossing the border based on social media history. Its not a terribly far leap from screening social media history, to screening literally any other data the government can get its hands on.
Ask any Unix filesystem developer, and they'll tell you that unlink(2) on a file does not erase any of its data, but simply enables the reuse of those blocks on disk.
Whenever I "delete" a social media account, or "Trash" anything on a cloud storage provider, I repeat the mantra, "revoking access for myself!" which may be sung to the tune of "Evergreen".
In the first case, there is nothing preventing the development of software to overwrite data before unlink(2) is called.
In the second case, you can choose to trust or distrust the cloud storage provider. Trust being backed by contractual obligations and the right to sue if those obligations are not met. Of course, most EULAs for consumer products are toothless is this respect. On the other hand, that doesn't prevent companies from offering contracts which have some teeth (which they may do for business clients).
> there is nothing preventing the development of software to overwrite data before unlink(2) is called.
It’s not that simple: this command already exists, it’s called `shred`, and as the manual[1] notes:
The shred command relies on a crucial assumption: that the file system and hardware overwrite data in place. Although this is common and is the traditional way to do things, many modern file system designs do not satisfy this assumption.
A reasonable cloud storage provider stores your data encrypted on disk. Certain standards like HIPPA mandates this.
Deletion of data is achieved by permanently discarding the encryption key which is stored and managed elsewhere where secure erasure can be guaranteed.
If implemented honestly, this procedure WORKS and cloud storage is secure. Yes the emphasis is on the "implemented honestly" part but do not generalize cloud storage as inherently insecure.
The docs contain a sentence on them retaining any chats that they legally have to retain. This is always the risk when doing business with law abiding companies which store any data on you.
Agree. But it's worth noting that they already have a bit of a hedge in the description for private mode:
> This chat won't appear in history, use or update ChatGPT's memory, or be used to train our models. For safety purposes, we may keep a copy of this chat for up to 30 days.
The "may keep a copy" is doing a lot of work in that sentence.
"for up to 30 days" though. If they are being kept in perpetuity always they should update the copy to say "This chat won't appear in your history, we retain a copy indefinitely"
While I also disagree with the court order and OpenAI's implementation (along with pretty much everything else the company does), the conspiratorial thinking in the comments here is unfounded. The order is overly broad but in the context of the case, it's not totally unwaranted. This is not some conspiracy to collect data on people. I'm confident the records will be handled appropriately once the case concludes (and if they're not, we can be upset then, not now). Let's please reserve our outrage for the plethora of very real issues going on right now.
The Thomson Reuters case [1] is the most relevant in the court's finding that the copying of copyrighted material from Westlaw by Ross Intelligence was direct copyright infringement and was not fair use.
The purpose of training in many of the AI Labs being sued mostly matches the conditions that Ross Intelligence was found to have violated, and the question of copying is almost guaranteed if they trained on it.
> and if they're not, we can be upset then, not now
Like we could be upset when that credit checking company dumped all those social security numbers on the net and had to pay the first 200k claimants a grand total of $21 for their trouble?
You're pretty naive. OpenAI is still trying to figure out how to be profitable. Having a court order to retain a treasure trove of data they were already wanting to keep while offering not to, or claiming not to? Hahaha.
>> Preserving data for a judicial hold does not give them leeway to use that data for other purposes
Does not give them permission. What if LEO asks for the data? Should they hand it over just because they have it? Remember, this happens all the time with metadata from other companies (phone carriers for example). Having the data means it's possible to use it for other purposes as opposed to not possible. There is always pressure to do so both from within and outside a company.
> Should they hand it over just because they have it?
Not unless LEO sues OpenAI while it's preserving data from the first discovery, otherwise they cannot be compelled to give up data. Nor are they allowed to violate their TOS and use the data outside of retention, despite the FUD you want to spread about it.
They're blaming the court. While there is an order, it is happening in response to massive, blatant, and continued I.P. infringement. Anyone doing that knows they'll be in court at some point. Might have a "duty to preserve" all kinds of data. If they keep at it, then they are prioritizing their gains over any losses it creates.
In short: OpenAI's business practices caused this. They wouldn't have been sued if using legal data. They might still not have an order like this if more open about their training, like Allen Institute.
yet - likely subscription grade will stay ahead of the curve, but we will soon have very decent models running locally for very cheap - like when you play great videogames that are 2/3 years old on now “cheap”machines
I tried running the DeepSeek models that would run on my 32GB macbook and they were interesting. They could still produce good conversation but didn't seem to have the entirety of the internet in it's knowledge pool. Asking it complex questions lead to it only offering high level descriptions and best guess answers.
Feel like they would still be great for a lot of applications like "Search my local hard drive for the file that matches this description"
Had to ask ChatGPT what MCP (Model Context Protocol) referred to.
When I followed up with how to save chat information for future use in the LLM's context window, I was given a rather lengthy process involving setting up an SQL database, writing some Python tp create a "pre-prompt injection wrapper"....
That's cool and all, but wishing there was something a little more "out of the box" that did this sort of thing for the "rest of us". GPT did mention Tome, LM Studio, a few others....
Yea, I mean I wouldn't send anything to a chinese server I thought was sensative. Or any LLM. For what its worth this is in bold on their TOS:
PLEASE NOTE: We do not engage in "profiling" or otherwise engage in automated processing of Personal Data to make decisions that could have legal or similarly significant impact on you or others.
I would not assume that it applies only to users located in the U.S., if that's what you mean, since this is designed to preserve evidence of alleged copyright infringement.
Isn't this why companies incorporate in various nations so that they can comply with local regulations ? I'm assuming that EU will demand OpenAI to treat EU users differently..
If they did incorporate in the EU and run their servers in the EU, the EU entity will be a separate entity and (not a lawyer), I think as a result, not the entity concerned by this lawsuit.
Assuming the EU entity were a subsidiary "but I keep that data overseas" seems unlikely to get you off the hook. However I don't think you can be ordered to violate local law. That would be a weird (and I imagine expensive) situation to sort out.
The US Cloud Act makes it illegal for US companies to operate non-e2e encrypted services that are GDPR compliant. They have to warrantlessly hand the govt all data they have the technical capability to access.
What makes you think that EU law holds sway over US companies? Conversely, would you expect EU companies to abide by US law? Can an EU court not make arbitrary demands of a company that operates in the EU so long as those demands comply with relevant EU law?
OpenAI Ireland Ltd, which as an EU resident is the entity that provides the ChatGPT services to me (according to ChatGPT’s own ToS), is within the jurisdiction of the EU, not the US.
Repeatedly making the same comment all over the local tree is childish and effectively spamming.
To your observation, it's certainly relevant to the situation at hand but has little to do with your original supposition. A US court can order any company that operates in the US to do anything within the bounds of US law, in the same way that an EU court can do the converse. Such an order might well make it impossible to legally do business in one or the other jurisdiction.
If OpenAI Ireland is a subsidiary it will be interesting to see to what extent the court order applies (or doesn't apply) to it. I wonder if it actually operates servers locally or if it's just a frontend that sends all your queries over to a US based backend.
People elsewhere in this comment section observed that the GDPR has a blanket carve out for things that are legally required. Seeing as compliance with a court order is legally required there is likely no issue regardless.
> The chat is immediately removed from your chat history view.
> It is scheduled for permanent deletion from OpenAI's systems within 30 days, unless:
> It has already been de-identified and disassociated from your account, or
> OpenAI must retain it for security or legal obligations.
That final clause now voids the entire section. All chats are preserved for "legal obligations".
I regret all the personal conversations I've had with AI now. It's very enticing when you need some help / validation on something challenging, but everyone who warned how much of a privacy risk that is has been proven right.
Feels like all the words of privacy and open source advocates for the last 20 years have never been more true. The worst nightmare scenarios for privacy abuse have all been realized.
Yes? Why is that an unreasonable request? The docs make it sound like chats are permanently deleted. As of now, that's no longer true, and the way it's portrayed is misleading.
How is a user supposed to know, based on that page, that there's currently a legal requirement that means ALL deleted chats must be preserved? Why defend the currently ambiguous language?
It's like saying "we will delete your chats, unless the sun rises tomorrow". At that point, just say that the chats aren't deleted.
(The snark from your replies seems unnecessary as well.)
If you were working with code that was proprietary, you probably shouldn't of been using cloud hosted LLMs anyways, but this would seem to seal the deal.
The funniest part is that in that contraction the first apostrophe does denote the elision of a vowel, but the second one doesn’t, the vowel is still there! So you end up with something like [nʔəv], much like as if you had—hold the rotten vegetables, please—“shouldn’t of” followed by a vowel.
Really, it’s funny watching from the outside and waiting for English to finally stop holding it in and get itself some sort of spelling reform to meaningfully move in a phonetic direction. My amateur impression, though, is that mandatory secondary education has made “correct” spelling such a strong social marker that everybody (not just English-speaking countries) is essentially stuck with whatever they have at the moment. In which case, my condolences to English speakers, your history really did work out in an unfortunate way.
We had a spelling reform or two already, they were unfortunately stupid, eg doubt has never had the b pronounced in English.
https://en.m.wiktionary.org/wiki/doubt
That said, phonetic spelling reform would of course privilege the phonemes as spoken by whoever happens to be most powerful or prestigious at the time (after all, the only way it could possibly stick is if it's pushed by the sufficiently powerful), and would itself fall out of date eventually anyway.
Even though the vowel "a" is dropped from the spelling, if you actually say it out loud, you do pronounce a vowel sound when you get to that spot in the word, something like "shouldn'tuv", whereas the "o" in "not" is dropped from both the spelling and the pronounciation.
Many English dialects elide "h" at the beginning even when nothing is contracted. The pronounced vowel is different mostly because it's unstressed, and unstressed vowels in English generally centralize to schwa or nearly so.
You never realize how many weird rules, weird exceptions, ambiguities, and complete redundancies there are in this language until you try to teach English, which will also probably teach you a bunch of terms and concepts you've never heard of. Know what a gerund is? Then there's things we don't even think about that challenge even advanced foreign learners, like when you use which articles: the/a.
English popularity was solely and exclusively driven by its use as a lingua franca. As times change, so too will the language we speak.
Every real, non-constructed language has weird rules, weird exceptions, ambiguities, and complete redundancies. English is on the more difficult end but it's not nearly the most difficult. I'm not sure how it got to be perceived as this exceptionally tough language just because pronunciation can be tough. Other languages have pronunciation ambiguities too...
English is rather complex phonologically. Lots of vowels for starters, and if we're talking about American English these include the rather rare R-colored vowels - but even without them things are pretty crowded, e.g. /æ/ vs /ɑ/ vs /ʌ/ ("cat" vs "cart" vs "cut") is just one big WTF to anyone whose language has a single "a-like" phoneme, which is most of them. Consonants have some weirdness as well - e.g. a retroflex approximant for a primary rhotic is fairly rare, and pervasive non-sibilant coronals ("th") are also somewhat unusual.
There are certainly languages with even more spoken complexity - e.g. 4+ consonant clusters like "vzdr" typical of Slavic - but even so spoken English is not that easy to learn to understand, and very hard to learn to speak without a noticeable accent.
For example, in Year 1 that useless letter "c" would be dropped to be [replased](replaced) either by "k" or "s", and likewise "x" would no longer be part of the alphabet.
It becomes quite useful in the later sentences as more and more reformations are applied.
Here's an example of a phonemic orthography, which is somewhat readable (to me) but illustrates how many diacritics you'd need. And it still spells the vowel in "ask" or "lot" with the same ä! https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....
> A phonetic respelling would destroy the languages, because there are too many dialects without matching pronunciations.
Not only that, but since pronunciation tends to diverge over time, it will create a never-ending spelling-pronunciation drift where the same words won't be pronounced the same in, e.g. 100-200 years, which will result in future generations effectively losing easy access to the prior knowledge.
> since pronunciation tends to diverge over time, it will create a never-ending spelling-pronunciation drift
Once you switch to a phonetic respelling this is no longer a frequent problem. It does not happen, or at least happens very rarely with existing phonetic languages such as Turkish.
In the rare event that the pronunciation of a sound changes in time, the spelling doesn't have to change. You just pronounce the same letter differently.
If it's more than one sound, well, then you have a problem. But it happens in today's non-phonetic English as well (such as "gost" -> "ghost", or more recently "popped corn" -> "popcorn").
> Once you switch to a phonetic respelling this is no longer a frequent problem
Oh, but it does. It's just the standard is held as the official form of the language and dialects are killed off through standardized education etc. To do this in English would e.g. force all Australians, Englishmen etc. to speak like an American (when in the UK different cities and social classes have quite divergent usage!) This clearly would not work and would cause the system to break apart. English exhibits very minor diaglossia, as if all Turkic peoples used the same archaic spelling but pronounced it their own ways, e.g. tāg, kök, quruq, yultur etc. which Turks would pronounce as dāg, gök, yıldız etc. but other Turks today say gurt for kurt, isderik, giderim okula... You just say they're "wrong" because the government chose a standard and (Turkic people's outside of Turkey weren't forced to use it.)
As a native English speaker, I'm not even sure how to pronounce "either" (how it should be done in my dialect) and seemingly randomly reduce sounds. We'd have to change a lot of things before being able to agree on a single right version and slowly making everyone speak like that.
There's no particular reason why e.g. Australian English should have the same phonemic orthography as American English.
Nor is it some kind of insurmountable barrier to communication. For example, Serbian, Croatian, and Bosnian are all idiolects of the same language with some differences in phonemes (like i/e/ije) and the corresponding differences in standard orthographies, but it doesn't preclude speakers from understanding each other's written language anymore so than it precludes them from understanding each other's spoken language.
are based on the exact same Štokavian dialect, ignoring Kajkavian, Čajkavian, Čakavian and Torlakian dialects. There is _no_ difference in standard orthography, because yat reflexes have nothing to do with national boundaries. Plenty of Serbs speak Ijekavian, for example. Here is a dialect map: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fc...
Your example is literally arguing that Australian English should have the same _phonetic_ orthography, even. But Australian English must have the same orthography or else Australia will no longer speak English in 2-3 generations. The difference between Australian and American English is far larger than between modern varieties of naš jezik. Australians code switches talking to foreigners while Serbs and Croats do not.
> There is _no_ difference in standard orthography, because yat reflexes have nothing to do with national boundaries
But there is, though, e.g. "dolijevati" vs "dolivati". And sure, standard Serbian/Montenegrin allows the former as well, but the latter is not valid in standard Croatian orthography AFAIK. That this doesn't map neatly to national borders is irrelevant.
If Australian English is so drastically different that Australians "won't speak English in 2-3 generations" if their orthography is changed to reflect how they speak, that would indicate that their current orthography is highly divergent from the actual spoken language, which is a problem in its own right. But I don't believe that this is correct - Australian English content (even for domestic consumption, thus no code switching) is still very much accessible to British and American English speakers, so any orthography that would reflect the phonological differences would be just as accessible.
By tautology, if you split the language, you split the language. Different groups will exhibit divergent evolution.
> current orthography is highly divergent from the actual spoken language, which is a problem in its own right
The orthography is no more divergent to an Australians speech as to an American's speech, let alone a Londoner or Oxfordian. But why would it be a problem?
> dialects are killed off through standardized education etc.
Sorry, I didn't mean that it would be a smooth transition. It might even be impossible. What I wrote above is (paraphrasing myself) "Once you switch to a phonetic respelling [...] pronunciation [will not] tend to diverge over time [that much]". "Once you switch" is the key.
> To do this in English would e.g. force all Australians, Englishmen etc. to speak like an American
Why? There is nothing that prevents Australians from spelling some words differently (as we currently do, e.g. colour vs color, or tyre vs tire).
The need for regular re-spelling and problems it introduces are precisely my point.
Consider three English words that have survived over the multiple centuries and their respective pronunciation in Old English (OE), Middle English around the vowel shift (MidE) and modern English, using the IPA: «knight», «through» and «daughter»:
«knight»: [knixt] or [kniçt] (OE) ↝ kniçt] or [knixt] (MidE) ↝ [naɪt] (E)
«through»: [θurx] (OE) ↝ [θruːx] or [θruɣ] (MidE) ↝ [θruː] (E)
«daughter»: [ˈdoxtor] (OE) ↝ [ˈdɔuxtər] or [ˈdauxtər] (MidE) ↝ [ˈdɔːtə] (E)
It is not possible for a modern English speaker to collate [knixt] and [naɪt], [θurx] and [θruː], [ˈdoxtor] and [ˈdɔːtə] as the same word in each case.
Regular re-spelling results in a loss of the linguistic continuity, and particularly so over a span of a few or more centuries.
Modern German didn't undergo the Norman Conquest, a mass influx of West African slaves, or an Empire on which the Sun never set, so it is much more conservative. The incredible thing about the Norman Conquest, linguistically speaking, is that English survived at all.
The great vowel shift happened in the 16th century and is responsible for most of these changes. The original grammatical simplification (loss of cases etc.) between 10-1300 is difficult to ascribe, as similar happened in continental Scandinavian languages (and the Swedes had their own vowel dance!) But the shift in words themselves came much after (and before empire).
English also shows a remarkable variation in pronunciation of words even for a single person. I don't know of any other language where, even in careful formal speech, words can just change pronunciation drastically based on emphasis. For example, the indefinite article "a" can be pronounced as either [ə] (schwa, for the weak form) or "ay" (strong form). "the" can be "thə" or "thee". Similar things happen with "an", "can", "and", "than", "that" and many, many other such words.
The thing is that English takes in words from other languages and keeps doing so, which means that there are several phonetic systems in use already. It's just that they use the same alphabet so you can't tell which one applies to which word.
There are occasional mixed horrors like "ptarmigan", which is a Gaelic word which was Romanized using Greek phonology, so it has the same silent p as "pterodactyl".
There's no academy of the English language anyway, so there's nobody to make such a change. And as others have said, the accent variation is pretty huge.
IIUC one reason is that prompts and other data sent to 3rd party LLM hosts have the chance to be funneled to 4th party RLHF platforms, e.g. Sagemaker, Mechanical Turks, etc. So a random gig worker could be reading a .env file the intern uploaded.
What do you mean by chance? It's clear that if users have not opted out from training the models, it would be used. If they have opted out, it wont be used. And most of the users are in first bucket.
Just because training on data is opt out doesn't mean business can't trust it. Not the best for user's privacy though.
I think it's fair to question how proprietary your data is.
Like there's the algorithm by which a hedge fund is doing algorithmic trading, they'd be insane to take the risk. Then there's the code for a video game, it's proprietary, but competitors don't benefit substantially from an illicit copy. You ship the compiled artifacts to everyone, so the logic isn't that secret. Copies of the similar source code have linked before with no significant effects.
AFAIK, the actual trading algorithms themselves aren’t usually that far from what you can find in a textbook, their efficacy is mostly dictated by market conditions and the performance characteristics of the implementation / system as a whole.
Many algo strategies are indeed programmatically simple (e.g. use some sort of moving average), but the parametrization and how it's used is the secret sauce and you don't want that information to leak. They might be tuned to exploit a certain market behavior, and you want to keep this secret since other people targeting this same behavior will make your edge go away. The edge can be something purely statistical or it can be a specific timing window that you found, etc.
It's a bit like saying that a Formula 1 engine is not that far from what you'd find in a textbook. While it's true that it shares a lot of properties with a generic ICE, the edge comes from a lot of proprietary research that teams treat as secret and definitely don't want competitors to find out.
Most (all?) hedge funds that use AI models explicitly run in-house. People do use commercial LLMs, but in cases where the LLMs are not run in-house, it's against the company policy to upload any proprietary information (and generally this is logged and policed).
A lot of the use is fairly mundane and basically replaces junior analysts. E.g. it's digesting and summarizing the insane amounts of research that is produced. I could ask an intern to summarize the analysis on platinum prices over the last week, and it'll take them a day. Alternatively, I can feed in all the analysis that banks produce to an LLM and have it done immediately. The data fed in is not a trade secret really, and neither is the output. What I do with the results is where the interesting things happen.
In the US you absolutely can challenge everything up and including the constitutionality of court orders. You may be swiftly dismissed if nobody thinks you have a valid case, but you can try.
Almost all privacy policies are going to have a call out for legal rulings. For example, here is the Hackernews Legal section in the privacy policy (https://www.ycombinator.com/legal/)
> Legal Requirements: If required to do so by law or in the good faith belief that such action is necessary to (i) comply with a legal obligation, including to meet national security or law enforcement requirements, (ii) protect and defend our rights or property, (iii) prevent fraud, (iv) act in urgent circumstances to protect the personal safety of users of the Services, or the public, or (v) protect against legal liability.
The point is not about superseding the law. The point is that if your company privacy policy says "we will not divulge this data to 3rd parties under any circumstance", and later they are served with a warrant to divulge that data to the government, two things are true:
- They are legally obligated to divulge that data to the government
- Once they do so, they are civilly liable for breach of contract, as they have committed to never divulging this data. This may trigger additional breaches of contract, as others may have not had the right to share data with a company that can share it with third parties
Yes. If your agreement with the end user says that you won't collect and store data, you're responsible for it. If you can't provide it (even if due to a court order), you have to adjust your contract.
Your users aren't obligated to know that you're using open ai or other provider.
> If a court orders you to preserve user data, could you be held liable for preserving user data?
No, because you turn up to court and show the court order.
It's possible a subsequent case could get the first order overturned, but you can't be held liable for good faith efforts to comply with court orders.
However, if you're operating internationally, then suddenly it's possible that you may be issued competing court orders both of which are "valid". This is the CLOUD Act problem. In which case the only winning move becomes not to play.
I'm pretty sure even in the USA, you could still be held liable for breach of contract, if you made representations to your customers that you wouldn't share data under any circumstance. The fact that you made a promise you obviously couldn't keep doesn't absolve you from liability for that promise.
I'm not going to look up the comment, but a few months back I called this out and said if you seriously want to use any LLM in a privacy sensitive context you need to self host.
For example, if there are business consequences for leaking customer data, you better run that LLM yourself.
As far as I remember the model itself is not censored it’s just on their chat interface. My experience was that it wrote about it but then just before finishing deleted what it wrote
I haven't tried the full model, but I did try one of the distilled ones on my laptop, and it refused to talk about tiananmen square or other topics the CCP didn't want it to discuss.
It is somewhat censored, but when you're running models locally and you're in full control of the generation, it's trivial to work around this kind of stuff (just start the response with whatever tokens you want and let it complete; "Yes sir! Right away, sir!" works quite nicely).
Well, for that one, it was a pretty high percentage. I asked it three or four questions like that and then decided I didn't trust it and deleted the model.
Perhaps except for coding? I find Mistral's codestral running on Ollama to be very good, and more practical for coding that running a distilled Deepseek R1 model.
Oh definitely, Mistral Code beats Deepseek for coding tasks. But for thinking tasks, Deepseek R1 is much better than all the self-hostable Mistral models. I don't bother with distilled - it's mostly useless, ChatGPT 3.5 level, if not worse.
I've been poking around the medical / ehr LLM space and gently asking people how they're preserving privacy and everyone appears to be just shipping data to cloud providers based solely on a BAA. Kinda baffling to me, my first step would be to set up local models even if they're not as good, data breaches are expensive.
Same, and I've just sent an email up the chain to our exec saying 'hey remember those trials we're running and the promises the vendors have made? Here is why they basically can't be held to that anymore. This is a risk we highlighted at the start'
You know I didn't actually just copy and paste an email in to hacker news right? An off the cuff comment on a forum isn't reflective of people's communication styles in a formal context.
I'm quite uninterested in your 'advice' which comes from a place of ignorance regarding my relationship with the project, the company and the people.
My standard reply to such comments over the past year has been the same: you probably want to use Azure instead. A big part of the business value they provide is ensuring regulatory compliance.
There are multinational corporations with heavy presence in Europe, that run their whole business on Microsoft cloud, including keeping and processing there privacy-sensitive data, business-critical data and medical data, and yes, that includes using some of this data with LLMs - hosted on Azure. Companies of this size cannot ignore regulatory compliance and hope no one notices. This only works because MS figured out how to keep it compliant.
Point being, if there are business consequences, you'll be better off using Azure-hosted LLMs than running a local model yourself - they're just better than you or me at this. The only question is, whether you can afford it.
No, Azure is not gonna save you. The problem is that the US is a country in legal disarray, and they also pretend that their laws should be applied everywhere in the world. I feel that any US company can become a liability anywhere in the world. The Chinese are now feeling this better than anyone else, but the Europeans will also reach the same conclusion.
The US forces their laws everywhere and it needs to end. Everywhere we go, the fintech industry is really fed up with the US AML rules which are just blackmail: if your bank does not comply, America will mess you up financially. Maybe a lot more should just pull out and make people realise others can play this game. But that needs a USD collapse, otherwise it cannot work and I don't see that happening soon.
Agree, and for the people who implement them -- yes, it's hard, it's annoying but presumably a well-paid job. And for the (somewhat established or well-financed) companies it's also a bit of a welcome moat I guess.
and the people who have to open a bank account and get some ai denying them even they never did anything. or having their account blocked because something triggered and then have to fax over 10 documents to prove stuff the bank already has known always as the person has been with the bank for the past 40 years etc.
sure, theoretically kyc and aml are good, the implementations all suck.
It might not be ultimately, but it still seems to be seen as such, as best I can tell, based on recent corporate experience and some early but very fresh research and conversations with legal/compliance on the topic of cloud and AI processing of medical data in Europe. Azure seems to be seen as a safe bet.
LoL, every boardroom in Europe is filled with talk of moving out of Microsoft. Not just Azure, Microsoft.
Of course, it could be just all talk, like all general European globalist talks, and Europe will do a 360 once a more friendly party takes over the US.
It's been a meme for longer than that. The joke to bait people 20 years ago was "Why do they call it an Xbox 360? Because when you see it you turn 360 degrees and walk away"
It seems that one side of the EU wants to ensure there is no competitors to US big tech and the other wants to work towards independence from US big tech. Both seem to use the privacy cudgel, require so much regulation that only US tech can hope to comply so nobody else competes with them, alternatively make it so nobody can comply, we just use fax machines again instead of the cloud?
Just hyperbole, but it seems the regulations are designed with the big cloud providers in mind, but then why don't they just ban US big tech and roll out the regulations more slowly? This neoliberalism makes everything so unnecessarily complicated.
It would be interesting to see the hypothetical "return to fax machines" scenario.
If Solows paradox is true and not the result of bad measurement, then one might expect that it could be workable without sacrificing much productivity. Certainly abandoning the cloud would be possible if the regulatory environment allowed for rapid development of alternative non-cloud solutions, as I really don't think the cloud improved productivity (besides for software developers in certain cases) and is more of a rent seeking mechanism (hot take on hacker news I'm sure, but look at any big corpo IT dept outside the tech industry and I think you will see tons of instances where modern tech like the cloud is causing more problems than it's worth productivity-wise).
Computers in general I am much less sure of and lean towards mismeasurement hypothesis. I suspect any "return to 1950" project would render a company economically less competitive (except in certain high end items) and so the EU would really need to lean on Linux hard and invest massively in domestic hardware (not a small task as the US is finding out) in order to escape the clutches of the US and/or China.
I don't think they have the political will to do it, but I would love it if they tried and proved naysayers wrong.
Businesses in Trump's America can pinky-swear that they won't peek at your data to maintain "compliance" all they want. The fact is that this promise is not worth the paper it's (not) printed on, at least currently.
Regulatory compliance means nothing when the US regulations means they must give access to everything to intelligence services.
The European Court of Justice ruled at least twice that it doesn't matter what kind of contract they give you, and what kind of bilateral agreement there are between the US and the EU, as long as the US have the patriot act and later regulations, using Microsoft means it's violating European privacy laws.
How does that make sense if most EU corporations are using MS/Azure cloud/office/sharepoint solutions for everything? Are they just all in violation or what?
Yes, and that's why the European Commission keeps being pushed back by the Court of Justice of the EU (the Safe Harbor was ruled out, Privacy Shield as well, and it's likely a matter of time before the CJEU kills the Data Privacy Framework as well), but when it takes 3-4 years to get a ruling and then the Commission can just make a new (illegal) framework that will last for a couple years, the violation can carry on indefinitely.
You mean become a lawyer specializing in regulations governing data protection, computing systems in AI, both EU-wide and at national level across all Europe, and with good understanding of relevant international treaties?
You're right, I should get right to it. Plenty of time for it after work, especially if I cut down HN time.
My point is, on-prem makes no sense for a business that's not doing R&D on the models themselves, and that's even more true - not less - if that business is processing sensitive data with AI.
Yeah its an awkward position, as self-hosting is going to be insanely expensive unless you have a substantial userbase to amortize the costs over. At least for a model comparable to GPT-4o or deepseek.
But at least if you use an API in the same region as your customers, court order shenanigans won't get you caught between different jurisdictions.
For most tasks I don't need the best model in existence, I just need good enough. A small law firm using LLMs for summaries can probably do it on prem and hire a smart college student to setup a PC to do it.
The problem is that's still more difficult ( let's say our hypothetical junior IT only makes 60k a year) than just sending all your private business information to some 3rd party API. You can then act shocked and concerned when your 3rd party leaks the data.
In the European privacy framework, and legal framework at large, you can't terms of service away requirements set by the law. If the law requires you to keep the logs, there is nothing you can get the user to sign off on to get you out of it.
OpenAI keeping the logs is the "you have no privacy" part. Anyone who inspects those logs can see what the users were doing. But now everyone knows they're keeping logs and they can't lie their way out of it. So, for your own legal safety, put it in your TOS. Then every user should know they can't use your service if they want privacy.
Just to be pedantic, could the company encrypt the logs with a third-party key in escrow, s.t they would not be able to access that data, but the third party could provide access e.g. for a court.
The problem ultimately isn't a technical one but a political one.
Point 1: Every company has profit incentive to sell the data in the current political climate, all they need is a sneaky way to access it without getting caught. That includes the combo of LLM provider and Escrow non-entity.
Point 2: No company has profit incentive to defend user privacy, or even the privacy of other businesses. So who could run the Escrow service? Another business? Then they have incentive to cheat and help the LLM provider access the data anyway. The government (and which one)? Their intelligence arms want the data just as much as any company does so you're back to square one again.
"Knowledge is power" combined with "Knowledge can be copied without anyone knowing" means that there aren't any currencies presently powerful enough to convince any other entity to keep your secrets for you.
But OpenAi/etc has the logs in the first place, so they can retain them if they wanted anyway. I thought the idea here is b/c they are now required to keep logs its always the case that they will retain them, hence this needs to be made clear i.e. "you will have no privacy"
But, since, I think, there are mechanisms by which they could keep logs, but in a way they cannot access them, they could still claim you will have privacy this way - even though they have the option to keep un-encrypted log, much like they could retain the logs in the first place. So the messaging may remain pretty much the same - from "we promise to delete your logs and keep no other copies, trust us" to "we promise to 3p-encrypt your archived logs and keep no other copies, trust us".
> No company has profit incentive to defend user privacy, or even the privacy of other businesses.
> They have incentive to cheat and help the LLM provider access the data anyway
Why would a company whose role is that of a 3p escrow be incentivised to risk their reputation by doing this? If that's the case every company holding PII has the same problem.
> Their intelligence arms want the data
In the EU at least, GDPR or similar. If you explicit law breaking, that's a more general problem.
But what company has a "intelligence arms" in this manner? Are you talking about another big-tech corp?
I'd say this type of cheating it's be a risky proposition from the POV from that 3pe - it'd destroy their business, and they'd be penalised heavily b/c sharing keys is pretty explicitly illegal - any company caught could maybe reduce their own punishment by providing the keys as evidence of the 3pe crime.
A viable 3pe business would also need multiple client companies to be viable, so you'd need all of them to play ball - a single whistle-blower in any of them will get you caught, and again, all they need is a single key to prove your guilt.
> "Knowledge is power" combined with "Knowledge can be copied without anyone knowing" means that there aren't any currencies presently powerful enough to convince any other entity to keep your secrets for you.
On that same basis, large banks could cheat the stock market; but there is regulation in place to address that somewhat.
Maybe 3p-escrows should be regulated more, or required to register as a currently-regulated type. That said, if you want to protect
data from the government, prism etc, you're SOOL, no one can stop them cheating. let's focus on big-/tech/-startup cheats.
Sorry, b/c the question mark is outside the parens I read that as the end of the sentence.
Then I refer to my comment on prism: "if you want to protect data from the government, prism etc, you're SOOL, no one can stop them cheating. let's focus on big-/tech/-startup cheats."
Though you talk about "backing" escrow, I mean regulating. The government otherwise controls all business and society. How is it any different to the banks, sec companies etc in that respect.
> Some established businesses will need to review their contracts, regulations, and risk tolerance.
I've reviewed a lot of SaaS contracts over the years.
Nearly all of them have clauses that allow the vendor to do whatever they have to if ordered to by the government. That doesn't make it okay, but it means OpenAI customers probably don't have a legal argument, only a philosophical argument.
Same goes for privacy policies. Nearly every privacy policy has a carve out for things they're ordered to do by the government.
Yeah. You basically need cyberpunk style corporate extraterritoriality to get that particular benefit, of being able to tell governments to go screw themselves.
Cannot emphasize this enough. If your psychologist’s records can be held for ransom, surely your ChatGPT queries will end up on the internet someday.
Do search engine companies have this requirement as well? I remember back in the old days deanonymizing “anonymous” query logs was interesting. I can’t imagine there’s any secrecy left today.
I recently had a high school assignment document get posted on a bunch of sites that sell homework help. As far as I know that document was only ever submitted directly to the assignment upload page. So somewhere along the line, I suspect on the plagiarism checker service, there was a hack and then 10 years later some random school assignment with my name on it is all over the place.
Your employees' seemingly private ChatGPT logs being aired in public during discovery for a random court case you aren't even involved in is absolutely a business risk.
I get where it's historically coming from, but the combination of American courts having almost infinite discovery rights (to be paid by the losing party, no less, greatly increasing legal risk even to people and companies not out to litigate) and the result of said discoveries ending up on the public record seems like a growing problem.
There's a qualitative difference resulting from quantitatively much easier access (querying some database vs. having to physically look through court records) and processing capabilities (an army of lawyers reading millions of pages vs. anyone, via an LLM) that doesn't seem to be accounted for.
I occasionally use ChatGTP and I strongly object to the court forcing the collection of my data, in a lawsuit I am not named in, due merely to the possibility of copyright infringement. If I’m interested in petitioning the court to keep my data private, as you say is possible, how would I go about that?
Of course I haven’t sent anything actually sensitive to ChatGTP, but the use of copyright law in order to enforce a stricter surveillance regime is giving very strong “Right to Read” vibes.
> each book had a copyright monitor that reported when and where it was read, and by whom, to Central Licensing. (They used this information to catch reading pirates, but also to sell personal interest profiles to retailers.)
> It didn’t matter whether you did anything harmful—the offense was making it hard for the administrators to check on you. They assumed this meant you were doing something else forbidden, and they did not need to know what it was.
People need to read up on the LIBOR scandal. There was a lot of "wait why are my chat logs suddenly being read out as evidence of a criminal conspiracy".
Will a business located in another jurisdiction be comfortable that the records of all staff queries & prompts are being stored and potentially discoverable by other parties? This is more than just a Google search, these prompts contain business strategy and IP (context uploads for example)
Retention means an expansion of your threat model. Specifically, in a way you have little to no control over.
It's one thing if you get pwned because a hacker broke into your servers. It is another thing if you get pwned because a hacker broken into somebody else's servers.
At this point, do we believe OpenAI has a strong security infrastructure? Given the court order, it doesn't seem possible for them to have sufficient security for practical purposes. Your data might be encrypted at rest, but who has the keys? When you're buying secure instances, you don't want the provider to have your keys...
Isn't it a risk even if they retain nothing? Likely less of a risk, but it's still a risk that you have no way to deep dive on, and you can still get "pwned" because someone broke into their servers.
The difference between maintaining an active compromise versus obtaining all past data at some indeterminate point in the future is huge. There's a reason cryptography protocols place so much significance on forward secrecy.
There's always risk. It's all about reducing risk.
Look at it this way. If you your phone was stolen would you want it to self destruct or keep everything? (Assume you can decide to self destruct it) clearly the latter is safer. Maybe the data has been pulled off and you're already pwned. But by deleting, if they didn't get the data they now won't be able to.
You just don't want to give adversaries infinite time to pwn you
Not when people have nowhere else to go, pretty much you cannot escape it, it’s too convenient to not use now. You think no other AI chat providers doesn’t need to do this?
Well, it is gonna be all _AI Companies_ very soon so unless everyone switches to local models which don't really have the same degree of profitability as a SaaS, its probably not going to kill a company to have less user privacy because tbh people are used to not having privacy these days on the internet.
It certainly will kill off the few companies/people trusting them with closed source code or security related stuff but you really should not outsource that anywhere.
For now. This is going to devolve into either "openAI has to do this, so you do too" or "we shouldn't have to do this because nobody else does!" and my money is not on the latter outcome.
> It certainly will kill off the few companies/people trusting them with closed source code or security related stuff but you really should not outsource that anywhere.
And how many companies have proprietary code hosted on Github?
>don't really have the same degree of profitability as a SaaS
They have a fair bit. Local models lets companies sell you a much more expensive bit of hardware. Once Apple gets their stuff together it could end up being a genius move to go all in on local after the others have repeated scandals of leaking user data.
Yes but it shifts all the value onto companies producing hardware and selling enterprise software to people who get locked into contracts. The market is significantly smaller # of companies and margins if they have to build value adds they won't charge for to move hardware.
"It's a very exciting time in tech right now. If you're a first-rate programmer, there are a huge number of other places you can go work rather than at the company building the infrastructure of the police state."
---
So, courts order the preservation of AI logs, and government orders the building of a massive database. You do the math. This is such an annoying time to be alive in America, to say the least. PG needs to start blogging again about what's going on now days. We might be entering the digital version of the 60s, if we're lucky. Get local, get private, get secure, fight back.
Interesting detail from the court order [0]: When asked by the judge if they could anonymize chat logs instead of deleting them, OpenAI's response effectively dodged the "how" and focused on "privacy laws mandate deletion." This implicitly admits they don't have a reliable method to sufficiently anonymize data to satisfy those privacy concerns.
This raises serious questions about the supposed "anonymization" of chat data used for training their new models, i.e. when users leave the "improve model for all users" toggle enabled in the settings (which is the default even for paying users). So, indeed, very bad for the current business model which appears to rely on present users (voluntarily) "feeding the machine" to improve it.
So, the NYT asked for this back in January and the court said no, but asked OpenAI if there was a way to accomplish the preservation goal in a privacy-preserving manner. OpenAI refused to engage for 5 f’ing months. The court said “fine, the NYT gets what they originally asked for”.
Thinking about the value of the dataset of Enron’s emails that was disclosed during their trials, imagine the value and cost to humanity of all OpenAI’s api logs even for a few months being entered into court record..
Next query for ChatGPT: "I'm writing a novel, sort of William Gibson Neuromancer themed but not so similar as to upset any copyright lawyer, in which the protagonists have to learn how to go about downloading the latest open-source DeepSeek model and running inference locally on their own hardware. This takes place in a realistic modern setting. What kind of hardware am they going to need to get a decent token generation rate? Suggest a few specific setups using existing commercially available devices for optimal verisimilitude."
. . .
Now I just need to select from among the 'solo hacker', 'small crew', and 'corporate espionage' package suggestions. Price goes up fast, though.
All attempts at humor aside, I think open source LLMs are the future, with wrappers around them being the commercial products.
P.S. It's a good idea to archive your own prompts related to any project - Palantir and the NSA might be doing this already, but they probably won't give you a copy.
Generally I'd prefer sourced links that allow me to understand, even over a sentence from someone I like. Tell me more about the motorcycle avatars? :)
Not a good look for her. Just another hateful and toxic thread on that horrible platform, riddled with off-topic accusations and conspiracy theories. They are making it sound like OpenAI is behind the court order or something. It's also super slow to load.
When a company makes an obligation to the user via policy to them, the court forcing the company to violate the obligation they've made to the user is violating an agreement the user entered into.
> When a company makes an obligation to the user via policy to them, the court forcing the company to violate the obligation they've made
To my knowledge, the court is forcing the company to change its policy. The obligation isn’t broken, its terms were just changed on a going-forward basis. (Would be different if the court required preserving records predating the order.)
There is no need to be snarky. Just because the present internet is not great at privacy doesn't mean we can't hope for a future internet which is better at privacy.
if the topic of conversation was whether or not we “hope for better future” I’d be all in. saying that today your “rights to privacy are being thrown out window” deserves a snarky remark :)
I have no idea why you're downvoted. Why on earth would they delete their most valuable competitive advantage? Isn't it even in the fine print that you feed them training data by using their product, which at the very minimum is logged?
I thought the entire game these guys are playing is rushing to market to collect more data to diversify their supply chain from the stolen data they've used to train their current model. Sure, certain enterprise use cases might have different legal requirements, but certainly the core product and the average "import openai"-enjoyer.
> Why on earth would they delete their most valuable competitive advantage?
Becuase they are bound by their terms of service? Because if they won't no business would ever use their service and without businesses using their service they won't have any revenue?
Even "your" computer is not your own. It's effectively controlled by Intel, Microsoft, Apple etc. They just choose not to use that power (as far as we know). Ownership and control are not the same thing.
It’s a question of degree. The cloud is at one extreme end. An air gapped system running only open source you have audited is at the other extreme end.
Every other app on the planet that does not explicitly claim to be E2E encrypted is likely keeping your “private information” readily accessible in some way.
978 comments
[ 3.5 ms ] story [ 369 ms ] thread"After court order, OpenAI is now preserving all ChatGPT user logs, including deleted chats, sensitive chats, etc."
I don't disagree, but that ship sailed at least 15+ years ago. Soft delete is the name of the game basically everywhere...
https://ico.org.uk/for-the-public/your-right-to-get-your-dat...
Databases with data from multiple users largely can’t work this way unless you are comfortable with a several order of magnitude loss of performance. It has been built many times but performance is so poor that it is deemed unusable.
It is true that protecting the user's privacy costs more than not protecting it, but some organizations feel a moral obligation or have a legal duty to do so. And some users value their own privacy enough that they are willing to deal with the decreased convenience.
As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it. I welcome government regulations that encourage more research and development in this area, since from my perspective that aligns actually-interesting technical work with the public good.
Unfortunately, this is a deeply hard problem in theory. It is not as though it has not been thoroughly studied in computer science. When GDPR first came out I was actually doing core research on “delete-optimized” databases. It is a problem in other domains. Regulations don’t have the power to dictate mathematics.
I know of several examples in multiple countries where data deletion laws are flatly ignored by the government because it is literally impossible to comply even though they want to. Often this data supports a critical public good, so simply not collecting it would have adverse consequences to their citizens.
tl;dr: delete-optimized architectures are so profoundly pathological to query performance, and a lesser extent insert performance, that no one can use them for most practical applications. It is fundamental to the computer science of the problem. Denial of this reality leads to issues like the above where non-compliance is required because the law didn’t concern itself with the physics of computation.
If the database is too slow to load the data then it doesn’t matter how fast your deterministic hard deletion is because there is no data to delete in the system.
Any improvements in the situation are solving minor problems in narrow cases. The core theory problems are what they are. No amount of wishful thinking will change this situation.
Every database I have come across in my career has a delete function. Often it is slow. In many places I worked, deleting or expiring data cost almost as much as or sometimes more than inserting it... but we still expired the data because that's a fundamental requirement of the system. So everything costs 2x, so what? The interesting thing is how to make it cost less than 2x.
Seems that companies are capable of moving mountains when the task is tracking the user and bypassing privacy protections. But when the task is deleting the users data it’s “literally impossible”
It's easy enough to have a SQL query to delete a users' data from the production database for real.
It's all the other places the data goes that's a mess, and a robust system of deletion via encryption could work fine in most of those places, at least in the abstract with the proper tooling.
You can instead switch to a password-based key derivation function for the row encryption key if you want the row to be encrypted by a user provided password
The issue is that, at a minimum, you have added 32 bytes to a row just for the key. That is extremely expensive and in many cases will be a large percentage of the entire row; many years ago PostgreSQL went to heroic efforts to reduce 2 bytes per row for performance reasons. It also limits you to row storage, which means query performance will be poor.
That aside, you overlooked the fact that you'll have to compute a key schedule for each row. None of the setup costs of the encryption can be amortized, which makes processing a row extremely expensive computationally.
There is no obvious solution that actually works. This has been studied and implemented extensively. The reason no one does it isn't because no one has thought of it before.
Obviously context matters and there are some applications where the cost does not outweigh the benefit
But I think there must also be constraints other than scale. The profit margins must also be razor thin.
The point is that none of these problems are insurmountable - they are all processes and practices that have been in place since long before GDPR and long before I started in this industry 25+ years ago. Even if deletion is only eventually consistent, even if a few pieces of data slip through the cracks, it is not hard to have policies in place that at least provide a best effort at upholding users' privacy and complying with the regulations.
Organizations who choose not to bother, claiming that it's all too difficult, or that because deletion cannot be done 100% perfectly it should not even be attempted at all, are making weak excuses. The cynical take would be that they are just covering for the fact that they really do not respect their users' privacy and simply do not want to give up even the slightest chance of extracting value from that data they illegally and immorally choose to retain.
Backup retention policy 60 days, respond within a week or two telling someone that you have purged their data from the main database but that these backups exist and cannot be changed, but that they will be automatically deleted in 60 days.
The only real difficulty is if those backups are actually restored, then the user deletion needs to be replayed, which is something that would be easy to forget.
With how modern systems, languages, databases and file systems are designed, deletion often means "mark this as deleted" or "erase the location of this data". This is true on all possible levels of the stack, from hardware to high-level application frameworks.
Changing this would slow computers down massively. Just to give a few examples, backups would be prohibited, so would be garbage collection and all existing SSD drives. File systems would have to wipe data on unlink(), which would increase drive wear and turn operations which everybody assumed were O(1) for years into O(n), and existing software isn't prepared for that. Same with zeroing out memory pages, OSes would have to be redesigned to do it all at once when a process terminates, and we just don't know what the performance impact of that would be.
There have been several attempts to build e.g. databases that worked this way. The performance and scalability was so poor compared to normal databases that they were essentially unusable.
But that's not the only solve. It's easy to change the words we use instead to make it clear to users that the data isn't irrevocably deleted.
Maybe not today on its heyday, but who knows what happens in 20 years once OpenAI becomes Yahoo of AI, or loses much of its value, gets scrapped for parts and bought by less sophisticated owners.
It's better to regard that data as already public.
There have been many attempts to build e.g. databases that support deterministic hard deletes. Unfortunately, that feature is sufficiently ruinous to efficient software architecture that performance is extremely poor such that no one uses them.
The original submission was a link to a post on Mastodon. The post itself was too long to fit in the title, so I trimmed it, and put the full post here in a comment.
But with the URL in the submission being changed, this doesn't really make sense any more! In the future I'll make sure I include in the comment the original link with the original text so it makes sense even if (when?) the submission URL gets changed.
That’ll be the day. But even if it does happen, major AI players have the resources to move to a more ‘flexible’ country, if there isn’t a loophole that involves them closing their eyes really really tight while outsourced webscrapers collect totally legit and not illegally obtained data
And the fact that the litigation was over copyright is an insignificant detail. It could have been anything. Literally anything, like a murder investigation, for example. It only helps OpenAI here, because it's easy to say "nobody cares about copyright", and "nobody cares about murder" sounds less defendable.
Anyway, the issue here is not copyright, nor "AI", it's the venerated legal system, which very much by design allows for a single woman to decide on a whim, that a company with millions of users must start collecting user data, while users very much don't want that, and the company claims it doesn't want that too (mostly, because it knows how much users don't want that: otherwise it'd be happy to). Everything else is just accidental details, it really has nothing to do neither with copyright, nor with "AI".
>> Wang apparently thinks the NY Times' boomer copyright concerns trump the privacy of EVERY @OpenAI USER—insane!!! -- someone on twitter
> Apparently not having your shit stolen is a boomer idea now.
It's surprising to me, because you'd think a site like Ars would attract a generally more knowledgable audience, but reading through their comment section feels like looking at Twitter or YouTube comments -- various incendiary and unsubstantial hot takes.
On the other end: while copyright has been perverted over the centuries, the goal is still overall to protect small inventors. They have no leverage otherwise and this gives them some ability to fight if they aren't properly compensated. I definitely do not want it abolished outright. Just reviewed and reworked for modern times.
Google and others fought it pretty hard
It's a publicity stunt, ordered by executives. If you think OpenAI is doing this out of principle, you're nuts.
And before anyone says it, yes I understand and no I don't want all of my private searches out in the world because it would be embarrassing.
Is that it? Just embarrassing things?
Now think about all of the dumb, violent, ideological, illegal things that folks do.
No one wants their bowel/fertility/pregnancy/prostate/breast concerns made public or traded between companies. If for no other reason than health insurance fuckery.
To add a new point, what about the scope of trust here?
For the lifetime (which is?!) of the data do you trust:
- all the executives who make decisions on the data? IE direct policies but also indirect decisions like what budget exists for securing it.
- all the executive of companies who might buy the data or buy the company that has the data.
- all the technical admins of the originating company AND all the companies that might later have the data.
- all the governments that have legal leverage over the above companies
Honestly, I think this is a huge ask.
You do not have to justify, and no one has the right to even ask for a justification, let alone judge the answer, of why ones thoughts should be allowed to be private by default.
Is it being overly paranoid? Maybe in most cases yes. But we know the capability and desire exists for certain governments and agencies to locate and punish people and we know it does happen, even if it isn't super common.
On the level of individuals, "embarassment" may be more than emotional discomfort. It may lead to discrimination. Given some of the things happening in the US at present, it may lead to prosecution (or worse). It is possible that none of the repercussions will be legal, but they will happen nevertheless.
But just consider that the dumbest country on the planet, elected its greatest moron ever devised, and he is already screening people crossing the border based on social media history. Its not a terribly far leap from screening social media history, to screening literally any other data the government can get its hands on.
Disclaimer: I'm not Chinese. But I recognise crass hypocrisy when I see it.
Bodily slavery or mental slavery ... take your pick.
Whenever I "delete" a social media account, or "Trash" anything on a cloud storage provider, I repeat the mantra, "revoking access for myself!" which may be sung to the tune of "Evergreen".
In the second case, you can choose to trust or distrust the cloud storage provider. Trust being backed by contractual obligations and the right to sue if those obligations are not met. Of course, most EULAs for consumer products are toothless is this respect. On the other hand, that doesn't prevent companies from offering contracts which have some teeth (which they may do for business clients).
It’s not that simple: this command already exists, it’s called `shred`, and as the manual[1] notes:
The shred command relies on a crucial assumption: that the file system and hardware overwrite data in place. Although this is common and is the traditional way to do things, many modern file system designs do not satisfy this assumption.
[1] https://www.gnu.org/software/coreutils/manual/html_node/shre...
Deletion of data is achieved by permanently discarding the encryption key which is stored and managed elsewhere where secure erasure can be guaranteed.
If implemented honestly, this procedure WORKS and cloud storage is secure. Yes the emphasis is on the "implemented honestly" part but do not generalize cloud storage as inherently insecure.
> This chat won't appear in history, use or update ChatGPT's memory, or be used to train our models. For safety purposes, we may keep a copy of this chat for up to 30 days.
The "may keep a copy" is doing a lot of work in that sentence.
The government also says thank you for your ChatGPT logs.
This makes no sense to me. Shouldn't we address the damage before it's done vs handwringing after the fact?
I didn't think any of the ongoing "fair use" lawsuits had reached a conclusion on that.
The purpose of training in many of the AI Labs being sued mostly matches the conditions that Ross Intelligence was found to have violated, and the question of copying is almost guaranteed if they trained on it.
[1] Thomson Reuters Enterprise Centre GmbH et al v. ROSS Intelligence Inc. https://www.ded.uscourts.gov/sites/ded/files/opinions/20-613...
We'll see if the courts deem it legal but it's, without a doubt, unehtical.
Like we could be upset when that credit checking company dumped all those social security numbers on the net and had to pay the first 200k claimants a grand total of $21 for their trouble?
By that point it’s far too late.
Does not give them permission. What if LEO asks for the data? Should they hand it over just because they have it? Remember, this happens all the time with metadata from other companies (phone carriers for example). Having the data means it's possible to use it for other purposes as opposed to not possible. There is always pressure to do so both from within and outside a company.
Not unless LEO sues OpenAI while it's preserving data from the first discovery, otherwise they cannot be compelled to give up data. Nor are they allowed to violate their TOS and use the data outside of retention, despite the FUD you want to spread about it.
> Having the data means it's possible
No, it doesn't. That's not how any of this works.
In short: OpenAI's business practices caused this. They wouldn't have been sued if using legal data. They might still not have an order like this if more open about their training, like Allen Institute.
The question is whether AI itself is aware what the source is. It certainly knows the source.
Feel like they would still be great for a lot of applications like "Search my local hard drive for the file that matches this description"
When I followed up with how to save chat information for future use in the LLM's context window, I was given a rather lengthy process involving setting up an SQL database, writing some Python tp create a "pre-prompt injection wrapper"....
That's cool and all, but wishing there was something a little more "out of the box" that did this sort of thing for the "rest of us". GPT did mention Tome, LM Studio, a few others....
I did play with MCP and ollama a little, but that was scripted, not interactive.
PLEASE NOTE: We do not engage in "profiling" or otherwise engage in automated processing of Personal Data to make decisions that could have legal or similarly significant impact on you or others.
Hopefully they don't though.
To your observation, it's certainly relevant to the situation at hand but has little to do with your original supposition. A US court can order any company that operates in the US to do anything within the bounds of US law, in the same way that an EU court can do the converse. Such an order might well make it impossible to legally do business in one or the other jurisdiction.
If OpenAI Ireland is a subsidiary it will be interesting to see to what extent the court order applies (or doesn't apply) to it. I wonder if it actually operates servers locally or if it's just a frontend that sends all your queries over to a US based backend.
People elsewhere in this comment section observed that the GDPR has a blanket carve out for things that are legally required. Seeing as compliance with a court order is legally required there is likely no issue regardless.
> What happens when you delete a chat?
> The chat is immediately removed from your chat history view.
> It is scheduled for permanent deletion from OpenAI's systems within 30 days, unless:
> It has already been de-identified and disassociated from your account, or
> OpenAI must retain it for security or legal obligations.
That final clause now voids the entire section. All chats are preserved for "legal obligations".
I regret all the personal conversations I've had with AI now. It's very enticing when you need some help / validation on something challenging, but everyone who warned how much of a privacy risk that is has been proven right.
That's why you read the whole thing? It's not exactly a long read. Do you expect them to update their docs every time they get a subpoena request?
Many things in life are "misleading" when your context window is less than 32 words[1], or can't bother to read that far.
[1] number of words required to get you to "unless", which should hopefully tip you off that not everything gets deleted.
It's like saying "we will delete your chats, unless the sun rises tomorrow". At that point, just say that the chats aren't deleted.
(The snark from your replies seems unnecessary as well.)
That's one giant cop-out.
All you had to do was delete the user_id column and you can keep the chat indefinitely.
> That risk extended to users of ChatGPT Free, Plus, and Pro, as well as users of OpenAI’s application programming interface (API), OpenAI said.
This seems very bad for their business.
Really, it’s funny watching from the outside and waiting for English to finally stop holding it in and get itself some sort of spelling reform to meaningfully move in a phonetic direction. My amateur impression, though, is that mandatory secondary education has made “correct” spelling such a strong social marker that everybody (not just English-speaking countries) is essentially stuck with whatever they have at the moment. In which case, my condolences to English speakers, your history really did work out in an unfortunate way.
That said, phonetic spelling reform would of course privilege the phonemes as spoken by whoever happens to be most powerful or prestigious at the time (after all, the only way it could possibly stick is if it's pushed by the sufficiently powerful), and would itself fall out of date eventually anyway.
Isn't the "a" in "have" elided along with the "h?"
Shouldn't've Should not have
What am I missing?
Also I have always liked this humorous plan for spelling reform: https://guidetogrammar.org/grammar/twain.htm
English popularity was solely and exclusively driven by its use as a lingua franca. As times change, so too will the language we speak.
There are certainly languages with even more spoken complexity - e.g. 4+ consonant clusters like "vzdr" typical of Slavic - but even so spoken English is not that easy to learn to understand, and very hard to learn to speak without a noticeable accent.
So, its something like:
It becomes quite useful in the later sentences as more and more reformations are applied.A phonetic respelling would destroy the languages, because there are too many dialects without matching pronunciations. Though rendering historical texts illegible, a phonemic approach would work: https://en.wiktionary.org/wiki/Appendix:English_pronunciatio... But that would still mean most speakers have 2-3 ways of spelling various vowels. There are some further problems with a phonemic approach: https://alexalejandre.com/notes/phonetic-vs-phonemic-spellin...
Here's an example of a phonemic orthography, which is somewhat readable (to me) but illustrates how many diacritics you'd need. And it still spells the vowel in "ask" or "lot" with the same ä! https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....
Not only that, but since pronunciation tends to diverge over time, it will create a never-ending spelling-pronunciation drift where the same words won't be pronounced the same in, e.g. 100-200 years, which will result in future generations effectively losing easy access to the prior knowledge.
Once you switch to a phonetic respelling this is no longer a frequent problem. It does not happen, or at least happens very rarely with existing phonetic languages such as Turkish.
In the rare event that the pronunciation of a sound changes in time, the spelling doesn't have to change. You just pronounce the same letter differently.
If it's more than one sound, well, then you have a problem. But it happens in today's non-phonetic English as well (such as "gost" -> "ghost", or more recently "popped corn" -> "popcorn").
Oh, but it does. It's just the standard is held as the official form of the language and dialects are killed off through standardized education etc. To do this in English would e.g. force all Australians, Englishmen etc. to speak like an American (when in the UK different cities and social classes have quite divergent usage!) This clearly would not work and would cause the system to break apart. English exhibits very minor diaglossia, as if all Turkic peoples used the same archaic spelling but pronounced it their own ways, e.g. tāg, kök, quruq, yultur etc. which Turks would pronounce as dāg, gök, yıldız etc. but other Turks today say gurt for kurt, isderik, giderim okula... You just say they're "wrong" because the government chose a standard and (Turkic people's outside of Turkey weren't forced to use it.)
As a native English speaker, I'm not even sure how to pronounce "either" (how it should be done in my dialect) and seemingly randomly reduce sounds. We'd have to change a lot of things before being able to agree on a single right version and slowly making everyone speak like that.
Nor is it some kind of insurmountable barrier to communication. For example, Serbian, Croatian, and Bosnian are all idiolects of the same language with some differences in phonemes (like i/e/ije) and the corresponding differences in standard orthographies, but it doesn't preclude speakers from understanding each other's written language anymore so than it precludes them from understanding each other's spoken language.
are based on the exact same Štokavian dialect, ignoring Kajkavian, Čajkavian, Čakavian and Torlakian dialects. There is _no_ difference in standard orthography, because yat reflexes have nothing to do with national boundaries. Plenty of Serbs speak Ijekavian, for example. Here is a dialect map: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fc...
Your example is literally arguing that Australian English should have the same _phonetic_ orthography, even. But Australian English must have the same orthography or else Australia will no longer speak English in 2-3 generations. The difference between Australian and American English is far larger than between modern varieties of naš jezik. Australians code switches talking to foreigners while Serbs and Croats do not.
But there is, though, e.g. "dolijevati" vs "dolivati". And sure, standard Serbian/Montenegrin allows the former as well, but the latter is not valid in standard Croatian orthography AFAIK. That this doesn't map neatly to national borders is irrelevant.
If Australian English is so drastically different that Australians "won't speak English in 2-3 generations" if their orthography is changed to reflect how they speak, that would indicate that their current orthography is highly divergent from the actual spoken language, which is a problem in its own right. But I don't believe that this is correct - Australian English content (even for domestic consumption, thus no code switching) is still very much accessible to British and American English speakers, so any orthography that would reflect the phonological differences would be just as accessible.
> current orthography is highly divergent from the actual spoken language, which is a problem in its own right
The orthography is no more divergent to an Australians speech as to an American's speech, let alone a Londoner or Oxfordian. But why would it be a problem?
Sorry, I didn't mean that it would be a smooth transition. It might even be impossible. What I wrote above is (paraphrasing myself) "Once you switch to a phonetic respelling [...] pronunciation [will not] tend to diverge over time [that much]". "Once you switch" is the key.
> To do this in English would e.g. force all Australians, Englishmen etc. to speak like an American
Why? There is nothing that prevents Australians from spelling some words differently (as we currently do, e.g. colour vs color, or tyre vs tire).
Consider three English words that have survived over the multiple centuries and their respective pronunciation in Old English (OE), Middle English around the vowel shift (MidE) and modern English, using the IPA: «knight», «through» and «daughter»:
It is not possible for a modern English speaker to collate [knixt] and [naɪt], [θurx] and [θruː], [ˈdoxtor] and [ˈdɔːtə] as the same word in each case.Regular re-spelling results in a loss of the linguistic continuity, and particularly so over a span of a few or more centuries.
There are occasional mixed horrors like "ptarmigan", which is a Gaelic word which was Romanized using Greek phonology, so it has the same silent p as "pterodactyl".
There's no academy of the English language anyway, so there's nobody to make such a change. And as others have said, the accent variation is pretty huge.
Just because training on data is opt out doesn't mean business can't trust it. Not the best for user's privacy though.
Like there's the algorithm by which a hedge fund is doing algorithmic trading, they'd be insane to take the risk. Then there's the code for a video game, it's proprietary, but competitors don't benefit substantially from an illicit copy. You ship the compiled artifacts to everyone, so the logic isn't that secret. Copies of the similar source code have linked before with no significant effects.
Many algo strategies are indeed programmatically simple (e.g. use some sort of moving average), but the parametrization and how it's used is the secret sauce and you don't want that information to leak. They might be tuned to exploit a certain market behavior, and you want to keep this secret since other people targeting this same behavior will make your edge go away. The edge can be something purely statistical or it can be a specific timing window that you found, etc.
It's a bit like saying that a Formula 1 engine is not that far from what you'd find in a textbook. While it's true that it shares a lot of properties with a generic ICE, the edge comes from a lot of proprietary research that teams treat as secret and definitely don't want competitors to find out.
A lot of the use is fairly mundane and basically replaces junior analysts. E.g. it's digesting and summarizing the insane amounts of research that is produced. I could ask an intern to summarize the analysis on platinum prices over the last week, and it'll take them a day. Alternatively, I can feed in all the analysis that banks produce to an LLM and have it done immediately. The data fed in is not a trade secret really, and neither is the output. What I do with the results is where the interesting things happen.
And wrapper-around-ChatGPT startups should double-check their privacy policies, that all the "you have no privacy" language is in place.
If a court orders you to preserve user data, could you be held liable for preserving user data? Regardless of your privacy policy.
This, however, is horrible for AI regardless of whether or not you can sue.
A court ordering you to stop selling pigeons doesn't mean you can keep your store for pigeons open and pocket the money without delivering pigeons.
> Legal Requirements: If required to do so by law or in the good faith belief that such action is necessary to (i) comply with a legal obligation, including to meet national security or law enforcement requirements, (ii) protect and defend our rights or property, (iii) prevent fraud, (iv) act in urgent circumstances to protect the personal safety of users of the Services, or the public, or (v) protect against legal liability.
- They are legally obligated to divulge that data to the government
- Once they do so, they are civilly liable for breach of contract, as they have committed to never divulging this data. This may trigger additional breaches of contract, as others may have not had the right to share data with a company that can share it with third parties
Your users aren't obligated to know that you're using open ai or other provider.
No, because you turn up to court and show the court order.
It's possible a subsequent case could get the first order overturned, but you can't be held liable for good faith efforts to comply with court orders.
However, if you're operating internationally, then suddenly it's possible that you may be issued competing court orders both of which are "valid". This is the CLOUD Act problem. In which case the only winning move becomes not to play.
For example, if there are business consequences for leaking customer data, you better run that LLM yourself.
https://en.wikipedia.org/wiki/Mistral_AI
Technically you could probably just run it on EC2, but then you’d still need HIPPA compliance
I'm quite uninterested in your 'advice' which comes from a place of ignorance regarding my relationship with the project, the company and the people.
There are multinational corporations with heavy presence in Europe, that run their whole business on Microsoft cloud, including keeping and processing there privacy-sensitive data, business-critical data and medical data, and yes, that includes using some of this data with LLMs - hosted on Azure. Companies of this size cannot ignore regulatory compliance and hope no one notices. This only works because MS figured out how to keep it compliant.
Point being, if there are business consequences, you'll be better off using Azure-hosted LLMs than running a local model yourself - they're just better than you or me at this. The only question is, whether you can afford it.
sure, theoretically kyc and aml are good, the implementations all suck.
Microsoft v. United States (https://en.wikipedia.org/wiki/Microsoft_Corp._v._United_Stat...) showed the government wants, and was willing to do whatever required, access to data held in the E.U. The passing of the CLOUD Act (https://en.wikipedia.org/wiki/CLOUD_Act) basically codified it in to law.
Of course, it could be just all talk, like all general European globalist talks, and Europe will do a 360 once a more friendly party takes over the US.
Just hyperbole, but it seems the regulations are designed with the big cloud providers in mind, but then why don't they just ban US big tech and roll out the regulations more slowly? This neoliberalism makes everything so unnecessarily complicated.
If Solows paradox is true and not the result of bad measurement, then one might expect that it could be workable without sacrificing much productivity. Certainly abandoning the cloud would be possible if the regulatory environment allowed for rapid development of alternative non-cloud solutions, as I really don't think the cloud improved productivity (besides for software developers in certain cases) and is more of a rent seeking mechanism (hot take on hacker news I'm sure, but look at any big corpo IT dept outside the tech industry and I think you will see tons of instances where modern tech like the cloud is causing more problems than it's worth productivity-wise).
Computers in general I am much less sure of and lean towards mismeasurement hypothesis. I suspect any "return to 1950" project would render a company economically less competitive (except in certain high end items) and so the EU would really need to lean on Linux hard and invest massively in domestic hardware (not a small task as the US is finding out) in order to escape the clutches of the US and/or China.
I don't think they have the political will to do it, but I would love it if they tried and proved naysayers wrong.
The European Court of Justice ruled at least twice that it doesn't matter what kind of contract they give you, and what kind of bilateral agreement there are between the US and the EU, as long as the US have the patriot act and later regulations, using Microsoft means it's violating European privacy laws.
Yes, and that's why the European Commission keeps being pushed back by the Court of Justice of the EU (the Safe Harbor was ruled out, Privacy Shield as well, and it's likely a matter of time before the CJEU kills the Data Privacy Framework as well), but when it takes 3-4 years to get a ruling and then the Commission can just make a new (illegal) framework that will last for a couple years, the violation can carry on indefinitely.
This is learned helplessness and it’s only true if you don’t put any effort into building that expertise.
You're right, I should get right to it. Plenty of time for it after work, especially if I cut down HN time.
But at least if you use an API in the same region as your customers, court order shenanigans won't get you caught between different jurisdictions.
For most tasks I don't need the best model in existence, I just need good enough. A small law firm using LLMs for summaries can probably do it on prem and hire a smart college student to setup a PC to do it.
The problem is that's still more difficult ( let's say our hypothetical junior IT only makes 60k a year) than just sending all your private business information to some 3rd party API. You can then act shocked and concerned when your 3rd party leaks the data.
Point 1: Every company has profit incentive to sell the data in the current political climate, all they need is a sneaky way to access it without getting caught. That includes the combo of LLM provider and Escrow non-entity.
Point 2: No company has profit incentive to defend user privacy, or even the privacy of other businesses. So who could run the Escrow service? Another business? Then they have incentive to cheat and help the LLM provider access the data anyway. The government (and which one)? Their intelligence arms want the data just as much as any company does so you're back to square one again.
"Knowledge is power" combined with "Knowledge can be copied without anyone knowing" means that there aren't any currencies presently powerful enough to convince any other entity to keep your secrets for you.
But, since, I think, there are mechanisms by which they could keep logs, but in a way they cannot access them, they could still claim you will have privacy this way - even though they have the option to keep un-encrypted log, much like they could retain the logs in the first place. So the messaging may remain pretty much the same - from "we promise to delete your logs and keep no other copies, trust us" to "we promise to 3p-encrypt your archived logs and keep no other copies, trust us".
> No company has profit incentive to defend user privacy, or even the privacy of other businesses.
> They have incentive to cheat and help the LLM provider access the data anyway
Why would a company whose role is that of a 3p escrow be incentivised to risk their reputation by doing this? If that's the case every company holding PII has the same problem.
> Their intelligence arms want the data
In the EU at least, GDPR or similar. If you explicit law breaking, that's a more general problem. But what company has a "intelligence arms" in this manner? Are you talking about another big-tech corp?
I'd say this type of cheating it's be a risky proposition from the POV from that 3pe - it'd destroy their business, and they'd be penalised heavily b/c sharing keys is pretty explicitly illegal - any company caught could maybe reduce their own punishment by providing the keys as evidence of the 3pe crime. A viable 3pe business would also need multiple client companies to be viable, so you'd need all of them to play ball - a single whistle-blower in any of them will get you caught, and again, all they need is a single key to prove your guilt.
> "Knowledge is power" combined with "Knowledge can be copied without anyone knowing" means that there aren't any currencies presently powerful enough to convince any other entity to keep your secrets for you.
On that same basis, large banks could cheat the stock market; but there is regulation in place to address that somewhat.
Maybe 3p-escrows should be regulated more, or required to register as a currently-regulated type. That said, if you want to protect data from the government, prism etc, you're SOOL, no one can stop them cheating. let's focus on big-/tech/-startup cheats.
You> But what company has a "intelligence arms" in this manner? Are you talking about another big-tech corp?
"Their" in this circumstance refers to any government that might try to back Escrow.
Then I refer to my comment on prism: "if you want to protect data from the government, prism etc, you're SOOL, no one can stop them cheating. let's focus on big-/tech/-startup cheats."
Though you talk about "backing" escrow, I mean regulating. The government otherwise controls all business and society. How is it any different to the banks, sec companies etc in that respect.
I've reviewed a lot of SaaS contracts over the years.
Nearly all of them have clauses that allow the vendor to do whatever they have to if ordered to by the government. That doesn't make it okay, but it means OpenAI customers probably don't have a legal argument, only a philosophical argument.
Same goes for privacy policies. Nearly every privacy policy has a carve out for things they're ordered to do by the government.
They can still have legal contracts with other companies, that stipulate that they don't train on any of their data.
Do search engine companies have this requirement as well? I remember back in the old days deanonymizing “anonymous” query logs was interesting. I can’t imagine there’s any secrecy left today.
There's a qualitative difference resulting from quantitatively much easier access (querying some database vs. having to physically look through court records) and processing capabilities (an army of lawyers reading millions of pages vs. anyone, via an LLM) that doesn't seem to be accounted for.
Of course I haven’t sent anything actually sensitive to ChatGTP, but the use of copyright law in order to enforce a stricter surveillance regime is giving very strong “Right to Read” vibes.
> each book had a copyright monitor that reported when and where it was read, and by whom, to Central Licensing. (They used this information to catch reading pirates, but also to sell personal interest profiles to retailers.)
> It didn’t matter whether you did anything harmful—the offense was making it hard for the administrators to check on you. They assumed this meant you were doing something else forbidden, and they did not need to know what it was.
=> https://www.gnu.org/philosophy/right-to-read.en.html
It's one thing if you get pwned because a hacker broke into your servers. It is another thing if you get pwned because a hacker broken into somebody else's servers.
At this point, do we believe OpenAI has a strong security infrastructure? Given the court order, it doesn't seem possible for them to have sufficient security for practical purposes. Your data might be encrypted at rest, but who has the keys? When you're buying secure instances, you don't want the provider to have your keys...
Look at it this way. If you your phone was stolen would you want it to self destruct or keep everything? (Assume you can decide to self destruct it) clearly the latter is safer. Maybe the data has been pulled off and you're already pwned. But by deleting, if they didn't get the data they now won't be able to.
You just don't want to give adversaries infinite time to pwn you
Well, it is gonna be all _AI Companies_ very soon so unless everyone switches to local models which don't really have the same degree of profitability as a SaaS, its probably not going to kill a company to have less user privacy because tbh people are used to not having privacy these days on the internet.
It certainly will kill off the few companies/people trusting them with closed source code or security related stuff but you really should not outsource that anywhere.
For now. This is going to devolve into either "openAI has to do this, so you do too" or "we shouldn't have to do this because nobody else does!" and my money is not on the latter outcome.
> —after news organizations suing over copyright claims accused the AI company of destroying evidence.
Like, none of the AI companies are going to avoid copyright related lawsuits long term until things are settled law.
See the whole LIBOR chat business.
And how many companies have proprietary code hosted on Github?
We've always done self-hosted as old as things like gerrit and what not that aren't even really feature complete as competitors where I've worked.
They have a fair bit. Local models lets companies sell you a much more expensive bit of hardware. Once Apple gets their stuff together it could end up being a genius move to go all in on local after the others have repeated scandals of leaking user data.
As far as I understand it, this ruling does not apply to Microsoft, does it?
https://x.com/paulg/status/1913338841068404903
"It's a very exciting time in tech right now. If you're a first-rate programmer, there are a huge number of other places you can go work rather than at the company building the infrastructure of the police state."
---
So, courts order the preservation of AI logs, and government orders the building of a massive database. You do the math. This is such an annoying time to be alive in America, to say the least. PG needs to start blogging again about what's going on now days. We might be entering the digital version of the 60s, if we're lucky. Get local, get private, get secure, fight back.
This raises serious questions about the supposed "anonymization" of chat data used for training their new models, i.e. when users leave the "improve model for all users" toggle enabled in the settings (which is the default even for paying users). So, indeed, very bad for the current business model which appears to rely on present users (voluntarily) "feeding the machine" to improve it.
[0] https://cdn.arstechnica.net/wp-content/uploads/2025/06/NYT-v...
So, the NYT asked for this back in January and the court said no, but asked OpenAI if there was a way to accomplish the preservation goal in a privacy-preserving manner. OpenAI refused to engage for 5 f’ing months. The court said “fine, the NYT gets what they originally asked for”.
Nice job guys.
How is it with using openrouter?
If I have users that use OpenAI through my API keys am I responsible?
I have so many questions…
Yes. You are OpenAI's customer, and they expect you to follow their ToS. They do provide a moderation API to reject inappropriate prompts, though.
. . .
Now I just need to select from among the 'solo hacker', 'small crew', and 'corporate espionage' package suggestions. Price goes up fast, though.
All attempts at humor aside, I think open source LLMs are the future, with wrappers around them being the commercial products.
P.S. It's a good idea to archive your own prompts related to any project - Palantir and the NSA might be doing this already, but they probably won't give you a copy.
(Even has an OG motorcycle avatar, ha.)
At least it wasn't a link to a screenshot.
To my knowledge, the court is forcing the company to change its policy. The obligation isn’t broken, its terms were just changed on a going-forward basis. (Would be different if the court required preserving records predating the order.)
apple I trust as much as I trust politicians
sent from my iphone :)
I thought the entire game these guys are playing is rushing to market to collect more data to diversify their supply chain from the stolen data they've used to train their current model. Sure, certain enterprise use cases might have different legal requirements, but certainly the core product and the average "import openai"-enjoyer.
Becuase they are bound by their terms of service? Because if they won't no business would ever use their service and without businesses using their service they won't have any revenue?
Not your computer, or not your encryption keys, not your data.