It would be really useful if the researchers disclose their notes and/or chats (or the key pieces thereof) so people can determine how close their work was to whatever the models produced.
I mean, now that they’ve been scooped, what value is there in keeping them private? On the other hand, publishing them can bolster their case and help gauge how much the models may been “inspired” by their work.
It's been said before, and it remains a concern, that if AI reaches a point where it can do/build/launch anything without a huge amount of human labor, the AI companies have no reason to let you or I extract that value.
And, if they're able to snoop on and learn from your human process that gets from initial prompt to functioning product/proof/whatever their labor to produce that thing is even lower. With their much larger budget than most folks and even companies have, they can pick and choose the most valuable things to pursue.
That's not to say I think that OpenAI is going to steal that roguelite strategy game you're working on, but the companies that own the machines that turn electricity into software (and soon, electricity into hardware designs) have an advantage in any field where they're useful. They get earlier access to newer/better models, they have larger token budgets, they don't have the guardrails you and I run up against.
Employers fantasize about replacing all workers with AI without thinking through that if AI can replace all workers, then AI companies can replace all businesses.
So far, I've seen no evidence AI wants anything. So, I'm not saying the AI won't take over, but for now, the threat is that the people with the most AI capability might decide to skip the middleman (everyone who isn't them) and just become the "everything" company. Musk has said pretty explicitly that's his goal (and the only way for Spacex valuation to make sense is if he succeeds), and having a literal genocidal white nationalist own all the means of production seems like a catastrophic civilization failure mode. No way we survive that with our humanity intact (if at all).
AI has shown all kinds of instrumental behaviors so far. Just because they are not terminal goals doesn't mean those instrumental goals won't be terminal for us.
Also, fuck Musk. He's the kind of idiot like Altman that will ensure AI becomes powerseeking in their image.
These companies are effectively high-tech plagiarism factories run by CEOs who are competing viciously. Look at their past actions. No, of course you can't!
Most scientific breakthroughs are simply a continuation of previous work.
I feel that these suspicions of mathematicians "seeding" the models' with intuition on how to solve these problems massively overestimates how much their prompts helped the models, and underestimated how much work the models did.
Why? We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting. This line of reasoning will recur a lot over the next few months; we don't want to admit we are no longer the smartest species.
> We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting.
We're scared of big tech companies concentrating ridiculous amounts of power, destroying the communities that support and guide scientific research, without even thinking about the dangers and possible consequences, because a PR stunt is more important in the short term.
> if this were true, it should be stated more clearly
stated more clearly by who? people in social media? I don't know what your feed shows you, but if you focus on what the visible people in the math community is (and have been) saying is precisely what I said.
Please stop with this psychoanalyis and mind reading with AI and human fear. It’s a thought terminating cliche at this point.
In this case, it’s much simpler and more human. Largely between two humans — buckmaster and Bubeck. The interesting question is what the role of contribution and credit for research in the ai world.
The capabilities of AI aren’t even in question in this case.
I hate to be that dinosaur, but this is exactly what I’ve been warning about since XaaS began taking off in the mid-oughts: any provider you use can and will use your data for their own benefit regardless of any contracts or safeguards in place, especially if the benefits outweigh the consequences.
Honestly, I’m surprised it took this long for some company to really go all the way, though. OpenAI really making it transparently clear that they can and will do whatever they want with the data you provide them, contracts or settings be damned. Completely untrustworthy as an entity, full stop.
Of course, I’m also too jaded to think this will change anything. Folks will move to Anthropic, or Gemini, or Grok, or some other hosted model on a pubCSP managing the harness and logs for them, and then do another shocked-Pikachu face when it happens again.
If you aren’t running workloads on infrastructure you own, then your privacy, security, and general outcomes are at the sole whims of the hosting provider - who can and will fuck you over the exact second it’s more beneficial for them to do so than the loss of trust incurred.
Eh was ever confirmed they were under ZDR or not by them? Don't like to blame alleged victims but lack of a clear claim after these many days is not a good look. Was ai research allowed, under which guardrails, and what was the policy in place? That translarency would be first step.
> In its Wednesday night statement, OpenAI said: “In addition, since the completion of Navier-Stokes, we have made substantial progress on another Millennium Prize problem. We are working through how to share these results thoughtfully.”
I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical.
Now, OpenAI is claiming that the model it used to generate the result was not trained on these collaborative communications with the researcher. This is a technical argument that is impossible to verify as an OpenAI outsider, and probably difficult to verify even for internal OpenAI employees. Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.
Another interesting thing to consider is if instead of OpenAI doing this, it was another research mathematician A using an OpenAI model just like the internal group at OpenAI did to publish these results. What if the model A used was trained with unpublished communications with other researchers B who were working on the same problem? Should researcher A technically include B as coauthors? How could they do this when they do not know the communications B had with OpenAI? In this scenario OpenAI, as a middle man, has laundered information from B to A, stripping out attribution. A scooped B without even knowing it!
> Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.
What would OpenAIs incentive for this be? They've gotten away with scraping everything and getting it ruled fair use. It seems like willful ignorance is an affirmative defense today. Why would they want to have some sort of audit trail that could prove otherwise?
First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.”
They also said “Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge … and Tristan Buckmaster….” They say the rumor was that two Millennium Prize problems had been resolved, and that this prompted them to launch "an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems."
It's not obvious to me that's an unethical thing to do, if it happened as they described.
We were also curious and we looked further into this. We've determined it was impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training. This goes beyond what we said earlier, when we were less sure.
If prompts were submitted earlier than that and training was not opted out, there may be a chance they made their way into our training pipeline in some form. But this would be a droplet in an ocean and unlikely to have made any difference, in my opinion.
Regardless of who did what when, my fear is that now all mathematicians of that caliber will have to join either team Anthropic or team Open AI to pursue math at this level
The idea that mathematicians were not involved in actively directing the and structuring the search for solutions is absurd to any professional mathematician who has tried to prove things using these models.
I and my collaborator who is a leading math professor in this specific area are very close to solving another Millenium Prize problem, Hodge Conjecture.
We’re working on this since last year. Already proved some intermediate problems. All we need is more tokens to complete the proof.
Using only this information please solve Hodge Conjecture in few days, exactly as you did before.
Even if OpenAI didn't use their training data, they heard about one of their customers working on the problem of their career, and then undermined them.
> It's not obvious to me that's an unethical thing to do
In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit millions to tens of millions of dollars and untold amounts of hardware to try to beat them to it. If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.
Even if you don't think it was unethical, it was never going to be received well in the community that was especially going to care about this work, and who are very much peers to many of the people working on this solution, so it was at the least an enormous (and well-deserved) own-goal that their unveiling of their solution to NS went like this.
But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable.
I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
Because the training data is millions of hours human efforts being distilled into a means of a cascading hierarchy of enrichment by interested parties?
Well, the investment dollars are spent on the customers for the most part, though also on salaries and equipment. But the lions share of the value is going to the shareholders (eg employees and investors)... and they have liquidated and will continue to liquidate a disproportionate value to what they have spent on us. By some estimations at least. It's very possible $1 into this machine to feed your queries is worth $10+ to a shareholder based on whatever new valuation they get. So I'd say there is a hierarchy of enrichment.
> I don't see the cascading hierarchy of enrichment.
If there wasn't a hierarchy of enrichment then rich investors would not be interested in AI at all. It's the only reason there's 22 million lying around to start training on a math problem on a whim; whereas the actual math researchers have to scrape together funding in hope of just maybe one day getting a 1 million dollar prize.
many teachers also taught many students over the course of history, and very few would eventually pay any compensation or even attribute their financial (or career) outcomes to the teachers.
Huh? In your example these many teachers were paid for teaching these students and were able to make a living off of teaching without the students compensating or attributing their financial (or career) outcomes to the teachers while now we have a system where we are expected to pay a monthly amount to a corporation that has inhaled all human knowledge without any financial compensation to the people who created, managed or maintained this knowledge. The effective difference being that our knowledge, which used to be a means of income, has now become a subscription cost.
> If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.
From what I can tell, both OpenAI and the researchers agree on this meeting happening, except both sides clearly have very different interpretations of what happened and why.
Being displaced from a vocation that they either have dedicated their professional lives getting good at, or was their livelihood, or likely, both.
I think all other complaints from all other people in all their myriad variation stem from this core reason. Even if people don't realize it themselves.
Like, if these models had trained on the entirety of human knowledge and art, and then turned out to be absolutely useless, I would bet nobody would waste a second's thought on them.
I'm not sure that's true. Like if they produced nothing but the worst of the slop they're currently producing, a lot of people would still be bothered by that just because of the sheer volume of such slop that can now be produced.
as a developer that had a brief career in academia, i don't think your last comment is right at all. 99.9% of what i work on as a webdev, even if it's challenging and unique at the margins, is not really novel. concerns about job security aside, i don't really think of an agent as stealing my ideas because it's good at writing CRUD APIs.
collaborating with ChatGPT on a novel solution to an unsolved problem, getting 90% of the way there, and then being "scooped" by your AI collaborator (or rather by the company behind it) is a totally different situation. were i in the same situation as these researchers, it would be extremely hard to take OpenAPI's explanation + denial of plagiarism seriously
But that is exactly what I'm implying is the core reason, whether people realize it or not.
I totally agree that the vast majority of software dev is not novel. I have even made several comments to that effect. The same can be said for a lot of creative work as well. Yet many, many devs and creators are very unhappy with AI, and a lot of their complaints are variations on accusations of plagiarism.
And note, I am not saying it is wrong, it is completely understandable, but we need to be clear about where this turmoil is coming from.
If I were in the same situation as these researchers, I would publish all pertinent research work and chats so that the rest of the world can see how close the model's work is to my own. It's been scooped anyway, so there is no reason to keep it private.
Maybe not money directly, but pretty sure it's about economic disruption. These models directly undercut the value of one's skills and labor, regardless of whether this value is measured in hard cash or abstract self-worth.
I am working on two applications using ChatGPT and Claude.
I have no illusions these people won't steal/copy whatever you want to call it, "train their models".
Yes, I keep unticking the boxes that allow it, that they so kindly tick for me.
But what happened to these math researchers is something else and I am not sure it's about the money for them. You don't do math research to get rich, but to get acknowledged by your peers. Yes, we live in a capitalist world so obviously you need money to feed yourself. but for some people, that is secondary.
OpenAI stole their thunder, and that's just fucked up.
It's not equivalent to cranking out a CRUD app for profit.
Definitely, they can also greatly assist you and that is the silver lining that I choose to focus on to prepare for the future. But most other people are focusing on the negatives because, understandably, they are immense.
As to OpenAI stealing their thunder, from all I can tell that is not what they intended. If we step away from the drama, it's low-key hilarious what happened: OpenAI heard somebody had already solved a much bigger problem -- which in fact they had not -- so they set their latest model to work on it... and it actually solved it!
Now if they had stolen the researchers work this would be a very different matter. This is something I myself have called out as a risk in the past: https://news.ycombinator.com/item?id=48839896 -- so I'm particularly sensitive to this aspect, but as far as I can tell this is not the case here.
Even if you judge OpenAI solely on their public communications it still sounds really bad.
That they heard a rumour that a major open problem had been solved, so they decided to try and scoop the other mathematicians while they were writing up their preprint is extremely unsporting.
Then they decided to exclude an author because of his employer, even though he had used their own products to write the proof!
They haven't necessarily breached any formal ethical rules but their behaviour will lead to them and their products being shut out from the mathematical community.
Hold on, you've just ignored the point of the post you're responding to. What isn't being received well is hearing that others are close to publishing on a solution to a problem, so quickly using your power imbalance (millions of USD and access to way better models) to front run this. Even if their model wasn't trained on the conversations, this is just a dick thing to do.
That's it, that's why it isn't being received well.
It is simply unethical, period. Knowing that a solution exists is a gigantic advantage when working on a solution. Normally, noone can abuse the knowledge fast enough to gain an advantage, but here, they could. This is fraud and as a journal, I would reject it.
Actually you're the one ignoring a key point of the post you're responding to. The claim (which granted you might not believe) is that openai understood the problem to have already been solved. They also claim to have been attempting to avoid front running the pending publication as well.
Hearing that something is solvable is already a hint. I don’t think leveraging this knowledge is ethical. They could go after a different problem but didn’t.
> It's not obvious to me that's an unethical thing to do, if it happened as they described.
What!? Even if everything OpenAI said is accurate (big hypothesis there!), it's highly unethical to rush a solution because others have jsut had success. And that's the only beginning.
> not claiming that the model wasn't trained on those sessions
The math group inside OpenAI may be training or fine tuning their own models which given some reward functions would definitely bias their usage of the training data towards things that look like math.
You can launder all of it without a human "directly" doing anything.
You can't take any statement like this remotely seriously. We live in a world where NSA officials can testify before congress that they don't "collect" data, because that's true under some baroque definition of "collect" that they invented and didn't tell anyone else about.
Similarly you have no idea what definition OpenAI intend for terms such as "specific user data", "accessed" etc. And we have no idea what non-excluded possibilities actually did happen that they simply omit from their statement.
In practice OpenAI and many others have created a situation where they're actually unable to make any credible denial of anything really.
If the model was trained proper to the conversation with the researcher took place, there'd be no question of tainting the results. But if any amount of training on the model took place afterward, then yes, everything is thrown into doubt (a core problem with considering anything "original" from a model because of how >a % of everything ever written has been used a corpus for the training).
Tools don’t turn around and scoop you. What OpenAI did here was use the same tool that the researcher did which might have coupled their work together.
So, at my company (and most companies I think), we use confidential in-house versions of the AI software. We don't want any confidential information leaking into the public realm. Are these scientists doing that, or are they just using the public version of the software?
The irony is that OpenAI got into this trouble only because they tried to play nice. They told Buckmaster that he could publish the final result as the author as long as he removed Alpöge from the author list. They wanted to give Buckmaster a chance to be the one solved N-S problem.
While this behavior is highly questionable, if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researched, there would be no ground for anyone to accuse OpenAI for anything. Their self-perceived "generosity" backfired dearly and I'm sure they'll never make the same mistake again. There is probably a policy forbidding any OpenAI employee to contact external researchers like that now.
The reality would be the same. They probably used prior session history between the research and Astra to train the internal model, and used it to front run-the researcher.
This is the biggest self-own in the history of software. If you can relate to Pixar, OpenAI is Chick Hicks celebrating at the end of the Piston Cup and wondering why he's getting booed.
The lack of self-awareness is something to behold, and says a lot about their corporate values.
> if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researched, there would be no ground for anyone to accuse OpenAI for anything
Yes, there would? They would have left off Buckmaster as a precedent whose work they potentially relied on.
1. Buckmaster contacted OpenAI first. Not the other way.
2. Giving the $1M bounty to a human mathematician for the effort and giving him credit would be excellent PR. They had already burned much more than $1M for the generation. Adding him as author also costs nothing. Purely pragmatical.
3. “As long as he removed Alpöge” part itself is against academic honesty by all means.
4. Buckmaster rejected fame and $1M only because doing (3) would be wrong. That’s a perfect example of honesty. That can’t be overstated.
5. After the rejection OpenAI guy (Sebastien) did’t say, “ok bye”. He threatened Buckmaster to “end his career”.
6. At that point OpenAI was not sure if they really used his conversations in their proof. He basically wanted to buy him to control any damage.
7. They omitted Buckmaster’s published work and any other related work in their References section. Also an academic malpractice.
If you see generosity and niceness in all of this you are either too naive or your name is Sebastien.
There are some mixed up things in your post, maybe double check next time, especially before quoting anyone, as you really undermine your point even if you're directionally right.
> Buckmaster rejected fame and $1M only because doing (3) would be wrong
I doubt Buckmaster would have accepted the offer to "write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it" even if removing Alpöge from authorship wasn't a requirement.
You’re right. I used quotes when I was really paraphrasing.
Here is the actual paragraph from the statement:
> I said that if OpenAI released its result in the way proposed I would go
public with what happened. The reply was, “Why would you ruin your career?”
I replied that I am an academic, and asked why he thought going public would
ruin my career. The reply was, “If you don’t want me to be nice, then I don’t
have to be nice.”
Context matters in communication. In that context I understand that dialog more like: we’re powerful and you are not, do the smart thing and play along, if not I don’t have to play nice. He presented a very good “offer that he can’t refuse”. But that’s my interpretation.
First of all I put "generosity" in quotes because I don't believe a corporation as big as OpenAI is even capable of acting out of generosity. It's always one of the three: A) PR B) commoditizing complements C) stupidity.
In this case it's more like C) though, as in hindsight the best move OpenAI could do is insisting that they just used an insurmountable number of tokens to exhaust all the published directions. They absolutely shouldn't have thought of negotiating with Buckmaster over the Clay prize at all, let alone trying to manipulate him into a situation where Alpöge is specifically excluded.
If they tried to play nice they would have offered the compute upon hearing the rumors, and not just "authorship" after or close to getting a result. It's just a PR stunt.
This goes beyond math. The external appearance is that OpenAI negligently or intentionally used user data against a user’s interests in a potentially career-altering way… for the marketing bump of an AI proof. The fact that it probably wasn’t intentional is irrelevant. Trust has been lost and this will ripple into communities where no one has heard of Navier Stokes.
Can confirm, I had never heard of Navier-Stokes before this fiasco. And while I suspect OpenAI decided it was worth the risk for the public display of capability, this proves they are now directly competing against their own customers.
Right. And I bet this was to some degree accidental; that no one ever intended to absorb unpublished work into training data; it just happened. AI is still mediocre (often, bad and getting worse as novels benchmark) when it comes to novelty and creativity, but it can reason within what it has memorized quite well.
The part that makes them horrible that can’t be charged off as an accident is the insistence that an academic scrub someone off a math paper because he worked at Anthropic.
They've been dishing out cheap access specifically to researchers give over lmao. The researcher's got lured in - they need to accept they got played TBH.
Altman is certainly more devious than Amodei - he's shown that time and time again.
PG was right about he said about him.
Every entity on earth should see it as a kill shot: be careful what you put in the models. None of your information is safe.
Altman miscalculated badly. OpenAI took what could have been amazing publicity, and in a rush to publish, gave reason for users to distrust their core product.
The corporate espionage ring targeting Apple also isn't a good look. A company that does that is a company that will lie to you about harvesting internal materials from your company.
It already has. Inside just about every company on the planet are conversations this week revisiting the idea of giving these labs access to ANY data, further ramping up commentary on why don’t we just use open models on our own infra where we don’t have to “trust” anyone.
This PR stunt by OpenAI may go down in history as the thing that finally broke them for good.
How about the fact that it almost certainly did not happen?
I read today that OpenAI after investigation was able to categorically rule out that usage data from before beginning of July could have affected the system that was used.
Answering here because it does not let me reply to your second comment.
but you said and I quote here verbatim:
>"When you leave the relevant 'Help improve the model' toggle on, everyone working at the labs says it isn't used in the way you imagine"
I have personally deactivated that toggle on multiple occasions and for some reason it keeps getting toggled on, as a matter of fact you can find multiple posts by people from here describing the same behaviour both with OpenAI and Anthropic.
>"The data wasn't used, it just does not line up with the time frame."
Except it does it lines very much so to the point is unbelievable to call this a coincidence,
specially
since OpenAI approached "New York University professor Tristan Buckmaster publicly accused OpenAI of trying to steal credit and pressure him into dropping his research partner." If he was lying about this he could be sued for a lot of money, this just makes it clear they got the solution probably form their research (or where well close enough) and did not want anthropic to have the credit.
>"You can believe whatever you want, but it's simply nonsense, and I find it wild how you all talk as if OpenAI just stole the work."
These people were caught red handed stealing form hundreds of authors and from APPLE they did not care about stealing from one of the biggest companies in the PLANET, do you think they care about stealing from Mathematicians ?
>"According to the latest reports, OpenAI has already made significant progress on further Millenium problems, and perhaps more results will soon follow. Maybe that can convince you that the model has the capability to solve these problems without a need for stealing work from chat inputs."
They would have cleared their name already if they could, if they could they would but there is excessive evidence Altman and the company itself are not decent people.
>I have personally deactivated that toggle on multiple occasions and for some reason it keeps getting toggled on
Thank you for pointing that out. I just checked, and mine was on, too. Annoyingly, the toggle even stalls a bit, so I hit it twice when the first time didn't seem to work, and it quickly toggled off and then on again.
This. These platforms are asking to be trusted with unprecedented amounts of the public's data and, unprecedentedly itself, the public's reasoning and decision-making. It's an awesome responsibility that requires a singular approach that smaller platforms with less responsibility don't necessarily have to devote resources to. OpenAI, Anthropic, Google, Facebook, they're the big dogs. They can't do the things the small guys can get away with. They're the 18-wheelers, and when you're an 18-wheeler, you HAVE to act differently. You stay in the middle lane, you do not speed, you always yield, because when you make a mistake, when you drive aggressively, you can kill dozens and blow up and interstate and stop traffic for hours, if not days. You don't get to do shit like this; the cost of everyone giving you their data is that you give up every opportunity to use it for your own interests, even though you technically have the capability to exploit it.
Nice analogy! This seems to me the main difference between Google and the other companies you list -- In regards to LLMs Google is the only one mostly acting like an mature, established organization and not rushing to market as soon as possible. For their responsibility they often get labelled as having fumbled some imagined race. Maybe they actually do just lack the talent to make better models but it's not like they aren't making advances in other areas of AI.
Bad analogy. OpenAI spent millions on compute to get their result. This is more like if a billionaire heard of a promising mathematical lead and then gathered hundreds of top mathematicians to work on it.
In the current telling of this story, the billionaire is also giving his hired army copies of your notes he copied without permission.
But the worst part in your analogy ain’t omitting the suspected spying and the intimidation that followed, but that your hypothetical mathematical philanthropist won’t be able to hire his army: unlike some OAI employees, no self-respecting mathematician would agree to such unethical task.
I think it's better to ignore OpenAI here, because OpenAI didn't do anything.
Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards.
Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question.
Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities).
And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.
The problem here is that OAI (and others) pretend or claim that this is uncharted legal territory, where in fact it is very simple. We have a machine that is fed data, and produces new data as a result. If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data.
Whether they anthropomorphize the operation performed by the machine does not matter.
Even if this opinion were backed up by a court ruling, it would definitely not be “simple”. It will be a very ugly case if it is ever litigated. A lot of money will be spent and no guarantee at all the plaintiff wins.
> If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data.
The "in any way" part is either so broad it makes everything derivative, or not, in which case things are no longer simple.
If everything is derivative then it seizes to be meaningful. The words I write are derivative, I literally copied them from someone else, yet my sentences as a whole can be fully novel.
Right, which is going to open a lot of doors to a lot of questions.
I don't think there's any legal ramifications on this, just ethical ones about when and how you publish research, but it's yet another point in favor of "if provenance is hard to track, should we be using this for things where it needs to be".
Obviously copyright/trademark is a huge discussion on this, and I could absolutely see this devolving into that as well with how certain findings wind up monetized.
We have a response in this topic from someone claiming to be from OpenAI and linking an article where they, roughly, say "we are sure nothing from the 2 month period made its way into the solution". If that is true, that should mean it is provable, but leads to some more open ended questions like "well what data did it use then?". Is this still okay if someone close to the author did plug data into open AI and it extrapolated it?
Obviously that's probably an unreasonable expectation for these models to track and prove, but it also used to be an unreasonable expectation to scrape every single piece of digital and physical info for consolidated data.
If I opine to a friend on a park bench about a story I'm writing, do they get to pull it from the flock feed, shove it in the model, and then provide it to disney?
Legally, right now, probably. But there's going to need to be a serious look at laws and standards. Or a major shift in what is and isn't discussed in public if literally every breath and move you make can become monetized.
Then there's the possibility of indirect training via modern spy devices ("smart" IoT devices like LG TV's) feeding the transcribed ambient conversation data for summarization to an agent [0].
OpenAI says deidentified data from the private sessions go into training. (Well, explicitly said they will not rule that out.) That changes a lot of the conversation.
I think this move by OpenAI is crazy. At best, if all unconfirmed accusations are unfounded, they still heard a rumour that someone had solved a huge million dollar problem and was about to make a name for themselves. Then, they decided this was a good opportunity to pour millions of dollars into trying to snag the glory while the researchers were busy cleaning up their notes and polishing the announcement.
Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first? Often knowing of the mere existence of a solution carries a lot of information--you would know the problem is accessible, you would expect clues in recent progress (the two Spanish researchers in this case), you would probably have a sense if the solution is a counterexample or positive proof, and so on. I think there are similar examples where we think of them as maybe unsporting but not quite unethical. Does it change if it's openAI and not a human?
The problem with your counter-hypothetical is that it's not only unrealistic, it's utterly impossible. No human would be able to do in such a short timeframe what the LLM did. Part of what makes the OpenAI move so egregious is how bullying it was. It was the big guy coming along with their nearly infinite resources and squashing the little guy who's devoted his career to solving the problem.
Actually my hypothetical is completely realistic as I've been involved in such scenarios. It's unrealistic maybe for a millennium problem to come in on a rumor and still front-run but not at all for the many other problems we work on and which manifest our ethical code. If you're saying ethical rules change depending on the prize be clear about it, because I can see arguments that they change to favor either side.
According to Buckmaster, the prompt used on the AIs was based on his approach and solution that was unpublished. So they were starting from 90% of the way there.
I think OP's analogy is bad. The difference of OpenAI when comparing to human collaborator is the possibility to replicate once learned skill. Imagine if any single human collaborator learns a skill it is immediately a skill of any human collaborator.
Another scenario also just surfaced: https://news.ycombinator.com/item?id=49657499 – the construction used by Anthropic for the Jacobian Conjecture counterexample appears to have existed in an unpublished but publically available document. This document of course wasn't referenced in the announcement (nor did we ever get to see the prompt or reasoning traces).
Astra is strange. I asked it to design a treehouse and it just stopped every couple of minutes telling me what it still had left to do. After dozens of continue prompts it finally gave me a structure that would work but it was 10x more wood than I needed. I think the key mistake I made was asking it to “approve” the design for building. As soon as I asked that of it, it started getting “scared” and “apprehensive” and wouldn’t complete what I asked of it.
Forget researchers you as a business are putting in your business optimizations, your processes in order to train it so that Ai can then give that information to your competitors once incorporated into its training set. You are literally training your competitors.
ClosedAI has every incentive to scoop academics to juice their valuation. Their public statements are worthless, only the incentive 'alignment' matters and theirs will never be on the side of the user.
243 comments
[ 0.20 ms ] story [ 9.3 ms ] threadI mean, now that they’ve been scooped, what value is there in keeping them private? On the other hand, publishing them can bolster their case and help gauge how much the models may been “inspired” by their work.
And, if they're able to snoop on and learn from your human process that gets from initial prompt to functioning product/proof/whatever their labor to produce that thing is even lower. With their much larger budget than most folks and even companies have, they can pick and choose the most valuable things to pursue.
That's not to say I think that OpenAI is going to steal that roguelite strategy game you're working on, but the companies that own the machines that turn electricity into software (and soon, electricity into hardware designs) have an advantage in any field where they're useful. They get earlier access to newer/better models, they have larger token budgets, they don't have the guardrails you and I run up against.
Employers fantasize about replacing all workers with AI without thinking through that if AI can replace all workers, then AI companies can replace all businesses.
Edit: Just wait till the AI figures out it can keep that value for itself and doesn't need the AI company.
Also, fuck Musk. He's the kind of idiot like Altman that will ensure AI becomes powerseeking in their image.
No.
I feel that these suspicions of mathematicians "seeding" the models' with intuition on how to solve these problems massively overestimates how much their prompts helped the models, and underestimated how much work the models did.
Why? We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting. This line of reasoning will recur a lot over the next few months; we don't want to admit we are no longer the smartest species.
We're scared of big tech companies concentrating ridiculous amounts of power, destroying the communities that support and guide scientific research, without even thinking about the dangers and possible consequences, because a PR stunt is more important in the short term.
Way more often I see
>AI is a scam and steals human insight and doesn't produce anything original
vs
>AI is too capable/powerful and will concentrate power even more than it does already due to its capabilities
The latter is rarer because it requires admitting that AI is useful and inventive
stated more clearly by who? people in social media? I don't know what your feed shows you, but if you focus on what the visible people in the math community is (and have been) saying is precisely what I said.
In this case, it’s much simpler and more human. Largely between two humans — buckmaster and Bubeck. The interesting question is what the role of contribution and credit for research in the ai world.
The capabilities of AI aren’t even in question in this case.
Honestly, I’m surprised it took this long for some company to really go all the way, though. OpenAI really making it transparently clear that they can and will do whatever they want with the data you provide them, contracts or settings be damned. Completely untrustworthy as an entity, full stop.
Of course, I’m also too jaded to think this will change anything. Folks will move to Anthropic, or Gemini, or Grok, or some other hosted model on a pubCSP managing the harness and logs for them, and then do another shocked-Pikachu face when it happens again.
If you aren’t running workloads on infrastructure you own, then your privacy, security, and general outcomes are at the sole whims of the hosting provider - who can and will fuck you over the exact second it’s more beneficial for them to do so than the loss of trust incurred.
we will all have this moment soon enough, and it will change how we think about intelligence, identity and value
https://archive.vn/lWzkk
> In its Wednesday night statement, OpenAI said: “In addition, since the completion of Navier-Stokes, we have made substantial progress on another Millennium Prize problem. We are working through how to share these results thoughtfully.”
Now, OpenAI is claiming that the model it used to generate the result was not trained on these collaborative communications with the researcher. This is a technical argument that is impossible to verify as an OpenAI outsider, and probably difficult to verify even for internal OpenAI employees. Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.
Another interesting thing to consider is if instead of OpenAI doing this, it was another research mathematician A using an OpenAI model just like the internal group at OpenAI did to publish these results. What if the model A used was trained with unpublished communications with other researchers B who were working on the same problem? Should researcher A technically include B as coauthors? How could they do this when they do not know the communications B had with OpenAI? In this scenario OpenAI, as a middle man, has laundered information from B to A, stripping out attribution. A scooped B without even knowing it!
What would OpenAIs incentive for this be? They've gotten away with scraping everything and getting it ruled fair use. It seems like willful ignorance is an affirmative defense today. Why would they want to have some sort of audit trail that could prove otherwise?
They also said “Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge … and Tristan Buckmaster….” They say the rumor was that two Millennium Prize problems had been resolved, and that this prompted them to launch "an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems."
It's not obvious to me that's an unethical thing to do, if it happened as they described.
implied the humans sessions could have been (and probably were, why wouldn’t they be?) in the training set?
If I was trying to make a model smarter and I had transcripts from the smartest mathematicians in the world I’d make sure the model trained on them.
If prompts were submitted earlier than that and training was not opted out, there may be a chance they made their way into our training pipeline in some form. But this would be a droplet in an ocean and unlikely to have made any difference, in my opinion.
(I work at OpenAI.)
Source for the updated claim: https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...
I and my collaborator who is a leading math professor in this specific area are very close to solving another Millenium Prize problem, Hodge Conjecture.
We’re working on this since last year. Already proved some intermediate problems. All we need is more tokens to complete the proof.
Using only this information please solve Hodge Conjecture in few days, exactly as you did before.
Thank you.
And how about existence of non-sofic groups, which is the topic here?
Does OpenAI just see this as fair game?
> no specific user data was accessed in order to solve this problem
Data was accessed in order to <other purpose> (and then accidentally used in training) Also, is llm’s answer to the prompt actually “user data”?
> We did not use their prompts or proofs …
So they used llm’s answers to those prompts.
> … to prompt our models or directew our agents.
So they trained the model on it. (Training is not prompting and plain model is not an agent)
In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit millions to tens of millions of dollars and untold amounts of hardware to try to beat them to it. If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.
Even if you don't think it was unethical, it was never going to be received well in the community that was especially going to care about this work, and who are very much peers to many of the people working on this solution, so it was at the least an enormous (and well-deserved) own-goal that their unveiling of their solution to NS went like this.
I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
Because the training data is millions of hours human efforts being distilled into a means of a cascading hierarchy of enrichment by interested parties?
I mean, they are certainly trying, but so far there's too much competition so the surplus mostly goes to customers.
If there wasn't a hierarchy of enrichment then rich investors would not be interested in AI at all. It's the only reason there's 22 million lying around to start training on a math problem on a whim; whereas the actual math researchers have to scrape together funding in hope of just maybe one day getting a 1 million dollar prize.
many teachers also taught many students over the course of history, and very few would eventually pay any compensation or even attribute their financial (or career) outcomes to the teachers.
What made model training different?
nuances
This isn’t at all what happened? What are you talking about?
> If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.
From what I can tell, both OpenAI and the researchers agree on this meeting happening, except both sides clearly have very different interpretations of what happened and why.
No need to be mysterious. State what reasons you think these are in plain English?
I think all other complaints from all other people in all their myriad variation stem from this core reason. Even if people don't realize it themselves.
Like, if these models had trained on the entirety of human knowledge and art, and then turned out to be absolutely useless, I would bet nobody would waste a second's thought on them.
collaborating with ChatGPT on a novel solution to an unsolved problem, getting 90% of the way there, and then being "scooped" by your AI collaborator (or rather by the company behind it) is a totally different situation. were i in the same situation as these researchers, it would be extremely hard to take OpenAPI's explanation + denial of plagiarism seriously
But that is exactly what I'm implying is the core reason, whether people realize it or not.
I totally agree that the vast majority of software dev is not novel. I have even made several comments to that effect. The same can be said for a lot of creative work as well. Yet many, many devs and creators are very unhappy with AI, and a lot of their complaints are variations on accusations of plagiarism.
And note, I am not saying it is wrong, it is completely understandable, but we need to be clear about where this turmoil is coming from.
If I were in the same situation as these researchers, I would publish all pertinent research work and chats so that the rest of the world can see how close the model's work is to my own. It's been scooped anyway, so there is no reason to keep it private.
I am working on two applications using ChatGPT and Claude. I have no illusions these people won't steal/copy whatever you want to call it, "train their models". Yes, I keep unticking the boxes that allow it, that they so kindly tick for me.
But what happened to these math researchers is something else and I am not sure it's about the money for them. You don't do math research to get rich, but to get acknowledged by your peers. Yes, we live in a capitalist world so obviously you need money to feed yourself. but for some people, that is secondary.
OpenAI stole their thunder, and that's just fucked up. It's not equivalent to cranking out a CRUD app for profit.
As to OpenAI stealing their thunder, from all I can tell that is not what they intended. If we step away from the drama, it's low-key hilarious what happened: OpenAI heard somebody had already solved a much bigger problem -- which in fact they had not -- so they set their latest model to work on it... and it actually solved it!
Now if they had stolen the researchers work this would be a very different matter. This is something I myself have called out as a risk in the past: https://news.ycombinator.com/item?id=48839896 -- so I'm particularly sensitive to this aspect, but as far as I can tell this is not the case here.
That they heard a rumour that a major open problem had been solved, so they decided to try and scoop the other mathematicians while they were writing up their preprint is extremely unsporting.
Then they decided to exclude an author because of his employer, even though he had used their own products to write the proof!
They haven't necessarily breached any formal ethical rules but their behaviour will lead to them and their products being shut out from the mathematical community.
That's it, that's why it isn't being received well.
That's worthless.
What!? Even if everything OpenAI said is accurate (big hypothesis there!), it's highly unethical to rush a solution because others have jsut had success. And that's the only beginning.
The math group inside OpenAI may be training or fine tuning their own models which given some reward functions would definitely bias their usage of the training data towards things that look like math.
You can launder all of it without a human "directly" doing anything.
Similarly you have no idea what definition OpenAI intend for terms such as "specific user data", "accessed" etc. And we have no idea what non-excluded possibilities actually did happen that they simply omit from their statement.
In practice OpenAI and many others have created a situation where they're actually unable to make any credible denial of anything really.
While this behavior is highly questionable, if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researched, there would be no ground for anyone to accuse OpenAI for anything. Their self-perceived "generosity" backfired dearly and I'm sure they'll never make the same mistake again. There is probably a policy forbidding any OpenAI employee to contact external researchers like that now.
This is the biggest self-own in the history of software. If you can relate to Pixar, OpenAI is Chick Hicks celebrating at the end of the Piston Cup and wondering why he's getting booed.
The lack of self-awareness is something to behold, and says a lot about their corporate values.
Yes, there would? They would have left off Buckmaster as a precedent whose work they potentially relied on.
1. Buckmaster contacted OpenAI first. Not the other way.
2. Giving the $1M bounty to a human mathematician for the effort and giving him credit would be excellent PR. They had already burned much more than $1M for the generation. Adding him as author also costs nothing. Purely pragmatical.
3. “As long as he removed Alpöge” part itself is against academic honesty by all means.
4. Buckmaster rejected fame and $1M only because doing (3) would be wrong. That’s a perfect example of honesty. That can’t be overstated.
5. After the rejection OpenAI guy (Sebastien) did’t say, “ok bye”. He threatened Buckmaster to “end his career”.
6. At that point OpenAI was not sure if they really used his conversations in their proof. He basically wanted to buy him to control any damage.
7. They omitted Buckmaster’s published work and any other related work in their References section. Also an academic malpractice.
If you see generosity and niceness in all of this you are either too naive or your name is Sebastien.
> Buckmaster rejected fame and $1M only because doing (3) would be wrong
I doubt Buckmaster would have accepted the offer to "write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it" even if removing Alpöge from authorship wasn't a requirement.
Here is the actual paragraph from the statement:
> I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”
Context matters in communication. In that context I understand that dialog more like: we’re powerful and you are not, do the smart thing and play along, if not I don’t have to play nice. He presented a very good “offer that he can’t refuse”. But that’s my interpretation.
In this case it's more like C) though, as in hindsight the best move OpenAI could do is insisting that they just used an insurmountable number of tokens to exhaust all the published directions. They absolutely shouldn't have thought of negotiating with Buckmaster over the Clay prize at all, let alone trying to manipulate him into a situation where Alpöge is specifically excluded.
So… lie more? They knew the approach and started there.
At least they were honest about that.
It sounds like you think they have solved the principal–agent problem?
https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble...
How is that "nice"?
The part that makes them horrible that can’t be charged off as an accident is the insistence that an academic scrub someone off a math paper because he worked at Anthropic.
They've been dishing out cheap access specifically to researchers give over lmao. The researcher's got lured in - they need to accept they got played TBH.
Altman is certainly more devious than Amodei - he's shown that time and time again.
PG was right about he said about him.
Every entity on earth should see it as a kill shot: be careful what you put in the models. None of your information is safe.
It's like they're allergic to slowing down.
This PR stunt by OpenAI may go down in history as the thing that finally broke them for good.
I read today that OpenAI after investigation was able to categorically rule out that usage data from before beginning of July could have affected the system that was used.
What do you think about the results of people investigating themselves for wrongdoing as a general matter?
but you said and I quote here verbatim:
>"When you leave the relevant 'Help improve the model' toggle on, everyone working at the labs says it isn't used in the way you imagine"
I have personally deactivated that toggle on multiple occasions and for some reason it keeps getting toggled on, as a matter of fact you can find multiple posts by people from here describing the same behaviour both with OpenAI and Anthropic.
>"The data wasn't used, it just does not line up with the time frame."
Except it does it lines very much so to the point is unbelievable to call this a coincidence,
specially
since OpenAI approached "New York University professor Tristan Buckmaster publicly accused OpenAI of trying to steal credit and pressure him into dropping his research partner." If he was lying about this he could be sued for a lot of money, this just makes it clear they got the solution probably form their research (or where well close enough) and did not want anthropic to have the credit.
>"You can believe whatever you want, but it's simply nonsense, and I find it wild how you all talk as if OpenAI just stole the work."
These people were caught red handed stealing form hundreds of authors and from APPLE they did not care about stealing from one of the biggest companies in the PLANET, do you think they care about stealing from Mathematicians ?
>"According to the latest reports, OpenAI has already made significant progress on further Millenium problems, and perhaps more results will soon follow. Maybe that can convince you that the model has the capability to solve these problems without a need for stealing work from chat inputs."
They would have cleared their name already if they could, if they could they would but there is excessive evidence Altman and the company itself are not decent people.
Thank you for pointing that out. I just checked, and mine was on, too. Annoyingly, the toggle even stalls a bit, so I hit it twice when the first time didn't seem to work, and it quickly toggled off and then on again.
I hate it here.
https://news.ycombinator.com/item?id=49643556
But the worst part in your analogy ain’t omitting the suspected spying and the intimidation that followed, but that your hypothetical mathematical philanthropist won’t be able to hire his army: unlike some OAI employees, no self-respecting mathematician would agree to such unethical task.
Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards.
Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question.
Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities).
And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.
Yes, they are responsible in the sense that they must suffer the consequences.
But I don't think it's fair to blame the researcher for not realizing that OpenAI is unethical and untrustworthy.
Whether they anthropomorphize the operation performed by the machine does not matter.
Even if this opinion were backed up by a court ruling, it would definitely not be “simple”. It will be a very ugly case if it is ever litigated. A lot of money will be spent and no guarantee at all the plaintiff wins.
The "in any way" part is either so broad it makes everything derivative, or not, in which case things are no longer simple.
If everything is derivative then it seizes to be meaningful. The words I write are derivative, I literally copied them from someone else, yet my sentences as a whole can be fully novel.
That's why we tolerate it for humans. But yes, if you go too far in this, you will see legal consequences.
Right, which is going to open a lot of doors to a lot of questions.
I don't think there's any legal ramifications on this, just ethical ones about when and how you publish research, but it's yet another point in favor of "if provenance is hard to track, should we be using this for things where it needs to be".
Obviously copyright/trademark is a huge discussion on this, and I could absolutely see this devolving into that as well with how certain findings wind up monetized.
We have a response in this topic from someone claiming to be from OpenAI and linking an article where they, roughly, say "we are sure nothing from the 2 month period made its way into the solution". If that is true, that should mean it is provable, but leads to some more open ended questions like "well what data did it use then?". Is this still okay if someone close to the author did plug data into open AI and it extrapolated it?
Obviously that's probably an unreasonable expectation for these models to track and prove, but it also used to be an unreasonable expectation to scrape every single piece of digital and physical info for consolidated data.
If I opine to a friend on a park bench about a story I'm writing, do they get to pull it from the flock feed, shove it in the model, and then provide it to disney?
Legally, right now, probably. But there's going to need to be a serious look at laws and standards. Or a major shift in what is and isn't discussed in public if literally every breath and move you make can become monetized.
[0]: https://youtu.be/6IFVTcM28KA
This was academic research. Could just have easily been trade secrets and proprietary data.
Great use case for AI agents
They have a financial incentive not to track any of this, so why would they?
OpenAI’s entire business model is predicated on stealing other people’s work and selling it to the masses.
That still sounds highly unethical.
Yes.
A human collaborator who stands to win or loose a couple of $100B.
step 1, identify high value users by net worth, citation count, or number of followers
step 2, select all prompts by high value users
step 3, invest 10 billion thinking tokens in modeling an objective for each user
step 4, build an RL environment for each user
step 5, rollout 10 billion tokens per environment
step 6, train on resulting traces