I would say there is a significant difference between AI discovering this completely on its own versus AI creating the finishing connecting part by connecting relevant data. Maybe this claim is too strong, but if part of it is true then the claims that OpenAI have made would be too strong as well.
To me it would feel more like how LLMs seem to work for me personally: incapable of unique work, but very capable of capturing large amounts of data and connecting the dots.
But this is what we do. Nobody ever invented or discovered anything in a vacuum - all discovery is synthesis of existing ideas and concepts applied to a novel domain. We laud Einstein for instance, but his work was a logical extension of Riemann - Riemann had a neat mathematical toy, Einstein described the universe with it - should we say Einstein was incapable of unique work?
“ It was Grossmann who emphasized the importance of a non-Euclidean geometry called Riemannian geometry (also elliptic geometry) to Einstein, which was a necessary step in the development of Einstein's general theory of relativity. Abraham Pais's book[9] on Einstein suggests that Grossmann mentored Einstein in tensor theory as well. Grossmann introduced Einstein to the absolute differential calculus, started by Elwin Bruno Christoffel[10] and fully developed by Gregorio Ricci-Curbastro and Tullio Levi-Civita.[11] Grossmann facilitated Einstein's unique synthesis of mathematical and theoretical physics in what is still today considered the most elegant and powerful theory of gravity: the general theory of relativity.”
Sounds like you just copy-pasted from AI without even understanding what you're talking about.
Based on what you're saying, you're claiming this is Grossman's work, not Einstein's. Why don't we rewrite scientific history too based on your copy-pasted AI slop?
It's so pointless talking to idiots who don't what they're talking about when they use AI, just because they think AI does everything, that reflects their own experience, not the experience of people who actually do real work. Some people are driven by AI, others drive it. As for those who are driven by it, they don't have sufficient imagination to think otherwise.
So you're just equivocating on terms like "prompt", "synthesis" and the like. Clearly a PhD in physics does not free people from scientistic modes of thinking and poor philosophy.
To think this discussion is about Einstein who had a much better mind on these things as well.
Actually, my undergraduate degree was physics and philosophy. And yes, synthesis is synthesis whether a human, a machine, or a duck does it, and people prompt one another all the time - “have you thought about trying X?” Or “I need the TPS report by EOB”.
I suppose my underlying point is that human cognition is not the unique and beautiful thing that we anthropocentrically suppose it to be - it is a physical process, with stochastic outcomes. Much like transformers.
Me, I’m just a machine made of meat. You can suppose yourself to be God’s perfect creation, and that’s your right, but I disagree.
Clearly your degrees did not make you immune from fallacies and simplistic reductions.
"Synthesis" is obviously of different kinds. A duck has a different level of intelligence than a human. We do not say both are "just doing synthesis".
So the question is how can you be so disingenuous about such terminology? Answer, you are relying on a classic form of scientistic reductivism.
The fact that intelligence is physical, emerges from chemistry, etc,. has nothing to do with there being also objectively different levels of computational sophistication.
If you want to be scientific about that you could look at neuropsychology on one hand and computability/complexity on the other. There are levels and so equivocation of "mentorship" as "prompting" and fallacious variants thereof is a) frankly intellectually obtuse, b) par for the course for SV-levels of philosophizing, c) and a disservice to philosophy, physics, and Einstein's own philosophical outlooks himself.
I am well aware of the Hinton-style physics argument about human cognition, and unlike others I am partial to it. But it is wrong to go about misunderstanding it so grossly.
What’s your basis for that “obviously”? You have a unique insight of the phenomenology of duck-ness? You can prove that your consciousness is somehow real? A duck synthesises with its cognition, or it would be incapable of, well, anything. Synthesis is purely the process of the integration of inputs into outputs.
“objectively different levels of computational sophistication”
Says who? We still have a very poor understanding of how cognition works in animals, humans included. For all we know ducks have rich inner lives - a remarkable amount can be achieved with a very small neurone count - cf. insects. Can you coordinate flight? Can you echolocate? Are you less intelligent because you cannot?
“equivocation of "mentorship" as "prompting" and fallacious variants thereof”
You are arguing semantics. Take Harry Nyquist. He sent people down new paths with insightful questions. You could call this mentorship if you choose, I could call it prompting, but this splits hairs. The core idea is that a novel input can produce a novel output, that synthesis can be induced through guided and deliberate external input.
I invoked credentials only in response to the previous derogatory comments about my cognition - which may or may not exist, anyway.
As to religiosity - the idea that human cognition is somehow unique and special and impossible to replicate, which is the prevailing argument in this comment tree is religious, and anthropocentrism of the highest order. I apologise for accusing you of it - I was evidently wrong - I had mistaken you for a previous poster.
You're wrong about the ducks. But getting back to your previous wrong argument from 14 hours ago, you basically deny the meaning of terms like "uninspired", "insipid", and "derivative", on the grounds that we're all standing on the shoulders of giants and therefore it's all good. This is incorrect, it's not all good, and the things the LLMs do really are unoriginal, a term that really does mean something.
They used words to mean what the words mean. What specific issue do you take with that?
"prompt", as in prompting an AI, has the same definition as "prompt", as in prompting a person. They mean the same thing, that's why the term was applied to AI after already applying people.
*Jingle-jangle fallacies are erroneous assumptions that either two different things are the same because they bear the same name (jingle fallacy); or two identical or almost identical things are different because they are labeled differently (jangle fallacy).[1][2][3] The term was coined by Truman Lee Kelley in his 1927 book Interpretation of educational measurements.[4] In research, a jangle fallacy is the inference that two measures (e.g., tests, scales) with different names measure different constructs. By comparison, a jingle fallacy is the assumption that two measures which are called by the same name capture the same construct.[5][6][7]
You are simply incorrect. It is not a fallacy of that type, or any other type, because the words do, in fact, mean the same thing, as multiple people have pointed out here. Whether referring to chatbots or people, "prompt" means "to move to action".
If you have some reliable source supporting your unilateral claims that "prompt" does not mean this, please share. Otherwise, the consensus seems to be contrary to your claims.
Why do I need a source? An LLM prompt does not "move to action", because an LLM does not act. People act, animals act, software doesn't act. Acting implies volition and volition implies cognition and if you think that LLMs have those things then you're the one who should provide a source for your claim.
How about an LLM connected to a robotic arm. What then?
As for cognition - can you prove that you possess it, to an external observer? Could you, confined to a box through which you can only communicate through textual messages, prove that you are thinking, and not just responding through a mechanistic process?
Absolutely you do. I can’t prove that I have cognition. I merely have the impression that I do, and we accept as a general convenience that others do too - but it is not provable. Je pense donc je suis is tautological, as you are examining a system from within the confines of that system.
And you’re seriously going with “don’t ask inconvenient questions” as an argument?
>> And you’re seriously going with “don’t ask inconvenient questions” as an argument?
Where did I say that?
The conversation you seem to want to have doesn't seem to have any obvious use and you seem to want to debate a point I never made. I don't seem to be needed here so I will now bow out of the conversation.
That commenter was already obnoxious under this thread, and today they are relitigating Searle-Turing via Hofstadter GEB slop, so they are also an ignoramus.
Because your entire post is a personal, philosohic opinion on what words should mean. The only way such an opinion could viably be considered "true" is if you convinced a majority of people to agree with you on the change, which you decidedly have not.
Until you do, "prompt" means "to move to action". "Act" means "the doing of a thing". By all reasonable accounts, LLMs "do things".
I'm not going articulate the answer but I will say whatever beef you have with the other commenter (or me), just copy this thread into ChatGPT and it will tell you why you are more in the wrong.
Grossmann collaborated with Einstein on GR, supplying quite a bit of the mathematical capacity required (which initially didn't come easily to Einstein). They published jointly, until Einstein was competent enough to work independently [1]. That's not equivalent to the situation being claimed here.
In this domain, an apparent single unique piece of work is often composed of several breakthroughs. For example, when Andrew Wiles proved Fermat's Last Theorem, he had to develop multiple new pieces of mathematical technology to get there.
The claim here seems to be that the human mathematicians, working with AI, developed technology to go A->B->C. By training on those conversations, OpenAI was then able to encourage the model to go A->B->C->D. In my opinion that should be acceptable, if it were openly disclosed, because it is in the public interest to make progress on these problems and because AI is clearly a useful tool for making progress. But the human mathematicians are saying that OpenAI is presenting the situation as if the model got from A->D entirely independently, without acknowledging their background contributions.
The incentive here for OpenAI's alleged intellectual dishonesty is presumably the commercial driver to "prove" progress towards AGI.
Going back to the specific topic at hand, who claimed data as their own when it wasn't? I don't see the interpretation of OpenAI solving the unsolved problem as claiming data that isn't theirs. I also don't recall them mentioning a particular method used in the solution, that was created by someone else, as theirs.
A lot of math is extremely specialized, to the extent that only a handful of other experts in some field have any experience with those mathematical ideas, with most of them not even yet present in the published literature. It's really not a stretch to claim that it's pretty dubious when the AI decides to use these highly specialized tools after it has trained on chat logs where these techniques were being discussed.
> They don't even claim to have had a proof, only to have been working on it.
Yeah, the guys who solved it for Euler and in the hypoviscous case, with the same technique that worked for full Navier--Stokes. They were "just" working on it.
"Discovery" is not a goal in itself. I could launch a project to find out how many people in the United States have names such that if you assign numbers to every character and then sum the values, the sum works out to 72. It's discovery, but it's useless unless it has some higher goal.
The labs are attacking these problems as a demonstration of capabilities, spending more money on the demos than any mathematician will ever see in their entire life. They don't care if the findings have any other value to anyone. Mathematicians have very different objectives for their work.
I don't know about you, but if I apply myself fully to a problem and study it to the point where I'm literally one of the world's experts on it and then some assholes in Silicon Valley take my research and claim it for themselves, I will probably not feel too great about that...
Are you trying to paint mathematicians as having an ulterior motive compared to Sam Altman? Are you trying to paint them as worse than other professions? Or what?
Humans are humans. I care about paying my bills and job security and peer recognition. You don't?
Yes, blame them for seeking out an upper middle class lifestyle with a relatively standard home in commuting distance of their place of work and dedicating the rest of their life to teaching mathematics to new generations of people. How vain a pursuit.
After all, the ascetics at openAI are having to make do with half a million total comp.
>Built a top a pyramid of failed math undergrads, grad students, and mediocre post docs.
Like much of things in this world, when you take a step back and realize that it was another human being who made that lunch time slop bowl for you, for the lowest wage the law allows for.
It looks to me more like they made a math engine that can sift through a huge number of combinations, most them absurd, to prove a statement. Just like a chess engine, but for math.
At least that's what I get from the NS result, they got from a point close to the solution to the solution by making it churn through 10 million bucks of compute.
If the allegations are true, I can't see that collaboration lasting. Unfortunately, researches need to earn a living too, and being front run by a lab for everything you do isn't going to pay the bills.
TL/DR: Mathematician opted out of training on 29-JUN and asked OpenAI whether they trained on his data and was told that it "did not happen" but it clearly did.
I've been suspecting over the last couple of years of the frontier companies using data for training anyway, regardless of training-use consent. "Using" the data doesn't have to mean they literally upload chat transcripts into pretraining datasets. My analogy has been money laundering -- if that can happen at massive scales, surely these companies can and will do the digital/data equivalent derivations/transformations. Even if one could have the access etc. to do so, how exactly would one prove that a given synthetic dataset that OAI/Anthropic uses is derived from particular user conversations that did not consent for the info to be used in training?
Consider, for instance that OpenAI's (consumer) terms say "If you do not want us to use your Content to train our models, you can opt out by following the instructions in this article ." but they also do say "We may use Content to provide, maintain, develop, and improve our Services". [1]
If you think that's quibbling, consider that OpenAI's business terms, in contrast, do state "OpenAI will not use Customer Content to develop or improve the Services, unless Customer explicitly agrees to such use.". [2]
And humanities have a word for this, exploitation, or appropriation, maybe it's time scientists and engineers revisited basic ethical notions. Skimming a dozen threads and nobody seems to have this vocabulary or willing to say it.
people seem to miss tge point of this. The problem isn't about credit, its about portraying these models as more competant than they really are. It fuels idiotic statements like jensen huangs recent "agi achieved" statement, which fuels an already dangerous financial fire.
As per the post, this mathematician has been working on this problem for 20 years. So either he was "just" about to breakthrough and this is a big coincidence, or Astra was able to push through the remaining block of 5-10-20-never years it might have taken.
That's still a pretty big marker of competence in my eyes.
The point of controversy seems to be who gets credit
To me that is not a credit thing because this removes a piece evidence for the ability of AI to come up with novel ideas while still making it a useful tool.
The big LLM providers, desperate for good PR before their IPOs, are all actively looking for 'almost finished' hard problems, e.g. where the conceptual / creative parts are almost done and they only need to throw their VC-backed resources at to brute-force through the remaining computationally expensive problem (lean, etc) and claim 'they have solved it'.
It's an utterly disrespectful, exploitive process, but all in line with exploitative predator capitalism of the stock market and big companies, now exploiting the knowledge / academia domain for scraps with a thin veneer of 'for science' PR.
The question's not new. In the early 1900s, women could not become PhD astronomers. Yet two women (Payne with stellar composition and Leavitt with cosmic distances) made fundamental, essential contributions to the science. Credit mostly went to male astronomers. The same might be said of Franklin and DNA.
It was nearly a century before the stories of all of them were revealed to public history. That the discoverers were not all equally rewarded is unjustifiable.
With regards to Franklin and DNA: the credit went to Watson and Crick because they had the fundamental insight: that DNA is an antiparalleldouble helix (Franklin knew it was a helix, but not an antiparallel double helix, which is key to the function of DNA). That data was shared in a departmental seminar. Further, she is explicitly acknowledged in W&C '53, and further, is the author of the paper immediately following W&C. She was never qualified to win the prize.
I want bunch of lawsuits, because the way things are described now produces perverse initiatives like try to discuss every possible idea that comes to mind with llm and if any of it works later claim the llm stole it.
I would like to see chat logs etc and understand how much of a progress was done by human.
Some mathematicians I know who've been following this have realized that they'd all gotten some emails from people they now know to be affiliated with OpenAI/Anthropic asking questions about their research in a way that seemed like scooping attempts.
Also, a lot of my mathematicians buddies have reported students basically asking if it's worth ever doing grad school for pure math, and even very motivated students are looking for other options now. It's not because they aren't passionate about it, it's that they don't want to work for another half decade or more just to have to start their careers all over.
All of this so that OpenAI and Anthropic can get into math result dick measuring to gas up their IPOs. Sickening.
I think you’re missing an important distinction. “Major damage” to the talent pipeline because models become capable of original end-to-end mathematics is what the community has been discussing. But if the models rely on sniping nearly complete work then this damage is antisocial without a lot of upside, it would be destroying a talent pipeline that would still necessary for continued progress.
Which is it? I don’t think OpenAI is being transparent enough for us to really understand whether these results would have been possible without relying on unpublished information from the solution strategies of the experts
You can't blame students for not seeing academia as the holy grail of knowledge anymore, when all the dialogue about technology and discovery has shifted to the hands of two private corporations
Relying on cloud services is a big liability. I'd think twice before feeding data to these LLM cloud products. If you make them a fundamental part of your product / development / workflow, be ready for the eventual moment the pricing and terms change.
I find it rather sad that the original posts have been posted on an open site, where anyone can read them, while the HN post points to an increasingly dubious site that doesn't even show the post unless you create an account and log in.
I thought the spirit of the open web would be more important to this community.
Doing some research and at this point doing it very much in the open with dates on GitHub so if any AI Lab says they re-discover my exact work it will be obvious that the AI used or was trained on my work. I am guessing anyone in a similar situation is now thinking about how they date their existing work if the math is done, but the proses are not.
At least in my experience, the issue with ArXiv is that the expectation is that the draft should be already in a good enough state. And polishing plus writing the meat around the main result can take a lot of time
This is true, as preprints are addressing the 'problem' at a later point in time. I guess git is better if you do not want to assign an identifier to an immutable version yet.
It’s crazy to me that companies/researchers share important data with these AI labs, you’re basically giving them your secret sauce which they then share with all of your competitors via training on conversations. At the same time I don’t really know alternatives other than a slightly less than frontier local LLM. Not sure how good they are at math.
Academic work is based on worldwide sharing, the sharing is not the problem, it's the lack of attribution. Unsurprisingly, these companies neglect standards of academic honor and attribution. Some human researchers also used to do that but in a discipline like mathematics this used to be a small problem because people tend to be so specialized that very few people could just grab someone's research and quickly piggyback on it, and if they do, colleagues will generally understand what happened. Unfortunately, AI is changing this.
You are both using a different definition of sharing I believe. When people have an expectation of privacy, use by others should be forbidden. Tech has gone completely off the rails with the use of private data.
The ultimate drive for some researches is the pursuit of knowledge. If I'm stuck at some block which prevents me from continuing in some direction that I want, of course I would like some help. I believe we already have nonzero collaborative proofs on math.SE, I can't recall good examples, but I have definitely seen citations to mathSE before.
So for me it sounds quite natural to also share this with AI especially under the privacy assumption. Also there's the assumption of scale -- maybe your problem is not large enough for anyone to care to scoop; and just for blind retraining, how do they know that the proof is even correct to include it into training? I have definitely received a ton of incorrect proofs before. So the SNR of such private chats is also not clear. I'm imagining millions of masters/phd students also trying to solve various random things with various capabilities, but how much real signal is there?
Because have not been paying attention to the discourse regarding AI for the last couple years?
That AIs unethical train on data wherever they may get it from has been in the news basically weekly.
But the very fact that you go to "chatgpt.com" and write to them; "Dear Diary, today I thought.."; there is no reason they would not receive and process your data, unless explicitly promising not to (which also requires us to trust them).
The fundamental rule in this case is that if we offload our data to a cloud provider we can assume they read it, if they can, unless they promised very clearly they will not.
Every single internet connect piece of software there is probably collects telemetry at this point. Why would this be any different? You know google logs your search data as well right? Not just the companies scan it but law enforcement too.
Don't make this our fault. I would even ask how is this not off by default or why aren't we asked upfront about it if they really care. It's disguising data collection as good faith. I don't even understand how this is legal under GDPR/EU given how much of PII they receive through chats.
If I dedicated my life to curing whatever, warts... and I'm making progress, but it's slow. And then here comes along this tool (LLM), and I use it, and it accelerates my progress to actually finding some sort of thing that makes warts more prone to being eradicated and then the lab throws a couple of million dollars of computes and lo and behold they eliminated warts. If I leave my ego and identity aside, which of course is hard for humans, wouldn't I be glad that warts is cured?
As a software developer that contributed to open source. Yeah. My code is there. It was the most beautiful code ever written and the labs stole it from me. And now they use it to progress much faster than I ever could. OK. Whatever. It's a tool. I solve problems. Can't I move on from this wart to the next?
To me, and I know this is gonna get me some heat, it just sounds like academics having their identity ruffled and turning their back to progress in the fields that they chose just because they don't get to play their little decades long of coffee, papers and ultimately identity politics.
Edit: never got to negative so fast on this board haha. This board is unfortunately turning, or has turned, to Reddit.
Downvotes might work as an abuse sponge, absorbing the impulse to make personal attacks. Other than that possible advantage, the downvote functionality seems contradictory to the concept of a discussion forum, I agree.
It’s more like you spend 4 years developing a product you’re passionate about. This product will gain you the respect of all your colleagues and either earn you money directly or lead to great career advancements. Then OpenAI takes it, changes the colour scheme, finishes the login flow and claims the whole thing as their own.
Not only would it piss you off but it would also misrepresent what OpenAIs models are capable of.
If I have infinite money to progress whatever problem solution I want but I always wait until I have an unfair advantage to get credit for whatever problem was just at the brink of a breakthrough anyway by sniping the last steps.
Am I actually doing a good thing or would it be better to let it run it's natural course and spend the money somewhere it's actually needed?
It's partly empathy with the person who did the work and had it stolen, in a field where the main thing people work for is credit. Maths isn't well paid, and doesn't make things that millions of people directly use.
It's also systemic, it cuts off the supply of results, if there is no reward any more for getting a result, the pipeline of maths will stop. It is the snake eating itself, which has a bad impact for all of us.
And I could dedicate my life to helping feed starving kids all across the globe. And then comes along this tool (a lockpick) and I use it, it accelerates my progress to actually getting money to fulfill my dream. If you leave your ego and identity aside, which of course is hard for you, wouldn't you be glad that I stole your money to feed starving kids?
If, as you say, it doesn't matter that the AI company gets praise for somebody else's discovery, then it also wouldn't matter if the praise went to the academic. You apparently resent the academic for seeking praise instead of being content with anonymously advancing human knowledge, but you don't resent the AI company seeking praise while leaching off the academic.
We love to do work that is useful and valuable to others, and we often form our identities around this. But identities are in large part socially constructed, so many of us need the recognition of others for our contribution. And it can be very painful when we perceive that the credit for our life's work got "stolen". Naturally, we fight against this. There's nothing shameful there. Sure, you can hold onto an ideal of egoless service. There's nothing wrong with that, either. But it's misanthropic to pass such harsh judgment on people for behaving in such a normal and natural manner.
Identity isn't the question. Eating is the question. If you can't come up with things you don't eat. If you come up with 90% of things and some overarching parasitic process comes in, puts in the 10%, and now they get 100% and you get 0%, you don't eat.
The problem is that AI is capital, and having to rent AI to keep up when it can just steal your mostly done work is something somehow even lower than wage-labor. They can use your own risked investment (the cash you paid to work) to get out in front of you and take credit.
I have yet to trust LLMs with anything important that can be capitalized on. I only use it to work on projects that if they stole and expanded on them, I'd actually be happy to see.
An academic's whole career is built on credit assignment for research breakthroughs. If someone else takes the credit, you lose. This is fundamentally different from a builder. You create things, solve problems and get paid for that instance. Nobody cares you 'invented' the blueprint for that building method. Your job is to instantiate. 100 Contractors can be building instance the exact same building somewhere else, it would not affect you. Most of IT builders are paid for what is basically 2 or 3 tier CRUD.
The real annoying thing it seems is mostly that openai is presumably doing this for internal reasons and this marginally increases the cost to users with no real gain.
It would be one thing to gain from it but removing prestige wins from customers AND reducing compute support just feels like being ultra mean if you zoom out.
If this was racing to cure cancer ahead of researchers we wouldn't be writing about this on HN.
It is suspicious that OpenAI decided to generate 300 billion output tokens from a model still in training, right after learning there was credible chance that a major math proof was in that model’s training data. Obviously there are reasonably plausible explanations for each step, but it does sort of feel like parallel construction.
I think people are focusing on the training data issue too much. If the data was contaminated, I can still blame that on negligence.
But, at least with the Navier-Stokes solution, it's clear [^1] that they learned that Alpöge and Buckmaster were getting close to a solution and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt.
What makes this worse to me is the intention. They intentionally threw $15 million in compute at the problem in order to scoop the result. They intentionally left Buckmaster and Alpöge out of the citations.
Data contamination should be enough to disqualify them from the prize, but I can believe it to be accidental. On the other hand, someone made an intentional decision to scoop the result by throwing money at the problem. That's so much worse.
[^1]: That's the timeline claimed by Buckmaster, and no one from OAI has disputed it.
I think you’re overlooking what I’m implying here. It’s not that they knew contamination was possible but they went ahead anyway. To spell it out just a little bit more: learning the answer might be in model X’s training data made them believe that model X specifically might be able to solve the question, and they were able to very quickly find enough certainty about the former to commit millions of dollars to the latter.
Their claim is that training on their solution is "unlikely but possible".
Consider this scenario.
Has a google crawler read my new novel, which I may or may not have posted on my blog, page by page, as I wrote it?
Can you, without knowledge of what I have actually done, claim that the google crawler has not seen the novel?
Without any evidence that I have posted the novel online, it might be tempting to say that the crawler has not seen the novel, but what if I were in an adversarial position against Google on this topic and were challenging them to make that claim. You would wonder if I were hoping Google to overreach by making a definitive claim without taking into account some action that they had no knowledge of. It becomes difficult to use the scientific expression "There is no evidence for this" when there is an accusation of malfeasance because it can be so easily be conflated as "You can't prove we did it". It seems like the best you could say would be 'Unlikely, but possible'
> Such intensive use of AI doesn't come cheap. In a post on X, LisanBench, an LLM benchmark evaluator, estimated that the output tokens alone would cost about $6.5 million at OpenAI's average consumer price. Including the far larger volume of input tokens, the post estimated the total could reach $10 million to $40 million.
When I worked at Google, we spent $100M in power on protein folding and drug discovery (this was long before AlphaFold). Never underestimate the willingness of smart rich people to invest in speculative science.
No, they asked one guy to do a joint publish conditioned on leaving the other collaborator out, with veiled threats. The joint publish part smells awfully like admission of guilt given there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work. The leaving out collaborator part is outright academic malpractice. Disclosure: I was an academic once.
This would be a new publication with OpenAI's novel solution. Buckmaster and Alpöge did not arrive at the NS solution, they used an approach that also happened to be used in OpenAI's proof.
> there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work
Unless the goal is to assuage the existential pain felt by many mathematicians with respect to what may feel like increasingly inconsequential efforts. It also seemed like a way to build good will amongst knowledge workers dealing with similar issues, with the goal of showing they are dedicated to easing the transition for them as society parades into the future. This has clearly backfired.
> The leaving out collaborator part is outright academic malpractice.
They would not be leaving a collaborator out. Again, this would be a new publication. Here's the quote from Buckmaster's post [1]:
"Two proposals were offered to me. The first was that we post our Euler
result, and that OpenAI post its Navier-Stokes result the next day. The second
was that, after posting Euler, I alone write a paper presenting the Navier-Stokes
result, acknowledging that an internal OpenAI model had resolved it."
These are utter bullshit that breaks every norm, but may be convincing enough for some non-academic OpenAI cheerleaders who post dozens of comments on these topics.
I'm genuinely surprised that more people - including this mathematician in particular - don't untick the "improve the model for everyone" box. Unless the suggestion is that OpenAI ignore this preference?
All the big AI labs were built on stealing IP; who is surprised that's still how they operate? And who believes, or has ever believed, their promises that your data is private and not logged, etc.?
The big AI labs are not trying to advance humanity, they are in this for the money, and as most (all?) private companies they don't care about ethics at all.
That doesn't mean they can't be useful, or that their products are trash, etc. It just means that they shouldn't ever be trusted. Buyer beware.
It's hilarious how people think they care about their reputation, and wouldn't circumvent ZDR policies. Like bro, they literally covertly hired Apple employees and had them steal IP and equipment form Apple. They aren't scared of Apple lawyers, so they definitely aren't scared of yours.
what the point and usefulness of the comments above? we shouldn't be surprised? is normal to steal? hiring apple employees?
can you realize what this means?
focus on this part:
"If his account is correct, this is not a minor dispute over attribution. It would mean that unpublished human work was absorbed into a model and then presented to the world as a breakthrough by the model itself"
Don’t they openly state that their product may cause IP issues but that is fine because they will take care of your legal problems caused by their product?
In the end, it’s not them stealing, it’s the AI doing stealing. What kind of moral compass are we talking about?
But I don't see how. AI is going to be a commodity in short order and best case the US will be a temporary leader in the supply of tokens. Meanwhile AI is going to destroy much of the Service and Software industry that make up most of the US economy. And the US is betting every last cent to bring about this future. It does make sense for Trump since this might be a sugar high that lasts till the end of his term.
In USA there is surprisingly little state involvement in the whole llm mania. Who needs the state with 800 lbs gorillas like Google, Amazon, Nvidia, etc
In China, it is the principal obsession of the entire communist party which eg funds the whole infrastructure without a single NIMBY peep.
The strange emphasis in China on humanoid robotic constructions is due to the CCP realization that with the cataclysmic fertility collapse they will increasingly have no one to rule.
They have throughout this period of AI products shown to reproduce works that they were trained on. They are getting sued all over the place for the theft of content right now and it seems courts and governments want to wave copyright protection (and ignore criminal acts because the "ai did it") to see where this leads.
Its why I stopped writing open source software, my code was stolen and put behind a paywall and the license under which it was published has not been adhered to. Doing work in the public domain at all now is just stupid, these companies are allowed to steal it and call it their own.
I wonder what’s more valuable in our prompts: the raw data or the feedback system that drives the exchange towards a goal.
For a long time it was clearly the former, but now I think it is the latter.
The models have enough knowledge (orders of magnitude more than a human could ever learn) but are now getting better at what to do with it thanks to learning from the decisions that we make in conversations with AI agents.
I think so too. The value is in the entire conversation. IMO, "domain experts" don't run LLMs blindly and hands free. This does not work for top level work (e.g., mathematical proofs, coding anything more complex than yet another slop game or website). Experts have long sessions where they prompt and guide LLM in response to what it produces. This is the discovery process. And frontier labs definitely train on that.
The billion dollar question is whether this works "out of the distribution". I.e., whether LLMs can only find and use the specific ideas buried in training data, or whether they can learn to apply the "thinking process" to a new problem. IMO this is still unanswered (due to these recent controversies).
But regardless of the answer, it seems we have a planet-scale positive feedback loop here. LLM became good (enough) by training on generally available data (books, internet, github) + RLFH, so experts tried to use them on hard tasks, which required lots of hand holding. These conversations became part of the training data, and the next generation of frontier LLMs were better. So, more experts used them on harder tasks, again requiring hand holding. These conversation became part of the training data... etc.
In a nutshell, top human minds across the world are pouring their skills into LLMs just by using them. This is not "continuous learning", but if you re-train on the most recent sessions every, say, quarter (which seems to be happening?) you get close to that in practice.
OpenAI is showing the world why they shouldn't trust AI hosted on some cloud somewhere.
If they're stealing math proofs to advertise their models, who's to say they won't steal your businesses IP to gain a competitive advantage?
They're not to be trusted with your data. I can't believe how short-sighted this is, they got a quick PR win at the expense of a much larger trust problem.
I wouldn't trust cloud AI at all at this point. Get an open Chinese model and host it yourself somewhere. The initial costs might be higher, but you'll break even pretty quickly and nobody will be able to steal your innovations.
238 comments
[ 0.18 ms ] story [ 11.5 ms ] threadThey don't even claim to have had a proof, only to have been working on it.
To me it would feel more like how LLMs seem to work for me personally: incapable of unique work, but very capable of capturing large amounts of data and connecting the dots.
“ It was Grossmann who emphasized the importance of a non-Euclidean geometry called Riemannian geometry (also elliptic geometry) to Einstein, which was a necessary step in the development of Einstein's general theory of relativity. Abraham Pais's book[9] on Einstein suggests that Grossmann mentored Einstein in tensor theory as well. Grossmann introduced Einstein to the absolute differential calculus, started by Elwin Bruno Christoffel[10] and fully developed by Gregorio Ricci-Curbastro and Tullio Levi-Civita.[11] Grossmann facilitated Einstein's unique synthesis of mathematical and theoretical physics in what is still today considered the most elegant and powerful theory of gravity: the general theory of relativity.”
Based on what you're saying, you're claiming this is Grossman's work, not Einstein's. Why don't we rewrite scientific history too based on your copy-pasted AI slop?
It's so pointless talking to idiots who don't what they're talking about when they use AI, just because they think AI does everything, that reflects their own experience, not the experience of people who actually do real work. Some people are driven by AI, others drive it. As for those who are driven by it, they don't have sufficient imagination to think otherwise.
To think this discussion is about Einstein who had a much better mind on these things as well.
I suppose my underlying point is that human cognition is not the unique and beautiful thing that we anthropocentrically suppose it to be - it is a physical process, with stochastic outcomes. Much like transformers.
Me, I’m just a machine made of meat. You can suppose yourself to be God’s perfect creation, and that’s your right, but I disagree.
"Synthesis" is obviously of different kinds. A duck has a different level of intelligence than a human. We do not say both are "just doing synthesis".
So the question is how can you be so disingenuous about such terminology? Answer, you are relying on a classic form of scientistic reductivism.
The fact that intelligence is physical, emerges from chemistry, etc,. has nothing to do with there being also objectively different levels of computational sophistication.
If you want to be scientific about that you could look at neuropsychology on one hand and computability/complexity on the other. There are levels and so equivocation of "mentorship" as "prompting" and fallacious variants thereof is a) frankly intellectually obtuse, b) par for the course for SV-levels of philosophizing, c) and a disservice to philosophy, physics, and Einstein's own philosophical outlooks himself.
I am well aware of the Hinton-style physics argument about human cognition, and unlike others I am partial to it. But it is wrong to go about misunderstanding it so grossly.
What’s your basis for that “obviously”? You have a unique insight of the phenomenology of duck-ness? You can prove that your consciousness is somehow real? A duck synthesises with its cognition, or it would be incapable of, well, anything. Synthesis is purely the process of the integration of inputs into outputs.
Here’s a paper on duck synthesis:
https://www.pbs.org/newshour/science/ducklings-make-way-abst...
“objectively different levels of computational sophistication”
Says who? We still have a very poor understanding of how cognition works in animals, humans included. For all we know ducks have rich inner lives - a remarkable amount can be achieved with a very small neurone count - cf. insects. Can you coordinate flight? Can you echolocate? Are you less intelligent because you cannot?
“equivocation of "mentorship" as "prompting" and fallacious variants thereof”
You are arguing semantics. Take Harry Nyquist. He sent people down new paths with insightful questions. You could call this mentorship if you choose, I could call it prompting, but this splits hairs. The core idea is that a novel input can produce a novel output, that synthesis can be induced through guided and deliberate external input.
I invoked credentials only in response to the previous derogatory comments about my cognition - which may or may not exist, anyway.
As to religiosity - the idea that human cognition is somehow unique and special and impossible to replicate, which is the prevailing argument in this comment tree is religious, and anthropocentrism of the highest order. I apologise for accusing you of it - I was evidently wrong - I had mistaken you for a previous poster.
"prompt", as in prompting an AI, has the same definition as "prompt", as in prompting a person. They mean the same thing, that's why the term was applied to AI after already applying people.
*Jingle-jangle fallacies are erroneous assumptions that either two different things are the same because they bear the same name (jingle fallacy); or two identical or almost identical things are different because they are labeled differently (jangle fallacy).[1][2][3] The term was coined by Truman Lee Kelley in his 1927 book Interpretation of educational measurements.[4] In research, a jangle fallacy is the inference that two measures (e.g., tests, scales) with different names measure different constructs. By comparison, a jingle fallacy is the assumption that two measures which are called by the same name capture the same construct.[5][6][7]
https://en.wikipedia.org/wiki/Jingle-jangle_fallacies
If you have some reliable source supporting your unilateral claims that "prompt" does not mean this, please share. Otherwise, the consensus seems to be contrary to your claims.
As for cognition - can you prove that you possess it, to an external observer? Could you, confined to a box through which you can only communicate through textual messages, prove that you are thinking, and not just responding through a mechanistic process?
And you’re seriously going with “don’t ask inconvenient questions” as an argument?
Where did I say that?
The conversation you seem to want to have doesn't seem to have any obvious use and you seem to want to debate a point I never made. I don't seem to be needed here so I will now bow out of the conversation.
Because your entire post is a personal, philosohic opinion on what words should mean. The only way such an opinion could viably be considered "true" is if you convinced a majority of people to agree with you on the change, which you decidedly have not.
Until you do, "prompt" means "to move to action". "Act" means "the doing of a thing". By all reasonable accounts, LLMs "do things".
[1] https://arxiv.org/pdf/1312.4068
The claim here seems to be that the human mathematicians, working with AI, developed technology to go A->B->C. By training on those conversations, OpenAI was then able to encourage the model to go A->B->C->D. In my opinion that should be acceptable, if it were openly disclosed, because it is in the public interest to make progress on these problems and because AI is clearly a useful tool for making progress. But the human mathematicians are saying that OpenAI is presenting the situation as if the model got from A->D entirely independently, without acknowledging their background contributions.
The incentive here for OpenAI's alleged intellectual dishonesty is presumably the commercial driver to "prove" progress towards AGI.
This is what research is; collecting data and connecting the dots.
If this wasn’t human driven, I’d expect to see other problems within that problem. Space solved not just the ones that it had chat data on.
Yeah, the guys who solved it for Euler and in the hypoviscous case, with the same technique that worked for full Navier--Stokes. They were "just" working on it.
The labs are attacking these problems as a demonstration of capabilities, spending more money on the demos than any mathematician will ever see in their entire life. They don't care if the findings have any other value to anyone. Mathematicians have very different objectives for their work.
Humans are humans. I care about paying my bills and job security and peer recognition. You don't?
After all, the ascetics at openAI are having to make do with half a million total comp.
That half a million total comp is the consolation prize for the disillusioned.
Like much of things in this world, when you take a step back and realize that it was another human being who made that lunch time slop bowl for you, for the lowest wage the law allows for.
At least that's what I get from the NS result, they got from a point close to the solution to the solution by making it churn through 10 million bucks of compute.
Consider, for instance that OpenAI's (consumer) terms say "If you do not want us to use your Content to train our models, you can opt out by following the instructions in this article ." but they also do say "We may use Content to provide, maintain, develop, and improve our Services". [1]
If you think that's quibbling, consider that OpenAI's business terms, in contrast, do state "OpenAI will not use Customer Content to develop or improve the Services, unless Customer explicitly agrees to such use.". [2]
[1] https://archive.is/EcwD8 [2] https://archive.is/yZdAF
That's still a pretty big marker of competence in my eyes.
The point of controversy seems to be who gets credit
It's an utterly disrespectful, exploitive process, but all in line with exploitative predator capitalism of the stock market and big companies, now exploiting the knowledge / academia domain for scraps with a thin veneer of 'for science' PR.
It was nearly a century before the stories of all of them were revealed to public history. That the discoverers were not all equally rewarded is unjustifiable.
I would like to see chat logs etc and understand how much of a progress was done by human.
Also, a lot of my mathematicians buddies have reported students basically asking if it's worth ever doing grad school for pure math, and even very motivated students are looking for other options now. It's not because they aren't passionate about it, it's that they don't want to work for another half decade or more just to have to start their careers all over.
All of this so that OpenAI and Anthropic can get into math result dick measuring to gas up their IPOs. Sickening.
It was long predicted that math and software developments would be the first domain where AI was going to do major damage.
If OpenAI and Anthropic didn't get into math result dick measuring, Internet anons would have in their place, 6 months later when it got cheaper.
Which is it? I don’t think OpenAI is being transparent enough for us to really understand whether these results would have been possible without relying on unpublished information from the solution strategies of the experts
https://mathstodon.xyz/@andreasthom/117240535270608201
https://mathstodon.xyz/@andreasthom/117240536885387540
https://mathstodon.xyz/@andreasthom/117240537520615623
I thought the spirit of the open web would be more important to this community.
So for me it sounds quite natural to also share this with AI especially under the privacy assumption. Also there's the assumption of scale -- maybe your problem is not large enough for anyone to care to scoop; and just for blind retraining, how do they know that the proof is even correct to include it into training? I have definitely received a ton of incorrect proofs before. So the SNR of such private chats is also not clear. I'm imagining millions of masters/phd students also trying to solve various random things with various capabilities, but how much real signal is there?
"Its crazy to me that some people are not egotistical, self-centered, and don't solely care about fame and wealth accumulation".
How was I not aware of this before?
The fundamental rule in this case is that if we offload our data to a cloud provider we can assume they read it, if they can, unless they promised very clearly they will not.
If I dedicated my life to curing whatever, warts... and I'm making progress, but it's slow. And then here comes along this tool (LLM), and I use it, and it accelerates my progress to actually finding some sort of thing that makes warts more prone to being eradicated and then the lab throws a couple of million dollars of computes and lo and behold they eliminated warts. If I leave my ego and identity aside, which of course is hard for humans, wouldn't I be glad that warts is cured?
As a software developer that contributed to open source. Yeah. My code is there. It was the most beautiful code ever written and the labs stole it from me. And now they use it to progress much faster than I ever could. OK. Whatever. It's a tool. I solve problems. Can't I move on from this wart to the next?
To me, and I know this is gonna get me some heat, it just sounds like academics having their identity ruffled and turning their back to progress in the fields that they chose just because they don't get to play their little decades long of coffee, papers and ultimately identity politics.
Edit: never got to negative so fast on this board haha. This board is unfortunately turning, or has turned, to Reddit.
It’s more like you spend 4 years developing a product you’re passionate about. This product will gain you the respect of all your colleagues and either earn you money directly or lead to great career advancements. Then OpenAI takes it, changes the colour scheme, finishes the login flow and claims the whole thing as their own.
Not only would it piss you off but it would also misrepresent what OpenAIs models are capable of.
If I have infinite money to progress whatever problem solution I want but I always wait until I have an unfair advantage to get credit for whatever problem was just at the brink of a breakthrough anyway by sniping the last steps. Am I actually doing a good thing or would it be better to let it run it's natural course and spend the money somewhere it's actually needed?
The thing is, everyone knows that Apple does that and Apple doesn't care if people have a beef with them.
It's also systemic, it cuts off the supply of results, if there is no reward any more for getting a result, the pipeline of maths will stop. It is the snake eating itself, which has a bad impact for all of us.
The problem is that AI is capital, and having to rent AI to keep up when it can just steal your mostly done work is something somehow even lower than wage-labor. They can use your own risked investment (the cash you paid to work) to get out in front of you and take credit.
I have yet to trust LLMs with anything important that can be capitalized on. I only use it to work on projects that if they stole and expanded on them, I'd actually be happy to see.
Please don't post comments saying that HN is turning into Reddit. It's a semi-noob illusion, as old as the hills.
:^)
It would be one thing to gain from it but removing prestige wins from customers AND reducing compute support just feels like being ultra mean if you zoom out.
If this was racing to cure cancer ahead of researchers we wouldn't be writing about this on HN.
But, at least with the Navier-Stokes solution, it's clear [^1] that they learned that Alpöge and Buckmaster were getting close to a solution and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt.
What makes this worse to me is the intention. They intentionally threw $15 million in compute at the problem in order to scoop the result. They intentionally left Buckmaster and Alpöge out of the citations.
Data contamination should be enough to disqualify them from the prize, but I can believe it to be accidental. On the other hand, someone made an intentional decision to scoop the result by throwing money at the problem. That's so much worse.
[^1]: That's the timeline claimed by Buckmaster, and no one from OAI has disputed it.
So such thing existed. In fact, what they learnt was some progress existed, not what the specific progress was.
Do you have any evidence of this? They don't dispute the timeline, but they never said they knew what Levant/Buckmaster were doing.
What accident is it when the system is designed to function that way?
Consider this scenario.
Has a google crawler read my new novel, which I may or may not have posted on my blog, page by page, as I wrote it?
Can you, without knowledge of what I have actually done, claim that the google crawler has not seen the novel?
Without any evidence that I have posted the novel online, it might be tempting to say that the crawler has not seen the novel, but what if I were in an adversarial position against Google on this topic and were challenging them to make that claim. You would wonder if I were hoping Google to overreach by making a definitive claim without taking into account some action that they had no knowledge of. It becomes difficult to use the scientific expression "There is no evidence for this" when there is an accusation of malfeasance because it can be so easily be conflated as "You can't prove we did it". It seems like the best you could say would be 'Unlikely, but possible'
what? really?
> Such intensive use of AI doesn't come cheap. In a post on X, LisanBench, an LLM benchmark evaluator, estimated that the output tokens alone would cost about $6.5 million at OpenAI's average consumer price. Including the far larger volume of input tokens, the post estimated the total could reach $10 million to $40 million.
https://www.businessinsider.com/openai-math-problem-solved-t...
No, they asked if they could do a joint publish.
> there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work
Unless the goal is to assuage the existential pain felt by many mathematicians with respect to what may feel like increasingly inconsequential efforts. It also seemed like a way to build good will amongst knowledge workers dealing with similar issues, with the goal of showing they are dedicated to easing the transition for them as society parades into the future. This has clearly backfired.
> The leaving out collaborator part is outright academic malpractice.
They would not be leaving a collaborator out. Again, this would be a new publication. Here's the quote from Buckmaster's post [1]:
"Two proposals were offered to me. The first was that we post our Euler result, and that OpenAI post its Navier-Stokes result the next day. The second was that, after posting Euler, I alone write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it."
[1] https://cims.nyu.edu/~tristanb/statement.pdf
(a) Click thumbs-up/down in the conversation [1]
(b) Have the conversation flagged for potential safety concerns
[1]: https://help.openai.com/en/articles/5722486-how-your-data-is....
The big AI labs are not trying to advance humanity, they are in this for the money, and as most (all?) private companies they don't care about ethics at all.
That doesn't mean they can't be useful, or that their products are trash, etc. It just means that they shouldn't ever be trusted. Buyer beware.
can you realize what this means?
focus on this part:
"If his account is correct, this is not a minor dispute over attribution. It would mean that unpublished human work was absorbed into a model and then presented to the world as a breakthrough by the model itself"
don't threat this as a minor dispute!
also why not nitter link? not even in comments?
https://nitter.xitter.cc/ValerioCapraro/status/2097791836269...
Two different things. It's not normal to steal, but we shouldn't be surprised thieves steal. It's what they do.
In the end, it’s not them stealing, it’s the AI doing stealing. What kind of moral compass are we talking about?
In China, it is the principal obsession of the entire communist party which eg funds the whole infrastructure without a single NIMBY peep.
The strange emphasis in China on humanoid robotic constructions is due to the CCP realization that with the cataclysmic fertility collapse they will increasingly have no one to rule.
Its why I stopped writing open source software, my code was stolen and put behind a paywall and the license under which it was published has not been adhered to. Doing work in the public domain at all now is just stupid, these companies are allowed to steal it and call it their own.
Why are you saying that this article explains it much better than the tweet that you clearly didn't even read..
For a long time it was clearly the former, but now I think it is the latter.
The models have enough knowledge (orders of magnitude more than a human could ever learn) but are now getting better at what to do with it thanks to learning from the decisions that we make in conversations with AI agents.
The billion dollar question is whether this works "out of the distribution". I.e., whether LLMs can only find and use the specific ideas buried in training data, or whether they can learn to apply the "thinking process" to a new problem. IMO this is still unanswered (due to these recent controversies).
But regardless of the answer, it seems we have a planet-scale positive feedback loop here. LLM became good (enough) by training on generally available data (books, internet, github) + RLFH, so experts tried to use them on hard tasks, which required lots of hand holding. These conversations became part of the training data, and the next generation of frontier LLMs were better. So, more experts used them on harder tasks, again requiring hand holding. These conversation became part of the training data... etc.
In a nutshell, top human minds across the world are pouring their skills into LLMs just by using them. This is not "continuous learning", but if you re-train on the most recent sessions every, say, quarter (which seems to be happening?) you get close to that in practice.
I’ve been working on a novel project for 1 year.
I now no longer use llm’s - the continual chatter I’ve had has resulted in my insights being found in the training data now.
Get stuffed OAI.
Every large firm will soon enough want its own on-prem servers eventually. Maybe nation’s will get involved and build out their own data centres.
Not a chance in hell I’d trust a tech firm to treat my IP as safe and sound.
There might be no books about human intuition but we teach it to LLMs by interacting with them
I don’t know why but it just ‘sounds right’. It’s the best analogy I can think of.
If they're stealing math proofs to advertise their models, who's to say they won't steal your businesses IP to gain a competitive advantage?
They're not to be trusted with your data. I can't believe how short-sighted this is, they got a quick PR win at the expense of a much larger trust problem.
I wouldn't trust cloud AI at all at this point. Get an open Chinese model and host it yourself somewhere. The initial costs might be higher, but you'll break even pretty quickly and nobody will be able to steal your innovations.
This is American AI companies committing suicide.