Interesting, and my main take away is that ~16 million sessions is enough to distill Claude. That's extremely doable - obviously, as it's been done repeatedly - but it just looks very feasible in general.
If I think of the number of lessons and educational conversations that a human would have to acquire their lifetime knowledge, I would hazard to say that AI-to-AI learning no longer requires many orders of magnitude beyond that.
MiniMax, DeepSeek, and Moonshot are all releasing models for the public to use for free.
Anthropic, OpenAI, Google ect have been scraping information to train their models that they had no right in scraping yet when these company pay them to scrap data we are suppose to be worried?
Labs like Anthropic always preach we are trying to build AI for everyone while releasing expensive models that are closed source.
The only reason AI is affordable at all is because of these Chinese AI labs.
It's been known for a long while that model outputs = data for training another model to copy the original model's behavior, also known as distillation.
What I didn't know is that the three groups mentioned "created over 24,000 fraudulent accounts and generated over 16 million exchanges with Claude, extracting its capabilities to train and improve their own models." There's some irony in that, given that Anthropic and all other established AI shops have been criticized for using copyrighted materials without permission to train their own models. I wouldn't be shocked if we subsequently find out tat every major AI shop has secretly engaged in distillation at some point in the past.
Still, wow, 24,000 accounts. I can't help but wonder, how many other AI shops have surreptitious accounts with other AI shops right now?
A) These models are trained by ignoring IP. It is hypocritical and absurd to then try to assert IP over them. And I am for the destruction of IP on all ends.
B) What this essentially means is that the Chinese labs are taking the work of these mega corporations into making it freely accessible to other labs and businesses, to serve inference, fine tune, and host privately on prem. That's clearly a good thing for competition in the market as a whole.
C) I don't see why we should have to duplicate the massive energy and infrastructure investment of building foundation models over and over forever just because we want to preserve the IP rights of a few companies. That seems a shame and it seems better to me for everything to learn from everything else for the whole ecosystem to get better by topping each other and building off each other; that's also why publishing research into the architecture and training of these models is so much better than what the proprietary labs do (keeping everything a secret), although tbf Anthropic's interpretability research is cool.
D) these Chinese models give 90% of the performance of frontier proprietary models at a 10th or 20th of the cost. That seems like a win for everyone. Not to mention the fact that this distilling also allows them to make much smaller local models that everyone can run. This is a win for actual democratization, decentralization, and accessibility for the little guy.
Well if you think about AI as another manhattan project this is clearly not that good. Or maybe it is? Maybe it will zero sum into some kind of mutually assured AI superiority destruction. Maybe anything else would be too tempting for the nation with access to super intelligence to not instantly submit the opponent that doesn’t possess an equivalent?
This is a bit overdramatizing. Let’s just think of it as strategic, tactical and planning superalgorithms. A nation without access to them becomes between severely disadvantaged and completely defenseless. Is it in the interest of the world to preserve the exiting status quo and balance of power?
Or let’s say we make china severely behind this technology. Now they cannot defend themselves from a potential USA swift nuclear strike so precise that a chance of retaliation is zero. Is this a net benefit to the world.
The company that claims all knowledge workers are going to be wiped out by their technology is asking these future disenfranchised workers to care about the Chinese ripping off their tech. That seems like a hard no.
It's crazy for their official account to post this when Anthropic itself is fighting multiple high-profile lawsuits over its unauthorized use of proprietary content to train its models. Did no one run this by legal?
Ironic phrasing used here. China is the only country that actually has the capacity to deeply integrate AI into industrial manufacturing in a way that will reduce costs of goods. They already have lights-off autonomous factories without AI.
Unless they stop selling APIs to the public this can’t be stopped.
Mind you that nuclear weapons are able to be regulated not because the tech itself is secret, it is because the refining is nation state effort, that is impossible to go unnoticed.
Realistically, the more tokens they are selling, the harder they can control it
Good that the US pissed so much softpower and goodwill away around the World that they won't convince other countries to ban these models. And only banning them domestically would hurt them on their own too much
70 comments
[ 5.1 ms ] story [ 71.3 ms ] threadIf I think of the number of lessons and educational conversations that a human would have to acquire their lifetime knowledge, I would hazard to say that AI-to-AI learning no longer requires many orders of magnitude beyond that.
MiniMax, DeepSeek, and Moonshot are all releasing models for the public to use for free.
Anthropic, OpenAI, Google ect have been scraping information to train their models that they had no right in scraping yet when these company pay them to scrap data we are suppose to be worried?
Labs like Anthropic always preach we are trying to build AI for everyone while releasing expensive models that are closed source.
The only reason AI is affordable at all is because of these Chinese AI labs.
What I didn't know is that the three groups mentioned "created over 24,000 fraudulent accounts and generated over 16 million exchanges with Claude, extracting its capabilities to train and improve their own models." There's some irony in that, given that Anthropic and all other established AI shops have been criticized for using copyrighted materials without permission to train their own models. I wouldn't be shocked if we subsequently find out tat every major AI shop has secretly engaged in distillation at some point in the past.
Still, wow, 24,000 accounts. I can't help but wonder, how many other AI shops have surreptitious accounts with other AI shops right now?
A) These models are trained by ignoring IP. It is hypocritical and absurd to then try to assert IP over them. And I am for the destruction of IP on all ends.
B) What this essentially means is that the Chinese labs are taking the work of these mega corporations into making it freely accessible to other labs and businesses, to serve inference, fine tune, and host privately on prem. That's clearly a good thing for competition in the market as a whole.
C) I don't see why we should have to duplicate the massive energy and infrastructure investment of building foundation models over and over forever just because we want to preserve the IP rights of a few companies. That seems a shame and it seems better to me for everything to learn from everything else for the whole ecosystem to get better by topping each other and building off each other; that's also why publishing research into the architecture and training of these models is so much better than what the proprietary labs do (keeping everything a secret), although tbf Anthropic's interpretability research is cool.
D) these Chinese models give 90% of the performance of frontier proprietary models at a 10th or 20th of the cost. That seems like a win for everyone. Not to mention the fact that this distilling also allows them to make much smaller local models that everyone can run. This is a win for actual democratization, decentralization, and accessibility for the little guy.
This is a bit overdramatizing. Let’s just think of it as strategic, tactical and planning superalgorithms. A nation without access to them becomes between severely disadvantaged and completely defenseless. Is it in the interest of the world to preserve the exiting status quo and balance of power?
Or let’s say we make china severely behind this technology. Now they cannot defend themselves from a potential USA swift nuclear strike so precise that a chance of retaliation is zero. Is this a net benefit to the world.
I hope so, I don't need their "safeguards".
Mind you that nuclear weapons are able to be regulated not because the tech itself is secret, it is because the refining is nation state effort, that is impossible to go unnoticed.
Realistically, the more tokens they are selling, the harder they can control it
* Likely they will seek regulation that would ban some models. Not sure this can work, but they will certainly try.
* Likely they will not release some of their next models in the API.