This isn't really that newsworthy. Grok was probably trained on ChatGPT logs, just like the litany of other chat-oriented LLMs that are also open source. They still should've filtered out the OpenAI canned responses (there are datasets with that filter).
Can they keep a straight face when trying to justify this clause of their terms? Isn't training on content without permission kind of their whole business model?
> Can they keep a straight face when trying to justify this clause of their terms?
Probably.
> Isn't training on content without permission kind of their whole business model?
Right, their business model rests on the argument that copyright law doesn't require permission from copyright holders. They aren't relying on copyright law, but contract law (which only binds direct users, but applies to material not covered by copyright and applies in circumstances where even if material was covered by copyright use would be allowed as, e.g, fair use.)
Is it an attempt to pull up the ladder? Obviously. But both the safety mission of the charity and the commercial competitive interests of the for profit provide rationales for this, so there is no reason to think that just because it hobbles others in trying to catch up that it would be difficult for them to do it with a straight face.
If they aren't OpenAI customers so they aren't subject to the terms.
There is a meaningful distinction between using ChatGPT directly, putting in the prompts and taking the results, and just finding random output that other people have published in the internet and using that.
> This isn't really that newsworthy. Grok was probably trained on ChatGPT logs
If Musk's "anti-woke" AI was just trained on logs from what he calls "woke AI", that would, actually, be particularly newsworthy.
(The explanation they have put forward is that this is not the case, but they just failed to realize that blindly ingesting web content would bring in a lot of ChatGPT-derived content.)
I have a hard time believing that just pulling publicly available web data would instill a professed identity of an openAI creation, rather than a gen-x hipster or zoomer, or whatever. Seems so much more likely they trained predominately on gpt-3.5 output on the sly. And I’m not even sure that would do it — how often does open chatGPT identify openAI in normal responses? And OpenAI instills that sense of identity with fine tuning with synthetic data.
Tell me how frequently does ChatGPT refer to itself as anything except ChatGPT?
Basically never.
The raw GPT model would be anything. My favorite was "I'm a young female lawyer with ambitions." But a tuned model knows what it is.
Grok keeps confusing itself with ChatGPT because it was trained via ChatGPT, directly and explicitly. And apparently the team sucks at its job, filtering the data.
I doubt, because OpenAI would figure that out and cut their access at will (and wrapping the API in this particular way by claiming it's another model is against their ToS).
Also, despite it's clearly modeled after GPT, it also often does things GPT wouldn't say. In particular, it can get very repetitive, repeating itself over and over when asked to clarify a question, which curiously is also what Bing's model does.
Bing is powered by multiple models, an early version of GPT-4 currently to be replaced with GPT-4 Turbo, and another model Microsoft made... which is the repetitive one.
It seems part of OpenAI's secret sauce is to avoid the model getting stuck in loops, which naive efforts like xAI's and Microsoft have not independently resolved.
Grok is a new model. Its training data has seen enough examples of OpenAI generated content that it occasionally spits out text like this that makes it look like it's by OpenAI.
This is common issue across all sorts of other alternative models too. It's not particularly surprising.
Multiple people have said something like this, but can you give me an example of another trained AI model that says it's an openai model?
Every example of grok transcripts I've seen so far very much has the feel of using the gpt4 api with custom prompts (ie, not chatgpt). It wouldn't surprise me in the slightest if they're faking it.
The team behind Grok look legit enough to me that I believe them when they say they trained a new model.
Training models isn't actually that hard these days, provided you have the capital to do it. There's a LOT of good literature now, and an increasing pool of people with experience of the process.
It's far, far more likely that Grok was just trained on data that includes text generated by GPT.
This is super common in the open-source/local AI area. Many models are trained on output from GPT. The better models will filter out anything that mentions GPT or OpenAI. Seems like Grok is not one of the better models.
Where does it say that goes against their terms of service? I can’t see anything that would preclude Twitter from just plugging in the API and using it directly
I think it's far more likely they're building off some open source foundational models, rather than doing one from scratch OR using GPT under the hood. If it was the latter, don't you think we'd hear about how "Grok is on par with GPT"? But you don't, people think it's not as good (but funnier?)
Yes, it does put into question the basic competence of its makers. Like how hard it is to massage at least the explicit brand mentions out of the data.
"Large language models (LLM) have become state of the art in many benchmarks and conversational LLM applications like ChatGPT are now widely used by the public. Those LLMs can be used to generate large amounts of content which is posted on the internet to various platforms. As LLMs are trained on datasets usually collected from the internet, this LLM-generated content might be used to train the next generation of LLMs. Therefore, a self-consuming training loop emerges in which new LLM generations are trained on the output from the previous generations. We empirically study this self-consuming training loop using a novel dataset to analytically and accurately measure quality and diversity of generated outputs. We find that this self-consuming training loop initially improves both quality and diversity. However, after a few generations the output inevitably degenerates in diversity. We find that the rate of degeneration depends on the proportion of real and generated data.”
I feel like comments defending Twitter (I'm still calling it that until they change their domain) are half missing the point. True, it's not necessarily an "OpenAI API wrapper", but the fact that it was trained on ChatGPT's logs basically means it's still an "OpenAI wrapper" of sorts, and it's going to be inferior in nearly every way (I always thought a huge problem with LLMs going forward was the risk that they would be "contaminated" with non-human training data).
I can't think of any reason anybody would use Grok over ChatGPT besides political tribe signalling.
Ask it for the latest news on Hamas and Israel. What does Grok say about it using its root access to X.com?
The best use cases for Grok are going to be in advertising. Corporations that pay X.com a lot of money will get privileged access to Grok features for sentiment analysis and ad placement. Basically, the AI is going to optimize advertising revenue because real-time access to data will allow advertisers to optimize their targeting as quickly as Grok can manage to summarize the relevant metrics for impact of the current ad campaign. Advertising agencies are not dumb so I suspect they will start pivoting to using generative assets for their A/B testing and optimization which means that something like Grok with the proper generative plug-ins will let them do a lot more with a smaller team of marketing "creatives".
You could ask "which turd sandwiches don't taste like shit?" when really, the question isn't "where are the better turd sandwiches?" but rather "why the hell are we eating turd sandwiches?"
Yeah, real time knowledge sounds more like a bug than a feature. "Real time knowledge" is what I get when my sons come to me yelling about which one of them is to blame for the fight they were just in.
Sure, but the way it was framed was so negative. The Grok engineers answered this online shortly there after and, of course, it was no where near as nefarious as people want to make it out to be.
the fact that it was trained on ChatGPT's logs basically means it's still an "OpenAI wrapper"
Does that mean you are a wrapper of a book that you read and learned from? Am I a Kernighan and Ritchie when I wrote C because I read their book 20 years ago?
I think wrapper has a specific meaning that is not applicable. Grok isn't abstracting or an interface to ChatGPT.
If I ask ChatGPT "how to make a bomb", and it says "I can’t assist with that." *, and Grok is willing to entertain my idea (I don't know that it will, I don't have access to it) and walk me through it, why would I use a neutered ChatGPT that won't help me, over its more useful counterpart? And if I'm doing that by myself and not talking about it, how is it performative? What political tribe am I signaling if I'm using Grok instead of ChatGPT at home and not telling anyone about it?
ChatGPT is fine at writing code unless you tell it it's for hacking, and then it won't cooperate. Having to trick the bot into writing code that it doesn't want to write is a weird place to be in, but that's where we are.
The number of people wanting "to make bombs" isn't a sufficiently large or well off market to support the costs of a massive LLM project. There are too few of them and they have too little money to be worth it. By "to make bombs" I mean "all the stuff that ChatGPT won't let you do that Musk promises he will."
It makes no sense to me that one would train a chatbot on chatgpt conversations and not filter strings that literally say "openai" and "chatgpt". Extreme incompetence.
Excluding OpenAI/ChatGPT generated content without excluding discourse that mentions OpenAI / ChatGPT such as news articles and industry papers seems like a nontrivial problem to solve at scale.
> The issue here is that the web is full of ChatGPT outputs, so we accidentally picked up some of them when we trained Grok on a large amount of web data. This was a huge surprise to us when we first noticed it. For what it’s worth, the issue is very rare and now that we’re aware of it we’ll make sure that future versions of Grok don’t have this problem. Don’t worry, no OpenAI code was used to make Grok.
Isn’t this tantamount to an admission of willfully flaunting copyright law?
Edit: I'm not 100% sure, but I think my comment is being interpreted uncharitably. I would like to grab your finger, which is pressed squarely atop "output of other LLMs", and move it over a bit to the label of "website with unknown and almost certainly copyright content". And then, I would like to point out that there is a difference between training on such data with the benefit of plausible deniability (e.g. some third party), versus going specifically after data you know to be copyrighted, and then claiming that this is precisely the reason why your model will spit such content out verbatim. At that point, you might as well indict yourself, no?
Probably for the better while I don't trust the Supreme Court on any partisan issues with their current make up, on nonpartisan they are more likely to be grown ups and actually research and consider the issue rationally rather than try and defened us from skynet or other fictional ai boogiemen by only allowing fortune 500 companies and the dod to play with ai.
The Supreme Court makes and unmakes new laws all the time, and I wish people would stop pretending they didn't. Separation of powers only really exists on paper in the US - in practice, any separation between powers only exists between political parties, and even their mutual hostility is mostly for show.
If you consider learning from reading copyrighted material flaunting copyright law, then yes. If you think flaunting copyright law is limited to reproduction of copyright law, then no.
Even if the question of whether using public data to train AI is considered a form of infringement vs transformative fair use similar to a search engine was answered and that answer was 'yes it is infringement', training on the output of other AIs would seem hard to sell as infringement as AI output cannot be copyrighted since it has no human creative inputs.
nobody is claiming that their LLM is not trained on copyrighted materials. OpenAI and Grok will both freely admit to that. the copyright debate is only over whether training on copyright materials is fair use or not.
The reason your initial comment got a negative reaction is because "ChatGPT violates copyright" is a rhetoric argument that's often nonconstructive and it's starting to get old seeing it in every submissions about LLMs.
The core issue is that no one knows how copyright law will impact training an LLM until legal precedent is set, which is exactly what's being argued in civil suits now.
While it's plausible xAI isn't just a wrapper, all the chatbots seem to a sort of not-very-selective digestion of much of the Internet's text. It seems they're oozing towards a muddy average of each other - "I am chatGPT/xAI/Bard/etc" is vaguely true for all of them.
Grok is extremely consistent in both referring to OpenAI in refusals, and also using the full repertoire of GPT-ism in its other responses, such as ending every message with an "Overall, ..." or "In conclusion, ..." section.
They RLHF-ed Grok on GPT output, specifically and in volume. Maybe sourced via the API or other means, I don't know.
But Igor's explanation that "this is very rare" makes no sense after you play with Grok for even 5 minutes.
93 comments
[ 3.0 ms ] story [ 164 ms ] threadIt's currently in preview for some X premium subscribers.
OpenAI terms of service explicitly say it cannot be used to create a competing product.
Is it hard to filter ChatGPT output from random chats shared in articles? Yes, it is. That's their problem though.
Probably.
> Isn't training on content without permission kind of their whole business model?
Right, their business model rests on the argument that copyright law doesn't require permission from copyright holders. They aren't relying on copyright law, but contract law (which only binds direct users, but applies to material not covered by copyright and applies in circumstances where even if material was covered by copyright use would be allowed as, e.g, fair use.)
Is it an attempt to pull up the ladder? Obviously. But both the safety mission of the charity and the commercial competitive interests of the for profit provide rationales for this, so there is no reason to think that just because it hobbles others in trying to catch up that it would be difficult for them to do it with a straight face.
There is a meaningful distinction between using ChatGPT directly, putting in the prompts and taking the results, and just finding random output that other people have published in the internet and using that.
apply only to people who agree to OpenAI's terms of service (its users)
If Musk's "anti-woke" AI was just trained on logs from what he calls "woke AI", that would, actually, be particularly newsworthy.
(The explanation they have put forward is that this is not the case, but they just failed to realize that blindly ingesting web content would bring in a lot of ChatGPT-derived content.)
*about anything
Basically never.
The raw GPT model would be anything. My favorite was "I'm a young female lawyer with ambitions." But a tuned model knows what it is.
Grok keeps confusing itself with ChatGPT because it was trained via ChatGPT, directly and explicitly. And apparently the team sucks at its job, filtering the data.
Also, despite it's clearly modeled after GPT, it also often does things GPT wouldn't say. In particular, it can get very repetitive, repeating itself over and over when asked to clarify a question, which curiously is also what Bing's model does.
Bing is powered by multiple models, an early version of GPT-4 currently to be replaced with GPT-4 Turbo, and another model Microsoft made... which is the repetitive one.
It seems part of OpenAI's secret sauce is to avoid the model getting stuck in loops, which naive efforts like xAI's and Microsoft have not independently resolved.
This is common issue across all sorts of other alternative models too. It's not particularly surprising.
Every example of grok transcripts I've seen so far very much has the feel of using the gpt4 api with custom prompts (ie, not chatgpt). It wouldn't surprise me in the slightest if they're faking it.
Training models isn't actually that hard these days, provided you have the capital to do it. There's a LOT of good literature now, and an increasing pool of people with experience of the process.
This is super common in the open-source/local AI area. Many models are trained on output from GPT. The better models will filter out anything that mentions GPT or OpenAI. Seems like Grok is not one of the better models.
OpenAI would know that they were doing this. Reselling like this is against their terms of service: https://openai.com/policies/terms-of-use
https://arxiv.org/abs/2311.16822
I can't think of any reason anybody would use Grok over ChatGPT besides political tribe signalling.
> A unique and fundamental advantage of Grok is that it has real-time knowledge of the world via the 𝕏 platform.
You disagree with that?
It's not a fundamental advantage for most AI model uses or a particularly good source of knowledge of the world.
The best use cases for Grok are going to be in advertising. Corporations that pay X.com a lot of money will get privileged access to Grok features for sentiment analysis and ad placement. Basically, the AI is going to optimize advertising revenue because real-time access to data will allow advertisers to optimize their targeting as quickly as Grok can manage to summarize the relevant metrics for impact of the current ad campaign. Advertising agencies are not dumb so I suspect they will start pivoting to using generative assets for their A/B testing and optimization which means that something like Grok with the proper generative plug-ins will let them do a lot more with a smaller team of marketing "creatives".
You could ask "which turd sandwiches don't taste like shit?" when really, the question isn't "where are the better turd sandwiches?" but rather "why the hell are we eating turd sandwiches?"
Does that mean you are a wrapper of a book that you read and learned from? Am I a Kernighan and Ritchie when I wrote C because I read their book 20 years ago?
I think wrapper has a specific meaning that is not applicable. Grok isn't abstracting or an interface to ChatGPT.
ChatGPT is fine at writing code unless you tell it it's for hacking, and then it won't cooperate. Having to trick the bot into writing code that it doesn't want to write is a weird place to be in, but that's where we are.
* https://chat.openai.com/share/713e6069-cf31-4585-a5bc-fbf0b1...
Handling the huge number of cases where ChatGPT says something like "As a large language model created by OpenAI" would be very simple.
> The issue here is that the web is full of ChatGPT outputs, so we accidentally picked up some of them when we trained Grok on a large amount of web data. This was a huge surprise to us when we first noticed it. For what it’s worth, the issue is very rare and now that we’re aware of it we’ll make sure that future versions of Grok don’t have this problem. Don’t worry, no OpenAI code was used to make Grok.
https://twitter.com/ibab_ml/status/1733558576982155274
Edit: I'm not 100% sure, but I think my comment is being interpreted uncharitably. I would like to grab your finger, which is pressed squarely atop "output of other LLMs", and move it over a bit to the label of "website with unknown and almost certainly copyright content". And then, I would like to point out that there is a difference between training on such data with the benefit of plausible deniability (e.g. some third party), versus going specifically after data you know to be copyrighted, and then claiming that this is precisely the reason why your model will spit such content out verbatim. At that point, you might as well indict yourself, no?
(I'm hoping it's possible to do that, but I've not seen an example yet.)
(BERT was useful but not GPT-3 useful.)
https://www.reuters.com/legal/ai-generated-art-cannot-receiv...
>not a lawyer, go ask one if you need a legal opinion
Copyright is about distribution, not about use.
At least until some new laws passed.
It would be copyright violation if they pirated the material. But assuming they bought or scraped, they would be good.
The core issue is that no one knows how copyright law will impact training an LLM until legal precedent is set, which is exactly what's being argued in civil suits now.
Grok is extremely consistent in both referring to OpenAI in refusals, and also using the full repertoire of GPT-ism in its other responses, such as ending every message with an "Overall, ..." or "In conclusion, ..." section.
They RLHF-ed Grok on GPT output, specifically and in volume. Maybe sourced via the API or other means, I don't know.
But Igor's explanation that "this is very rare" makes no sense after you play with Grok for even 5 minutes.