Primarily training data. A lot of times it’s a hedge to show two things and let the user decide which fits their framing best
Most of the large models collected data from marketing speak, opinion posts, etc (the whole internet basically). I feel like it’s a common sales pitch I’ve heard many times over - now it’s just compounded with everyone having access to the same pattern generator clanker
Remember in Jurassic Park where the dinosaur DNA was extracted from mosquitoes trapped in amber? That wasn't necessarily the best DNA, just the easiest to extract. The best writing might be love letters or high school term papers or PRDs meant for internal distribution, and these never get put online to be fed into LLM training sets. Instead we have 20 years of StackOverflow and Reddit posts saying "it's not a bug, it's a feature" or "Die Hard isn't an action flick, it's a Christmas movie" with 10k upvotes and LLMs learned that's how you get the biggest reaction with the least amount of words. It's not a bug, it's a...you get the point
Such a clean example and a funny reaction. It's true though. Do we get to complain when the problem is we built ourselves a mirror and wow does it reflect!?
It's a valuable pattern because a lot of communication is about drawing distinctions and making delineations. Saying what a thing isn't (despite being close to it) gives you a more clear understanding of what it is.
I wouldn't put too much emphasis on pre-training data, there are so many steps afterwards in modern LLM development that completely change the output. Generally, if you're looking at LLM output in 2026 and thinking something along the lines of "this must be what humans were doing with their language in the training data", you need to pause and reconsider, because modern LLMs are much more than machines which repeat what was in the training data.
Unless the American conscious has had a mass amnesia in the past few years, I could swear that pattern is common in American English. It’s probably a bad example but I clearly remember hearing it a lot in the 90s, often on television shopping channels (bored kid is bored) but also in sitcoms and the like.
My American media consumption tapered off to mostly popular movies after the 00s so maybe there was a change after.
7 comments
[ 2.0 ms ] story [ 5.5 ms ] threadMost of the large models collected data from marketing speak, opinion posts, etc (the whole internet basically). I feel like it’s a common sales pitch I’ve heard many times over - now it’s just compounded with everyone having access to the same pattern generator clanker
I wouldn't put too much emphasis on pre-training data, there are so many steps afterwards in modern LLM development that completely change the output. Generally, if you're looking at LLM output in 2026 and thinking something along the lines of "this must be what humans were doing with their language in the training data", you need to pause and reconsider, because modern LLMs are much more than machines which repeat what was in the training data.
My American media consumption tapered off to mostly popular movies after the 00s so maybe there was a change after.