It might be fun to collect the same data if not for any other reason than to note the changes but adding the caveat that it doesn’t represent human output.
The point was it’s getting harder and harder to do that as things get locked down or go behind a massive paywall to either profit off of or avoid being used in generative AI. The places where previous versions got data is impossible to gather from anymore so the dataset you would collect would be completely different, which (might) cause weird skewing.
But that would always be the case. Twitter will not last forever; heck, it may not even be long before an open alternative like Bluesky competes with it. Would be interesting to know what percentage of the original mined data was from Twitter.
I created https://lowbackgroundsteel.ai/ in 2023 as a place to gather references to unpolluted datasets. I'll add wordfreq. Please submit stuff to the Tumblr.
>Low-background steel, also known as pre-war steel[1] and pre-atomic steel,[2] is any steel produced prior to the detonation of the first nuclear bombs in the 1940s and 1950s. Typically sourced from ships (either as part of regular scrapping or shipwrecks) and other steel artifacts of this era, it is often used for modern particle detectors because more modern steel is contaminated with traces of nuclear fallout.[3][4]
> and only available from pre-war / pre-atomic sources.
From the same wiki you linked:
"Since the end of atmospheric nuclear testing, background radiation has decreased to very near natural levels, making special low-background steel no longer necessary for most radiation-sensitive uses, as brand-new steel now has a low enough radioactive signature"
and
"For the most demanding items even low-background steel can be too radioactive and other materials like high-purity copper may be used"
the easiest solution is growing dense vegetation including trees, then using that for things[0] or burying it until we have a better mitigation strategy for atmospheric carbon.
Another solution, and one that, if i weren't such a lazy, is ocean based carbon binding. You can run electricity directly through ocean water and precipitate the carbon out as calcium carbonate, which is both: useful to humans as is and after processing; and useful to the coral reefs and crustaceans/mollusks or whatever in the oceans.
If anyone wants to kick me about a million US dollars, i can make a POC on a used barge with solar panels and as much recycled material as possible, and have that just run off the coast of florida or something. I figure the total cost to get a barge is around a quarter million, all-in[1], the electronics and seawater stuff is about another $150-200 thousand, and the rest is mine for the idea and the lawyers' to get this approved and left alone to do the research.
[0] burning it for heat is fine, as the net CO2 levels will remain constant, but i mean things like houses and boardwalks and boats, furniture, and so on.
[1] could be more, now, the last time i was researching seaworthy barge costs it was between $100,000 and $200,000. I'm hoping there's someone that can donate the barge so i can make the rest more fit for purpose - redundancy, better solar, better mppt, better batteries, better materials for the electrodes (it takes platinum and titanium iirc, i haven't looked at my documents for a long while.)
And in a few million years, the next intelligent life form will examine remains of human texts, and wonder: with all the tools and knowledge they possessed, how could they not have prevented their demise?
I wouldn't be so optimistic. The thing you call "way" is actually just time. Yes, anything humanity does (good or bad) will fade with time. But do we have the amount of time to clean up X (and i don't refer to X as in "formally twitter")?
This is (one of the many) reasons why I care primarily about biodiversity and preventing as many human-caused extinctions as we can. Those are a permanent loss to the beauty and complexity of the universe built up over millions of years, and they are permanent and irreversible.
It's a reference to the practise of scavenging steel from sources that were produced before nuclear testing began, as any steel produced afterwards is contaminated with nuclear isotopes from the fallout. Mostly ship wrecks, and WW2 means there are plenty of those. The pun in question is that his project tries to source text that hasn't been contaminated with AI generated material.
> Low Background Steel (and lead) is a type of metal uncontaminated by radioactive isotopes from nuclear testing. That steel and lead is usually recovered from ships that sunk before the Trinity Test in 1945.
To whomever downvoted parent: please don't act against people brave enough to state that they don't know something.
This is a desired quality, increasingly less present in IT work environments. People afraid of being shamed for stating knowledge gaps are not the folks you want to work with.
I feel like there's a minimum "due diligence" bar to meet though before asking, otherwise it comes across as "I'm too lazy to google the reference and connect the dots myself, but can someone just go ahead and distill a nice summary for me"
In this particular case, I was out of the loop regarding the clever analogy myself. I'm now a tad smarter because someone else expressed lack of understanding, and I learned from responses to this (grayed due to downvotes) comment.
Yeah pay an illustrator if this is important to you.
See a lot of people upset about AI still using AI image generation because it's not in their field so they feel less strongly about it and can't create art themselves anyway, hypocritical either use it or don't but don't fuss over it then use it for something thats convenient for you.
I have updated my comment with "/s" as that is closer to what I've meant. However, seriously, from ethical point of view it is unlikely illustrators were asked or compensated for their work being used for training AI to produce the image.
That's exactly the opposite of what the author wanted IMO. The author no more wants to be a part of this mess. Aggregating these sources would just makes it so much more easier for the tech giants to scrape more data.
The sources are just aggregated. The source doesn't change.
The new stuff generated does (and this is honestly already captured).
This author doesn't generate content. They analyze data from humans. That "from humans" is the part that can't be discerned enough and thus the project can't continue.
The main concerns expressed in Robyn's note, as I read them, seem to be 1) generative AI has polluted the web with text that was not written by humans, and so it is no longer feasible to produce reliable word frequency data that reflects how humans use natural language; and 2) simultaneously, sources of natural language text that were previously accessible to researchers are now less accessible because the owners of that content don't want it used by others to create AI models without their permission. A third concern seems to be that support for and practice of any other NLP approaches is vanishing.
Making resources like wordfreq more visible won't exacerbate any of these concerns.
Congratulations on "shipping", I've had a background task to create pretty much exactly this site for a while. What is your cutoff date? I made this handy list, in research for mine:
2017: Invention of transformer architecture
June 2018: GPT-1
February 2019: GPT-2
June 2020: GPT-3
March 2022: GPT-3.5
November 2022: ChatGPT
You may want to add kiwix archives from before whatever date you choose. You can find them on the Internet Archive, and they're available for Wikipedia, Stack Overflow, Wikisource, Wikibooks, and various other wikis.
:'( I thought I was clever for realising this parallel myself! Guess it's more obvious than I thought.
Another example is how data on humans after 2020 or so can't be separated by sex because gender activists fought to stop recording sex in statistics on crime, medicine, etc.
I wonder if anyone will fork the project. Apart from anything else, the data may still be useful given that we know it is polluted. In fact, it could act as a means of judging the impact of LLMs via that very pollution.
I guess it would be interesting but differentiating pollution from language evolution seems very tricky since getting a non polluted corpus gets harder and harder
One way to tackle it would be to use LLMs to generate synthetic corpuses, so you have some good fingerprints for pollution. But even there I'm not sure how doable that is given the speed at which LLMs are being updated. Even if I know a particular page was created in, say, January 2023, I may no longer be able to try to generate something similar now to see how suspect it is, because the precise setups of the moment may no longer be available.
Arguably it is a form of language evolution. I bet humans have started using "delve" more too, on average. I think the best we can do is look at the trends and think about potential causes.
“Seamless”, “honed”, “unparalleled”, “delve” are now polluting the landscape because of monkeys repeating what ChatGPT says without even questioning what the words mean.
Everything is “seamless” nowadays. Like I am seamlessly commenting here.
Arguably, the meaning of these words evolve due to misuse too.
I see a lot of writing in my day-to-day, and the words that stick out most are things like "plethora" and "utilized". They're not terribly obscure, but they're just 'odd' and, maybe, formal enough to really stick out when overused.
Man the AI folks really wrecked everything. Reminds me of when those scooter companies started just dumping their scooters everywhere without asking anybody if they wanted this.
perhaps germane to this thread, I think the scooter thing was an investment bubble. it was easier to burn investment money on new scooters than to collect and maintain old ones. until the money ran out.
Did they? A lot of then were barely used, got damaged or vandalized, etc. And when the companies folded or communities outlawed the scooters, they end up as trash. I don't believe for a second that the amount of pollutants and greenhouse gasses saved by usage is larger than the amount produced by manufacturing, shipping and trashing all those scooters.
All those writers who'll soon be out of job and/or already are and basically unhireable for their previous tasks should be paid for by the AI hyperscalers to write anything at all on one condition: not a single sentence in their works should be created with AI.
(I initially wanted to say 'paid for by the government' but that'd be socialising losses and we've had quite enough of that in the past.)
Yes. This was always the failure with the argument that copyright was the relevant issue... Once the model was proven out, we knew some wealthy companies would hire humans to generate the training data that the companies could then own in whole, at the relative expense of all other humans that didn't get paid to feed the machines.
This idea could also be extended to domains like Art. Create new art styles for AI to learn from. But in future, that will also get automated. AI itself will create art styles and all humans would do is choose whether something is Hot or Not. Sort of like art breeder.
There are already several companies doing this - I do occasional contract work for a couple -, and paying rates sometimes well above what an average earning writer can expect elsewhere. However, the vast majority of writers have never been able to make a living from their writing. The threshold to write is too love, too many people love it, and most people read very little.
Transformers read a lot during training, it might actually be beneficial for the companies to the point those works never see the light of day, only machines would read them. That's so dystopian I'd say those works should be published so they eventually get into the public domain.
People have been paid to generate noise for a decade+ now. Garbage in, garbage out will always be true.
Next token-seeking is a solved problem. Novel thinking can be solved by humans and possibly by AI soon, but adding more garbage to the data won't improve things.
_Thank you_. I read this story probably around 1980 (I think in a magazine that was subsequently trashed or garage-saled), and I have spent my adult life remembering the bones of the story, but not the author or the title.
> As one example, Philip Shapira reports that ChatGPT (OpenAI's popular brand of generative language model circa 2024) is obsessed with the word "delve" in a way that people never have been, and caused its overall frequency to increase by an order of magnitude.
The fun thing is that while GPTs initially learned from humans (because ~100% of the content was human-generated), future humans will learn from GPTs, because almost all available content would be GPT-generated very soon.
This will surely affect how we speak. It's possible that human language evolution could come to a halt, stuck in time as AI datasets stop being updated.
In the worst case, we will see a global "model collapse" with human languages devolving along with AI's, if future AIs are trained on their own outputs...
> I'd love to see by how much the use of world "delve" has increased since 2021...
There are charts / graphs in the link, both since 2021, and since earlier.
The final graph suggests the phenomenon started earlier, possibly correlated in some way to Malaysian / Indian usages of English.
It does seem OpenAI's family of GPTs as implemented in ChatGPT unspool concepts in a blend of India-based-consultancy English with American freshmen essay structure, frosted with superficially approachable or upbeat blogger prose ingratiatingly selling you something.
Anthropic has clearly made efforts to steer this differently, Mistral and Meta as well but to a lesser degree.
I've wondered if this reflects training material (the SEO is ruining the Internet theory), or is more simply explained by selection of pools of Hs hired for RLHF.
I agree in general but the web was already polluted by Google's unwritten SEO rules. Single-sentence paragraphs, multiple keyword repetitions and focus on "indexability" instead of readability, made the web a less than ideal source for such analysis long before LLMs.
It also made the web a less than ideal source for training. And yet LLMs were still fed articles written for Googlebot, not humans. ML/LLM is the second iteration of writing pollution. The first was humans writing for corporate bots, not other humans.
Yes but not quite as far as you imply. The training data is weighted by a quality metric, articles written by journalists and wikipedia contributors are given more weight than Aunt May's brownie recipe and corpoblogspam.
It certainly feels like the amount of regurgitated, nonsensical, generated content (nontent?) has risen spectacularly specifically in the past few years. 2021 sounds about right based on just my own experience, even though I can't point to any objective source backing that up.
SEO grifters have fully integrated AI at this point, there are dozens of turn-key "solutions" for mass-producing "content" with the absolute minimum effort possible. It's been refined to the point that scraping material from other sites, running it through the LLM blender to make it look original, and publishing it on a platform like Wordpress is fully automated end-to-end.
Or check out "money printer" on github: a tongue in cheek mashup of various tools to take a keyword as input and produce a youtube video with subtitles and narration as output.
I personally am yet to see this beyond some slop on youtube. And I am here for the AI meme videos. I recognize the dangers of this, all I am saying is that I don't feel the effect, yet.
There's been a ton of low-rent listicle writing out there for ages. Certainly not new in the past few years. I admit I don't go on YouTube much and don't even have a tiktok account so it's possible there's a lot of newer lousy content I'm not really exposed to.
It seems to me that the fact it's so cheap and relatively easy for people with dreams of becoming wealthy influencers to put stuff out there has more to do with the flood of often mediocre content than AI does.
Of course the vast majority don't have much real success and get on with life and the crank turns and a new generation perpetuates the cycle.
LLMs etc. may make things marginally easier but there's no shortage of twenty somethings with lots of time imagining riches while making pennies.
I'm seeing it a lot when searching for some advice in a well-defined subject, like, say, leatherworking or sewing (or recipes, obviously). Instead of finding forums with hobbyists, in-depth blog posts, or manufacturers advice pages, increasingly I find articles which seem like natural language at first, but are composed of paragraphs and headers repeating platitudes and basic tips. It takes a few seconds to realize the site is just pushing generated articles.
Increasingly I find that for in-depth explanations or tutorials Youtube is the only place to go, but even there the search results can lead to loads of videos which just seem… off. But at least those are still made by humans.
Looking forward to watch perfect generated videos. We need so much more power and chips but it’s completely worth it. After that? Maybe generated videogames. But the video stuff will be awesome and changing the video dominated social media content for ever. Virtual headsets will become useful finally generating anything you want to see and jump tru space and time.
I do hope it catches on! I did come up with this myself, but I really doubt I'm the only one — and indeed: Wiktionary lists it already with a 2023 vintage:
Aunt may's brownie recipe (or at least her thoughts on it) are likely something you'd want if you want to reflect how humans use language. Both news-style and encyclopedia-style writing represent a pretty narrow slice.
That's why search engines rated them highly, and why a million spam sites cropped up that paid writers $1/essay to pretend to be Aunt May, and why today every recipe website has a gigantic useless fake essay in front of their copypasted made up recipes.
Ok, but what i said is true regardless of SEO, and that SEO has also fed back into english before LLMs were a thing. If you only train on those subsets you'll also end up with a chatbot that doesn't speak in a way we'll identify as natural english.
I hate how looking for recipes has become so… disheartening. Online recipes are fine for reputable sources like newspapers where professional recipe writers are paid for their contributions, but searching for some Aunt May's recipe for 'X' in the big ocean of the internet is pointless — too much raw sewage dumped in.
It sucks, because sharing recipes seemed like one of those things the internet could be really good at.
And they're all flooded with low effort trash and useless.
The only remaining reliable source - now that many newspapers are axing the remaining staff in favour of LLMs - is pre-2020 print cookbooks. Anything online or printed later must be assumed to be tainted, full of untested sewage and potentially dangerous suggestions.
I absolutely love Chef John. Great recipes and the cadence of his speech on YouTube (foodwishes) is very soothing, while he cooks up something amazing. If you're a home cook I highly recommend his recipes and his channel.
The wife and I use the internet for recipe ideas... but we hardly ever follow them directly anymore. We're no formally-trained chefs but we've been home cooks for over 20 years now, and so many of them are self-evidently bad, or distinctly suboptimal. The internet chef's aversion to flavor is a meme with us now; "add one-sixty-fourth of a teaspoon of garlic powder to your gallon of soup, and mix in two crystals of table salt". Either that or they're all getting some seriously potent spices all the time and I'd like to know where they shop because my spices are nowhere near as powerful as theirs.
It's not just online recipes, but cookbooks written for the Better Home & Gardens crowd. The ones who write "curry powder" (and mean the yellow McCormick stuff which is so bland as to have almost no flavour) or call for one clove of garlic in their recipe.
I joke with folks that my assumption with "one clove of garlic" is that they really mean "one head of garlic" if you want any flavour. (And if the recipe title has "garlic" in it and you are using one clove, you’re lying.)
It's interesting to search for recipes in other languages and not find junk as we do in English.
I read Spanish and Italian fluently and stumble my way through Japanese (with translation). It's easier to find a good recipe in these languages, provided you can find the ingredients or substitutes.
And here I thought my defacement of printed recipes by bracketing everything that goes together at each stage was just me. There are, well, maybe not dozens but at least two of us! Saves a lot of bowls when you know without further checking that you can, say, just dump the flour and sugar, butter and eggs into the big bowl without having to prepare separately because they're in the "1: big bowl" bracket.
Depends on what you’re doing. For best cookies, you want to cream the butter with the sugar, then add the eggs, and finally add the flour. If you’re interested and can find one, it’s worth taking a vegan baking class. You learn a lot about ingredient substitutions for baking, about what the different non-vegan ingredients are doing that you have to compensate for…and it does something that I’ve only recently started seeing happen in non-vegan baking recipes: it separates the wet ingredients from the dry ingredients.
That is, when baking, you can usually (again, exceptions for creaming the sugar in butter, etc.) take all of your dry ingredients and mix/sift them together, and then you pour your wet ingredients in a well you’ve made in the dry ingredients (these can also usually be mixed together).
No need to cakesplain, that was an example with three ingredients of the top of my head, very, very obviously the exact ingredients and bracket assignments vary depending on what you are making.
But for shortbread or fork biscuits those three could indeed all go in the bowl in one go (but that one admittedly doesn't really need a bracket because the recipe is "put in bowl, mix with hands, bake").
I love the table-diagrams at the end. I've never seen anything like that until now and it really seems useful for visualization of the recipe and the sequence of steps.
Interestingly my wife has been writing recipes on post-it notes for years in that same style, with arrows instead of tables. And she's the opposite to an Engineer, a psychologist (interest in people vs objects).
When I saw them, they blew my mind. Short to store and easy to understand.
> The training data is weighted by a quality metric
At least in Googles case, they're having so much difficulty keeping AI slop out of their search results that I don't have much faith in their ability to give it an appropriately low training weight. They're not even filtering the comically low-hanging fruit like those YouTube channels which post a new "product review" every 10 minutes, with an AI generated thumbnail and AI voice reading an AI script that was never graced by human eyes before being shat out onto the internet, and is of course always a glowing recommendation since the point is to get the viewer to click an affiliate link.
Google has been playing the SEO cat and mouse game forever, so can startups with a fraction of the experience be expected to do any better at filtering the noise out of fresh web scrapes?
I don’t think they were talking about the quality of Google search results. I believe they were talking about how the data was processed by the wordfreq project.
> Google has been playing the SEO cat and mouse game forever, so can startups with a fraction of the experience be expected to do any better at filtering the noise out of fresh web scrapes?
Google has been _monetizing_ the SEO game forever. They chose not to act against many notorious actors because the metric they optimize for is ad revenue and and those sites were loaded with ads. As long as advertisers didn’t stop buying, they didn’t feel much pressure to make big changes.
A smaller company without that inherent conflict of interest in its business model can do better because they work on a fundamentally different problem.
Google has those problems because the company's revenue source (Ads) and the thing that puts it on the map (Search) are fundamentally at odds with one another.
A useful Search would ideally send a user to the site with the most signal and the fewest noise. Meanwhile, ads are inherently noise; they're extra pieces of information inserted into a webpage that at best tangentially correlate to the subject of a page.
Up until ~5 years ago, Google was able to strike a balance on keeping these two stable; you'd get results with some Ads but the signal generally outweighed the noise. Unfortunately from what I can tell from anecdotes and courtroom documents, the Ad team at Google has essentially hijacked every other aspect of the company by threatening that yearly bonuses won't be given out if they don't kowtow to the Ad teams wishes to optimize ad revenue somewhere in 2018-2019 and has no sign of stopping since there's no effective competition to Google. (There's like, Bing and Kagi? Nobody uses Bing though and Kagi is only used by tech enthusiasts. The problem with Google is that to copy it, you need a ton of computing resources upfront and are going up against a company with infinitely more money and ability to ensure users don't leave their ecosystem; go ahead and abandon Search, but good luck convincing others to give up say, their Gmail account, which keeps them locked to Google and Search will be there, enticing the average user.)
Google has absolutely zero incentive to filter out generative AI junk from their search results outside the amount of it that's damaging their PR since most of the SEO spam is also running Google Ads (since unless you're hosting adult content, Google's ad network is practically the only option). Their solution therefore isn't to remove the AI junk, but to instead reduce it enough to the degree where a user will not get the same type of AI junk twice.
My understanding is that Google Ads are what makes Google Search unassailable.
A search engine isn't a two-sided market in itself but the ad network that supports it is. A better search engine is a technological problem, but a decently paying ad network is a technological problem and a hard marketing problem.
>At least in Googles case, they're having so much difficulty keeping AI slop out of their search results that I don't have much faith in their ability to give it an appropriately low training weight.
I've noticed that lately. It used to be the top google result was almost always what you needed. Now at the top is an AI summary that is pretty consistently wrong, often in ways that aren't immediately obvious if you aren't familiar with the topic.
> those YouTube channels which post a new "product review" every 10 minutes, with an AI generated thumbnail and AI voice reading an AI script that was never graced by human eyes before being shat out onto the internet
The problem is that, of the signals you mention,
• the highly-informative ones (posting a new review every 10 minutes, having affiliate links in the description) are contextual — i.e. they're heuristics that only work on a site-specific basis. If the point is to create a training pipeline that consumes "every video on the Internet" while automatically rejecting the videos that are botspam, then contextual heuristics of this sort won't scale. (And Google "doesn't do things that don't scale.")
• and, conversely, the context-free signals you mention (thumbnail looks AI-generated, voice is synthesized) aren't actually highly correlated with the script being LLM-barf rather than something a human wrote.
Why? One of the primary causes is TikTok (because TikTok content gets cross-posted to YouTube a lot.) TikTok has a built-in voiceover tool; and many people don't like their voice, or don't have a good microphone, or can't speak fluent/unaccented English, or whatever else — so they choose to sit there typing out a script on their phone, and then have the AI read the script, rather than reading the script themselves.
And then, when these videos get cross-posted, usually they're being cross-posted in some kind of compilation, through some tool that picks an AI-generated thumbnail for the compilation.
Yet, all the content in these is real stuff that humans wrote, and so not something Google would want to throw away! (And in fact, such content is frequently a uniquely-good example of the "gen-alpha vernacular writing style", which otherwise doesn't often appear in the corpus due to people of that age not doing much writing in public-web-scrapeable places. So Google really wants to sample it.)
Reminds me of a Google search I did yesterday: “Hezbollah” yields a little info box with headings “Overview”, “History”, “Apps” and “Return policy”.
I’m guessing that the association between “pagers” and “Hezbollah” ended up creating the latter two tabs, but who knows. Maybe some AI video out there did a product review of Hezbollah.
We do know that the open web consitutes the bulk of the trainig data, although we don't get to know the specific webpages that got used. Plus some more selected sources, like books, of which again we only know that those are books but not which books were used. So it's just a matter of probability that there was a good amount of SEO spam as well.
This feels like a second, magnitudes larger Eternal September. I wonder how much more of this the Internet can take before everyone just abandons it entirely. My usage is notably lower than it was in even 2018, it's so goddamn hard to find anything worth reading anymore (which is why I spend so much damn time here, tbh).
I think it's an arms race, but it's an open question who wins.
For a while I thought email as a medium was doomed, but spammers mostly lost that arms race. One interesting difference is that with spam, the large tech companies were basically all fighting against it. But here, many of the large tech companies are either providing tools to spammers (LLMs) or actively encouraging spammy behaviors (by integrating LLMs in ways that encourage people to send out text that they didn't write).
I'm not saying this is impossible but that's going to be an uphill sell for me as a concept. According to some quick stats I checked I'm getting roughly 600 emails per day, about 550 of which go directly to spam filtering, and of the remaining 50, I'd say about 6 are actually emails I want to be receiving. That's an impressive amount overall for whoever built this particular filter, but it's also still a ton of chaff to sort wheat from and as a result I don't use email much for anything apart from when I have to.
Like, I guess that's technically usable, I'm much happier filtering 44 emails than 594 emails? But that's like saying I solved the problem of a flat tire by installing a wooden cart wheel.
It's also worth noting there that if I do have an email thats flagged as spam that shouldn't be, I then have to wade through a much deeper pond of shit to go find it as well. So again, better, but IMO not even remotely solved.
I’m not sure what you’ve done to get that level of spam, but I get about 10 spam emails a day at most and that’s across multiple accounts including one that I’ve used for almost 30 years and had used on Usenet which was the uber-spam magnet. A couple newer (10–15 year old) addresses which I’ve published on webpages with mailto links attract maybe one message a week and one that I keep for a specialized purpose (fiction and poetry submissions) gets maybe one to two messages per year, mostly because it’s of the form example@example.com so easily guessed by enterprising spammers.
Looking at the last days’ spam¹ I have three 419-style scams (widows wanting to give away their dead husbands’ grand piano or multi-million euro estate) and three phishing attempts. There are duplicate messages in each category.
About fifteen years ago, I did a purge of mailing list subscriptions and there’s very little that comes in that I don’t want, most notably a writer who’s a nice guy, but who interpreted my question about a comment he made on a podcast as an invitation to be added to his manually managed email list and given that it’s only four or five messages a year, I guess I can live with that.
⸻
1. I cleaned out spam yesterday while checking for a confirmation message from a purchase.
I'm having a hard time finding reliably sourced statistics here, but I suspect you're an outlier. My personal numbers are way better, both on Gmail and Fastmail, despite using the same email addresses for decades.
Another problem with this arms race is that spam emails actually are largely separable from ham emails for most people... or at least they were, for most of their run. The thousandth email that claims the UN has set aside money for me due to my non-existent African noble ancestry that they can't find anyone to give it to and I just need to send the Thailand embassy some money to start processing my multi-million yuan payout and send it to my choice of proxy in Colombia to pick it up is quite different from technical conversation about some GitHub issue I'm subscribed to, on all sorts of metrics.
However, the frontline of the email war has shifted lately. Now the most important part of the war is being fought over emails that look just like ham, but aren't. Business frauds where someone convinces you that they are the CEO or CFO or some VP and they need you to urgently buy this or that for them right now no time to talk is big business right now, and before you get too high-and-mighty about how immune you are to that, they are now extremely good at looking official. This war has not been won yet, and to a large degree, isn't something you necessarily win by AI either.
I think there's an analogy here to the war on content slop. Since what the content slop wants is just for you to see it so they can serve you ads, it doesn't need anything else that our algorithms could trip on, like links to malware or calls to action to be defrauded, or anything else. It looks just like the real stuff, and telling that it isn't could require a human rather vast amounts of input just to be mostly sure. Except we don't have the ability to authenticate where it came from. (There is no content authentication solution that will work at scale. No matter how you try to get humans to "sign their work" people will always work out how to automate it and then it's done.) So the one good and solid signal that helps in email is gone for general web content.
I don't judge this as a winning scenario for the defenders here. It's not a total victory for the attackers either, but I'd hesitate to even call an advantage for one side or the other. Fighting AI slop is not going to be easy.
The fight against spam email also led to mass consolidation of what was supposed to be a decentralised system though. Monoliths like Google and Microsoft now act as de-facto gatekeepers who decide whether or not you're allowed to send emails, and there's little to no transparency or recourse to their decisions.
There's probably an analogy to be made about the open decentralised internet in the age of AI here, if it gets to the point that search engines have to assume all sites are spam by default until proven otherwise, much like how an email server is assumed guilty until proven innocent.
At some point though you have to acknowledge that a specific use of language belongs to the medium through which you're counting word frequencies. There are also specific writing styles (including sentence/paragraph sizes, unnecessary repetitions, focusing on other metrics than readability) associated with newspapers, novels, e-mails to your boss, anything really. As long as text was written by a human who was counting on at least some remote possibility that another human might read it, this is way more legitimate use of language than just generating it with a machine.
> I agree in general but the web was already polluted by Google's unwritten SEO rules. Single-sentence paragraphs, multiple keyword repetitions and focus on "indexability" instead of readability, made the web a less than ideal source for such analysis long before LLMs.
Blog spam was generally written by humans. While it sucked for other reasons, it seemed fine for measuring basic word frequencies in human-written text. The frequencies are probably biased in some ways, but this is true for most text. A textbook on carburetor maintenance is going to have the word "carburetor" at way above the baseline. As long as you have a healthy mix of varied books, news articles, and blogs, you're fine.
In contrast, LLM content is just a serpent eating its own tail - you're trying to build a statistical model of word distribution off the output of a (more sophisticated) model of word distribution.
SEO text carefully tuned to tf-idf metrics and keyword stuffed to them empirically determined threshold Google just allows should have unnatural word frequencies.
LLM content should just enhance and cement the status quo word frequencies.
Outliers like the word "delve" could just be sentinels, carefully placed like trap streets on a map.
To be pedantic, the F in TFA has the same meaning as the F in RTFM.
It’s the same origin. On Slashdot (the HN of the early 00’s) people would admonish others to RTFA. Then they started using it as a referent: TFA was the thing you were supposed to have read.
1. People don't generally use the (big, whole-web-corpus-trained) general-purpose LLM base-models to generate bot slop for the web. Paying per API call to generate that kind of stuff would be far too expensive; it'd be like paying for eStamps to send spam email. Spambot developers use smaller open-source models, trained on much smaller corpuses, sized and quantized to generate text that's "just good enough" to pass muster. This creates a sampling bias in the word-associational "knowledge" the model is working from when generating.
2. Given how LLMs work, a prompt is a bias — they're one-and-the-same. You can't ask an LLM to write you a mystery novel without it somewhat adopting the writing quirks common to the particular mystery novels it has "read." Even the writing style you use in your prompt influences this bias. (It's common advice among "AI character" chatbot authors, to write the "character card" describing a character, in the style that you want the character speaking in, for exactly this reason.) Whatever prompt the developer uses, is going to bias the bot away from the statistical norm, toward the writing-style elements that exist within whatever hypersphere of association-space contains plausible completions of the prompt.
3. Bot authors do SEO too! They take the tf-idf metrics and keyword stuffing, and turn it into training data to fine-tune models, in effect creating "automated SEO experts" that write in the SEO-compatible style by default. (And in so doing, they introduce unintentional further bias, given that the SEO-optimized training dataset likely is not an otherwise-perfect representative sampling of writing style for the target language.)
if you are a sales & marketing intern, have a potato laptop and $100 budget to spend on seo, you aren't going to be self hosting anything even if you know what that means.
This is about high-volume blog/news-spam created specifically to serve ads and affiliate links, not about occasional content marketing for legitimate companies.
There are two costs associated with using a hosted inference platform: the OpEx of API calls, and the CapEx of setting up an account in the first place. This second cost is usually trivial, as it just requires things any regular person already has: an SSO account, a phone number for KYC, etc.
But, insofar as your use-case is against the TOUs of the big proprietary inference platforms, this second cost quickly swamps the first cost. They keep banning you, and you keep having to buy new dark-web credentials to come back.
Given this, it’s a lot cheaper and more reliable — you might summarize these as “more predictable costs” — to design a system around a substrate whose “immune system” won’t constantly be trying to kill the system. Which means either your own hardware, or a “being your own model” inference platform like RunPod/Vast/etc.
(Now consider that there are a bunch of fly-by-night BYO-model hosted inference platforms, that are charging unsustainable flat-rate subscription prices for use of their hardware. Why do these exist? Should be obvious now, given the facts already laid out: these are people doing TOU-violating things who decided to build their own cluster for doing them… and then realized that they had spare capacity on that cluster that they could sell.)
This makes sense. But now I’m wondering if people here are speaking from experience or reasoning their way into it. Like are there direct reports of which models people are using for blogspam, or is it just what seems rational?
But you can already see it with Delve. Mistral uses "delve" more than baseline, because it was trained on GPT.
So it's classic positive feedback. LLM uses delve more, delve appears in training data more, LLM uses delve more...
Who knows what other semantic quirks are being amplified like this. It could be something much more subtle, like cadence or sentence structure. I already notice that GPT has a "tone" and Claude has a "tone" and they're all sort of "GPT-like." I've read comments online that stop and make me question whether they're coming from a bot, just because their word choice and structure echoes GPT. It will sink into human writing too, since everyone is learning in high school and college that the way you write is by asking GPT for a first draft and then tweaking it (or not).
Unfortunately, I think human and machine generated text are entirely miscible. There is no "baseline" outside the machines, other than from pre-2022 text. Like pre-atomic steel.
Oh no, it's been here for quite a while. Our culture is already heavily glued to the machine. The way we express ourselves, the language we use, even our very self-conception originates increasingly in online spaces.
Have you ever seen someone use their smartphone? They're not "here," they are "there." Forming themselves in cyberspace -- or being formed, by the machine.
I think they meant culture in the sense of knowledge that gets passed down from one generation to the next. Not a human culture of using machines, but a machine culture of using human languages.
Consider that the algorithm cannot evolve without human interaction. That's what I'm saying, it's a symbiote to us. If you consider "weights in the Instagram recommendation algorithm" to be "the machine", what we are talking about here has been happening for a long time now and has seen many generations, with each entity influencing the other.
I don't think we'll have true machine culture until we have fully autonomous agents in the wild that are interacting with the world independently on its own terms. Right now the substrate is text which comes from a human mind -- it does not arise naturally from nothing. So the machine is a symbiote for now until we solve some difficult robotics problems.
Hm, it's probably true that recommendation algorithms do something similar already, training on "human likes" that were influenced by the previous generation. But "human language" is a richer medium to carry information.
I don't think you need to be independent or autonomous to develop a culture. And a lot of human culture was passed down over generations without understanding why it worked. We just imitate the behaviour and rituals from our most successful ancestors or role models.
If new LLMs can access the past generation's knowledge of how to please human evaluators, they will use it. It's not a deliberate decision by an "agent", it's just the best text source to copy from. This is a new feedback loop between generations of assistants, and it bypasses whatever the human designer had in mind. Phrases like "it is always best to ask an expert" will pop up just because you tuned the LLM to sound like a helpful assistant, and that's what helpful assistants sound like in the training data. You'd have to actively steer the new generation away from using their ancestral knowledge.
I guess it comes down to what your definition of "culture" is. There is no targeted teaching of the next generation, for example - but is this a requirement? I agree that talking about "machine culture" right now sounds like a stretch, but now I wonder what pieces are actually missing.
Human me just thought it was a good word for this. It implies some irreversible process of mixing, I think that characterizes this process really well.
There were dozens of 20th Century ideological movements which developed their own forms of "Newspeak" in their own native languages. Largely, natural human dialog between native speakers and between those opposed to the prevailing regime recoils violently at stilted, official, or just "uncool" usages in daily vernacular. So I wouldn't be too surprised to see a sharp downtick in the popular use of any word that becomes subject to an LLM's positive-feedback loop.
Far from saying the pool of language is now polluted, I think we now have a great data set to begin to discern authentic from inauthentic human language. Although sure, people on the fringes could get caught in a false positive for being bots, like you or I.
The biggest LLM of them all is the daily driver of all new linguistic innovation: Human society, in all its daily interactions. The quintillions of daily phrases exchanged and forever mutating around the globe - each mutation of phrase interacting with its interlocutor, and each drawing from not the last 500,000 tokens but the entire multi-modal, if you will, experience of each human to date in their entire lives - vastly eclipses anything any hardware could ever emulate given the current energy constraints. Software LLMs are just a state machine stuck in a moment in time. At best they will always lag, the way Stalinist language lagged years behind the patois of average Russians, who invented daily linguistic dodges to subvert and mock the regime. The same process takes place anywhere there is a dominant official or uncool accent or phrasing. The ghetto invents new words, new rhythm, and then it becomes cool in the middle class. The authorities never catch up, precisely because the use of subversive language is humanity's immune system against authority.
If there is one distinctly human trait, it's sniffing out anyone who sounds suspiciously inauthentic. (Sadly, it's also the trait that leads to every kind of conspiracy theorizing imaginable; but this too probably confers in some cases an evolutionary advantage). Sniffing out the sound of a few LLMs is already happening, and will accelerate geometrically, much faster than new models can be trained.
I'm a little more cautious though. I think GPT will be way more integrated, simply because it's useful. Stalinist language was artificial, in the sense that it was basically imposed on you from outside for no good reason. When you wanted to get real stuff done (either talking to close friends, being productive with colleagues, etc) you wouldn't use socialist newspeak because it got in the way. GPT will be imposed by the outside world, but it's actually a useful thing to be able to converse with a language model; you'll do it every day at work, when buying things, when using your phone/PC.
And also, unlike in USSR times, so much of our communication is online and visible. It would not surprise me if we develop a model that can train continuously on the firehose. Text is small. Data rate of every person on earth speaking simultaneously:
- 150 words per minute spoken
- 150 words × (5 characters/word + 1 space) = 150 × 6 = 900 characters per minute
It's crazy to attribute the downfall of the web/search to Google. What does Google have to do with all the genuine open web content, Google's source of wealth, getting starved by (increasingly) walled gardens like Facebook, Reddit, Discord?
I don't see how Google's SEO rules being written or unwritten has any bearing. Spammers will always find a way.
Prior to Google we had Altavista and in those days it was incredibly common to find keywords spammed hundreds of times in white text on a white background in the footer of a page. SEO spam is not new, it's just different.
Don't forget Google's adsense rules which penalized useful straightforward websites and mandated websites be full of "content". Doesn't matter if the "content" is garbage nonsense rambling and excessive word use - it's content and much more likely to be okayed by adsense!
>"Now Twitter is gone anyway, its public APIs have shut down, and the site has been replaced with an oligarch's plaything, a spam-infested right-wing cesspool called X. Even if X made its raw data feed available (which it doesn't), there would be no valuable information to be found there.
>Reddit also stopped providing public data archives, and now they sell their archives at a price that only OpenAI will pay.
>And given what's happening to the field, I don't blame them."
Given just how many AI bots scrape up everything they can, oftentimes ignoring robots.txt or any rate limits (there have been a few complaint threads on HN about that), I can hardly blame the operators of large online services just cutting off data feeds.
Twitter however didn't stop their data feeds due to AI or because they wanted money, they stopped providing them because its new owner does everything he can to hinder researchers specializing in propaganda campaigns or public scrutiny.
Reddit shut their API access down only very recently, after the AI craze went off. Twitter did so right after Musk took over, way before Reddit, way before AI ever went nuts.
X shut down API access in Feb 2023, Reddit shut theirs down at the end of June of the same year. Just barely 6 months apart.
Furthermore, while X had also only announced this in February, Reddit announced their API shutdown just 2 months later in April.
And, to further add to that, X was pretty upfront that they think they have access to a large and powerful dataset in X and didn't want to give it out for free. Reddit used very similar wording when announcing their changes.
Enshittification is accelerating. A good 70% of my Facebook feed is now obviously AI generated images with AI generated text blurbs that have nothing to do with the accompanying images likely posted by overseas bot farms. I'm also noticing more and more "books" on Amazon that are clearly AI generated and self published.
I read that as 3 books per year at first and thought to myself that that was a rather harsh limitation but surely any true respectable author wouldn't be spitting more than that...
...and then I realized you wrote 3 books a day. What the hell.
> A good 70% of my Facebook feed is now obviously AI generated images with AI generated text blurbs that have nothing to do with the accompanying images likely posted by overseas bot farms.
This is a self-inflicted problem, IMO.
Do you just have shitty friends that share all that crap? Or are you following shitty pages?
I use Facebook a decent amount, and I don't suffer from what you're complaining about. Your feed is made of what you make it. Unfollow the pages that make that crap. If you have friends that share it, consider unfriending or at the very least, unfollowing. Or just block the specific pages they're sharing posts from.
Did we (the humans) somehow managed to pollute the internet so much with AI that's it's now barely usable ?
In my opinion the internet can be considered as the equivalent of a natural environment like the earth. it's a space where people share, meet, talk, etc.
I find it astonishing that after polluting our natural environment we know polluted the internet.
> Did we (the humans) somehow managed to pollute the internet so much with AI that's it's now barely usable
If we haven't already, we will be very soon. I'm sure there are people working on this problem, but I think we're starting to hit a very imminent feedback loop moment. Most of human's recorded information is digitized and most of that is generating non-human content at an incredible pace. We've injected a whole lot of noise into our usable data.
I don't know if the answer is more human content (I'm doing my part!) or novel generative content but this interim period is going to cause some medium-term challenges.
I like to think the LLM more-tokens-equals-better era is fading and we're getting into better use of existing data, but there's a very real inflection point we're facing.
That's a nice analogy. Fortunately (un)real estate is easier to manufacture out of thin air online. We have lost some valuable spaces like Twitter and Reddit to some degree though.
There are smaller, gated communities that are still very valuable. You're posting in one. But yes, the open Internet is basically useless now, thanks ultimately to advertising as a business model.
Sure, there's bad actors everywhere, but there's really no incentive to do it here so I don't think it's a problem in the same way it is on the open internet, where slop is actively rewarded.
True, that is maybe too strong a phrase, but I think it's close to accurate. I think the culture & medium provide kind of a self-selecting gate: it's just plain text and links to articles, with the discussion expected by culture to be fairly serious. I think that turns off enough people that it kind of forms its own gate shutting out the people that make "eternal Septembers" happen. But yeah, ultimately, you're right.
The people who make Eternal Septembers happen show up here all the time. They show up here from Reddit and 4chan and elsewhere expecting this forum to be a free-for-all because the word "hacker" is in the title, and are inevitably shocked and disappointed to learn how aggressively moderated and tone-policed this place is.
They're the ones who post racist diatribes under green alt account and anti-vaxx conspiracy theories and complain about wokeness and talk out of their ass about subjects they have no actual expertise in. The ones who think flagging is just a super downvote.
I mean, the premise that simply having a minimalist layout would filter for a better class of people has always been a bit silly. Eternal September is an inevitable result of having new people join a community. Every new person, every new comment, increases entropy. It doesn't matter how technically proficient they are, they will change the culture simply by virtue of their existence and participation at scale.
> Did we (the humans) somehow managed to pollute the internet
Corporations did that, not humans.
"few people recognize that we already share our world with artificial creatures that participate as intelligent agents in our society: corporations"
- https://arxiv.org/abs/1204.4116
The public Internet has been relentlessly strip-mined for profit by ever since Canter & Siegel posted their immigration services ad to every single Usenet newsgroup.
I don’t know how, but you manage to consistently flood hit this site with garbage content. And while there is a lot of it, your content is poor enough that I remember you.
It could be used to differentiate LLM text from pre-LLM human text maybe. The thing, our AIs may not be very good at learning but our brains are. The more we use AI, the more we integrate LLMs and other tools into our life, the more their output will influence us. I believe there was a study (or a few anecdotes) where college papers checked for AI material were marked AI written even though they were written by humans because the students used AI during their studying and learned from it.
> ... compare the frequency of words to those used in human natural writings and you spot the computer from the human.
But that's a losing endeavor: if you can do that, you can immediately ask your LLM to fix its output so that it passes that test (and many others). It can introduce typos, make small errors on purpose, and anything you can think of to make it look human.
it may work for a short time, but after a while natural language will evolve due to natural exposure of those new words or word patterns and even human will write in ways that, while being different from the LLMs, will also be different from the snapshot captured by this snapshot. It's already the case that we used to write differently 20 years ago from 50 years ago and even more so 100 years ago, etc
Hardly. You are talking about a statistical test, which will have rather large errors (since it is based on word frequencies). Not to mention word frequencies will vary depending on the type of text (essay, description, advertisement, etc).
Two of the languages we support, Serbian and Chinese, are written in multiple scripts. To avoid spurious differences in word frequencies, we automatically transliterate the characters in these languages when looking up their words.
Serbian text written in Cyrillic letters is automatically converted to Latin letters, using standard Serbian transliteration, when the requested language is sr or sh."
I'd support keeping both scripts (српска ћирилица and latin script) , similarly to hiragana (ひらがな) and katakana (カタカナ) in Japanese.
I don't care about the political part, but twitter used to have nice, high-enough quality expert discussion around topics, and now its just a shithole with very stupid takes polluting the few spots free in replies between engagement farmers and llm slop-spewers
The problems are well-known and highly-documented. You should leave the determination of (b) up to those who know and understand (a), which includes the author.
It does poison the article, when someone is talking about subject X, and then they feel the need to insert their political opinion about person Y and that dont' like that Z is left-wing/right-wing. It makes them seem not at all objective, and calls into doubt what else they are not being objective about in their article.
As far as I noticed, the “right” effectively gets the boot in most online communities which abide to a Code of Conduct, leaving mostly the “left” (the most recent example I have in mind of such moderation efforts is the save-nix-together.org open letter). It’s interesting that you don’t notice this happening in the communities you seem to frequent.
No, I see a lot of the "right" discourse. Many are openly supporting Putin now. I follow many conservative (US) pundits and journalists and they have either taken a hard right turn or are raising the alarm and supporting Harris. I see similar trends here in Canada.
Yes, I see that the left has become more authoritarian, but it pales to the hard shift I see on the right.
Perhaps they only find the increase shocking on the left.
There's a bunch of ways to measure political opinions. The Authoritarian-liberal one being one of many. The economic-left and the economic-right are becoming more separated from the social-left and the social-right.
Tribalism also causes people to take on the positions of their 'tribe' which may be distinct to what their own personalities might normally gravitate to.
In the past, it has been the economic-left and social-right that were more prone to authoritarianism with their proponents believing that their ideals should be enforced.
The economic-right and social-left was more of a logic vs empathy tension ('this works' vs 'this is right') and a lot of people seemed to reconcile the two for one flavour of centrism.
To me it is a little shocking how authoritarian elements of the social-left have become, an ideology that has long been characterized by empathy and supporting others seems to have become blended with opinions which are exclusionary or dogmatic, which seem counter to their own principles.
In some respects maybe this is just the march of time making the progressive opinions of one generation the orthodoxy of the next and these people are just finding a new conservatism rooted in a new orthodoxy.
Isn't that just what happens when you keep pushing the Overton Window to the right? What would have been 'centrists' have to become more authoritarian to stand ground or else they let their position get absorbed by the stronger leaning side. When one side refuses to compromise even slightly, you have two options: give in or dig in.
If the left-most edge of the Overton Window had been pushed right of the centrist position of, say, the 90s that would be correct. But it's obvious that isn't at all true -- many moderate left positions today would have been considered outlandishly radical in the 90s.
It's rather more likely that the increase of left-authoritarianism is due to increasing resistance to moving the Overton Window, both the left-most edge and the right-most edge, leftwards. As resistance increases more forceful techniques are necessary.
Gay marriage, for instance. In the 90s (and even the 2000s) it was pretty common even for people on the left to come down on the side of "I don't think we should allow gay marriage". Whereas now, it's so firmly within the Overton window that even most of the right thinks gay marriage is fine.
The fact is that society changes over time and what was unacceptable during one time becomes acceptable in another. This does not indicate a shift to the left, it indicates humanity becoming less intolerant. Are you going to argue that allow black people and white people to marry and repealing miscegenation laws mean that the country shifted to the left?
Are you saying that the Overton window can never move right as long as we maintain more human rights and tolerance than we had decades before whatever the current time period is?
No. I'm saying once you smooth over five or ten year periods, that it is empirically false that the Overton window in North America is moving rightwards.
It may very well feel that way to people who live in strong bubbles, but it just isn't true across the general population -- which of course is how the Overton Window is defined.
How does 'moderate positions now would be extreme in the 90s' act to bolster that contention, without the unsaid requirement that 'any included tolerance or added human rights that didn't exist more than a decade ago means de-facto left-wing Overton movement', which precludes any democratic society from having the Overton move right?
> It's interesting that you are only seeing it on the "left".
One explanation is that now things have switched round, and people with left-wings beliefs, sometimes extremely life-wing beliefs, control a lot of institutions and structures. People who are my age (too old) grew up with the right being in that position, but I don't think that's a contemporary instinct to possess.
Even if this is true it being an emotional decision, so much of twitter/X itself is now AI slop anyway, so it'd be worth it to just not even include it whether it was right wing or not.
Regardless, the owner is well within his right to make an emotional decision based on his beliefs to stop anyway.
Yeah, I'm no fan of Musk or Trump, but I think Twitter always was a spam-infested, hateful cesspool where people with online-addiction yelled at each other. There was nothing for Musk to ruin, because the whole concept was rotten from the start. Allowing only short messages doesn't promote intelligent discussion, it does the opposite.
Somehow related, paper books from before 2020 could be a valuable commodity in a in a decade or two, when the Internet will be full of slop and even contemporary paper books will be treated with suspicion. And there will be human talking heads posing as the authors of books written by very smart AIs. God, why are we doing this????
"I don't think anyone has reliable information about post-2021 language usage by humans."
We've been past the tipping point when it comes to text for some time, but for video I feel we are living through the watershed moment right now.
Especially smaller children don't have a good intuition on what is real and what is not. When I get asked if the person in a video is real, I still feel pretty confident to answer but I get less and less confident every day.
The technology is certainly there, but the majority of video content is still not affected by it. I expect this to change very soon.
I find issue with this statement as content was never a clean representation of human actions or even thought. It was always driven by editorials, SEO, bot remixing and whatnot that heavily influences how we produce content. One might even argue that heightened content distrust is _good_ for our society.
Did you think the same thing when photoshop came out?
It's relatively trivial to photoshop misinformation in a really powerful and undetectable way- but I don't see (legitimate) instances of groundbreaking news over a fake photo of the president or a CEO etc doing something nefarious. Why is AI different just because it's audio/video?
And it's not the grounbreaking the problem, it's the little constant lies.
Last week a photoshopped Musk tweet was going around, people getting all up in arms against it despite the fact it was very easy to spot as a fabricated one.
People didn't care, they hate the guy, they just wanted to fuel their hate more.
The whole planet run on fake content, magazin covers, food packaging, instagram pics of places that never looks that way...
And now, with AI, you can automate it and scale it up.
People are not ready. And in fact, they don't want to be.
This video's worth a watch if you want to get a sense of the current state of things. Despite the (deliberately) clickbait title, the video itself is pretty even-handed.
It's by Language Jones, a YouTube linguist. Title: "The AI Apocalypse is Here"
It's even worse than that. Most people have no idea how far CGI has come, and how easily it is wielded even by a couple of dedicated teens on their home computer, let alone people with a vested interest in faking something for some financial reason. People think they know what a "special effect" looks like, and for the most part, people are wrong. They know what CGI being used to create something obviously impossible, like a dinosaur stomping through a city, looks like. They have no idea how easy a lot of stuff is to fake already. AI just adds to what is already there. Heck, to some extent it has caused scammers to overreach, with things like obviously fake Elon Musk videos on YouTube generated from (pure) AI and text-to-speech... when with just a little bit more learning, practice, and amounts of equipment completely reasonable for one person to obtain, they could have done a much better fake of Elon Musk using special effects techniques rather than shoveling text into an AI. The fact that "shoveling text into an AI" may in another few years itself generate immaculate videos is more a bonus than a fundamental change of capability.
Even what's free & open source in the special effects community is astonishing lately.
This is a common survivorship bias fallacy since you only notice the bad CGI.
I'm certain you'd be shocked to see the amount of CG that's in some of your favorite movies made in the last ~10-20 years that you didn't notice because it's undetectable
I won’t be, I’m aware that lots of movies are mostly CGI.
But, yeah, I do think it is some kind of bias. Maybe not survivorship, though… maybe it is a generalized sort of Malmquist bias? Like the measurement is not skewed by the tendency of movies with good CGI to go away. It is skewed by the fact that bad CGI sticks out.
Actually wait I take it back, I mean, I was aware that lots of Digital Touch-up happens in movie sets, more than lots of people might expect, and more often that one might expect even in mundane movies, but even still, this comment’s video was pretty shocking anyway.
Luckily, for those of us who prefer when film photography meant at least mostly actually filming things, there’s plenty of very good film and TV (and even more of lesser quality) to keep a person occupied for a couple lifetimes.
And you see things like the The Lion King remake or its upcoming prequel being called "live action" because it doesn't look like a cartoon like the original. But they didn't film actual lions running around -- it's all CGI.
I mean, it's already apparent to me that a lot of people don't have a basic process in place to detect fact from fiction. And it's definitely not always easy, but when I hear some of the dumbest conspiracy theories known to man actually get traction in our media, political figures, and society at large, I just have to shake my head and laugh to keep from crying. I'm constantly reminded of my favorite saying, "people who believe in conspiracy theories have never been a project manager."
Oh they definitely are. A lot of people are now calling out real photos as fake. I frequently get into stupid Instagram political arguments and a lot of times they come back with "yeah nice profile with all your AI art haha". It's all real high quality photography. Honestly, I don't think the avg person can tell anymore.
I've reached a point where even if my first reaction to a photo is to be impressed, I then quickly think "oh but what it this is AI?" and then immediately my excitement for the photo is ruined because it may not actually be a photo at all.
These are a little bit unfair, in that we're comparing handpicked examples, but I don't think many experts will pass a test like this. Technology only moves forward (and seemingly, at an accelerating pace).
What's a little shocking to me is the speed of progress. Humanity is almost 3 million years old. Homosapiens are around 300,000 years old. Cities, agriculture, and civilization is around 10,000. Metal is around 4000. Industrial revolution is 500. Democracy? 200. Computation? 50-100.
The revolutions shorten in time, seemingly exponentially.
Comparing the world of today to that of my childhood....
One revolution I'm still coming to grips with is automated manufacturing. Going on aliexpress, so much stuff is basically free. I bought a 5-port 120W (total) charger for less than 2 minutes of my time. It literally took less time to find it than to earn the money to buy it.
+100w chargers are one of the products I prefer to spend a little more on, so I get something from a company that knows it can be sued if they make a product that burns down your house or fries your phone.
Flashlights? Sure, bring on aliexpress. USB cables with pop-off magnetically attached heads, no problem. But power supplies? Welp, to each their own!
But seriously, it's harder to accidentally make a USB cable that fries your equipment. The more common failure mode is it fails to work, or wears out too fast. Chargers on the other hand, handle a lot of voltage, generate a lot of heat, and output to sensitive equipment. More room to mess up, and more room for mistakes to cause damage.
> One revolution I'm still coming to grips with is automated manufacturing. Going on aliexpress, so much stuff is basically free. I bought a 5-port 120W (total) charger for less than 2 minutes of my time. It literally took less time to find it than to earn the money to buy it.
Is there a big recent qualitative change here? Or is this a continuation of manufacturing trends (also shocking, not trying to minimize it all, just curious if there’s some new manufacturing tech I wasn’t aware of).
For some reason, your comment got me thinking of a fully automated system, like: you go to a website, pick and choose charger capabilities (ports, does it have a battery, that sort of stuff). Then an automated factor makes you a bespoke device (software picks an appropriate shell, regulators, etc). I bet we’ll see it in our lifetimes at least.
$2.48 Eastern and Southern Africa (PIP)
$2.78 Sub-Saharan Africa (PIP)
$3.22 Western and Central Africa (PIP)
$3.72 India (rural)
$4.22 South Asia (PIP)
$4.60 India (urban)
$5.40 Indonesia (rural)
$6.54 Indonesia (urban)
$7.50 Middle East and North Africa (PIP)
$8.05 China (rural)
$10.00 East Asia and Pacific (PIP)
$11.60 Latin America and the Caribbean (PIP)
$12.52 China (urban)
And more generally:
$7.75 World
I looked around on Ali, and the cheapest charger that doesn't look too dangerous costs around five bucks. So it's roughly equal to one day's income of at least half the population of our planet.
Okay. You're right about what I wrote. Let me rephrase what I meant. I was missing the words "the widespread adoption of"
Athens had a democracy over 2500 years ago. A few Native American tribes had long-lasting democracies. Ukrainian cities were democratically self-governing 500 years ago, and Poland had elected kings.
Those were isolated examples. This was not a revolution. We also haven't regressed; isolated examples continued throughout history. If you point to a year, you can probably find some democracy somewhere. The only major regression I know in history was around 1000BC. Regressions are rare.
What changed was a revolution. From just before 1800 to just a little after 1900, virtually every country had a revolution which led to either being some form of democracy, or pretending to be one. Democracy was no longer isolated. We had the creation of a free world covering much of the world's population, and the creation of what was pretending to be a democracy (today, even the Democratic People's Republic of Korea pretends to be a democracy).
The number of countries which claim to not be a democracy, you can count on your fingers. Iran. Vatican City. Saudi Arabia. UAE. Oman. Eswanti. Did I miss any?
Democracy (and Republics) are thousands of year old. Computation is also quite old though it only sky-rocketed with electricity and semiconductors. This is not the first time the global world created a potential for exponential growth (I'll consider the Pharaohs and Roman empires to be ones).
There is the very real possibility that everything just stalls and plateau where we are at. You know, like our population growth, it should have gone exponentially but it did not. Actually, quite the reverse.
> When I get asked if the person in a video is real, I still feel pretty confident to answer
I don't share your confidence in identifying real people anymore.
I often flag as "false-ish" a lot of things from genuinely real people, but who have adopted the behaviors of the TikTok/Insta/YouTube creator. Hell, my beard is grey and even I poked fun at "YouTube Thumbnail Face" back in 2020 in a video talk I gave. AI twigs into these "semi-human" behavioral patterns super fast and super hard.
There is a video floating around with pairs of young ladies with "This is real"/"This is not real" on signs. They could be completely lying about both, and I really can't tell the difference. All of them have behavioral patterns that seems a little "off" but are consistent with the small number of "influencer" videos I have exposure to.
That is very true, but for now we have a baseline of videos that we either remember or that we remember key details of, like the persons in the video. I'm pretty sure if I watch The Primeagen or Tom Scott today, that they are real. Ask me in year, I might not be so sure anymore.
I hear this complaint often but in reality I have encountered fairly little content in my day to day that has felt fully AI generated? AI assisted sure, but is that a problem if a human is in the mix, curating?
I certainly have not encountered enough straight drivel where I would think it would have a significant effect on overall word statistics.
I suspect there may be some over-identification of AI content happening, a sort of Baader–Meinhof effect cognitive bias. People have their eye out for it and suddenly everything that reads a little weird logically "must be AI generated" and isn't just a bad human writer.
Maybe I am biased, about a decade ago I worked for an SEO company with a team of copywriters who pumped out mountains the most inane keyword packed text designed for literally no one but Google to read. It would rot your brain if you tried, and it was written by hand by a team of humans beings. This existed WELL before generative AI.
> I hear this complaint often but in reality I have encountered fairly little content in my day to day that has felt fully AI generated?
How confident are you in this assessment?
> straight drivel
We're past the point where what AI generates is "straight drivel"; every minute, it's harder to distinguish AI output from actual output unless you're approaching expertise in the subject being written about.
> a team of copywriters who pumped out mountains the most inane keyword packed text designed for literally no one but Google to read.
And now a machine can generate the same amount of output in 30 seconds. Scale matters.
> every minute, it's harder to distinguish AI output from actual output unless you're approaching expertise in the subject being written about.
So, then what really is the problem with just including LLM-generated text in wordfreq?
If quirky word distributions will remain a "problem", then I'd bet that human distributions for those words will follow shortly after (people are very quick to change their speech based on their environment, it's why language can change so quickly).
Why not just own the fact that LLMs are going to be affecting our speech?
> So, then what really is the problem with just including LLM-generated text in wordfreq?
> Why not just own the fact that LLMs are going to be affecting our speech?
The problem is that we cannot tell what's a result of LLMs affecting our speech, and what's just the output of LLMs.
If LLMs result in a 10% increase of the word "gimple" online, which then results in a 1% increase of humans using the word "gimple" online, how do we measure that? Simply continuing to use the web to update wordfreq would show a 10% increase, which is incorrect.
541 comments
[ 3.1 ms ] story [ 309 ms ] threadMight even change the tool name.
LLMs pollute the internet like atomic bombs polluted the environment.
The analogy is that data is now contaminated with AI like steel is now contaminated with nuclear fallout.
https://en.wikipedia.org/wiki/Low-background_steel
>Low-background steel, also known as pre-war steel[1] and pre-atomic steel,[2] is any steel produced prior to the detonation of the first nuclear bombs in the 1940s and 1950s. Typically sourced from ships (either as part of regular scrapping or shipwrecks) and other steel artifacts of this era, it is often used for modern particle detectors because more modern steel is contaminated with traces of nuclear fallout.[3][4]
From the same wiki you linked:
"Since the end of atmospheric nuclear testing, background radiation has decreased to very near natural levels, making special low-background steel no longer necessary for most radiation-sensitive uses, as brand-new steel now has a low enough radioactive signature"
and
"For the most demanding items even low-background steel can be too radioactive and other materials like high-purity copper may be used"
Another solution, and one that, if i weren't such a lazy, is ocean based carbon binding. You can run electricity directly through ocean water and precipitate the carbon out as calcium carbonate, which is both: useful to humans as is and after processing; and useful to the coral reefs and crustaceans/mollusks or whatever in the oceans.
If anyone wants to kick me about a million US dollars, i can make a POC on a used barge with solar panels and as much recycled material as possible, and have that just run off the coast of florida or something. I figure the total cost to get a barge is around a quarter million, all-in[1], the electronics and seawater stuff is about another $150-200 thousand, and the rest is mine for the idea and the lawyers' to get this approved and left alone to do the research.
[0] burning it for heat is fine, as the net CO2 levels will remain constant, but i mean things like houses and boardwalks and boats, furniture, and so on.
[1] could be more, now, the last time i was researching seaworthy barge costs it was between $100,000 and $200,000. I'm hoping there's someone that can donate the barge so i can make the rest more fit for purpose - redundancy, better solar, better mppt, better batteries, better materials for the electrodes (it takes platinum and titanium iirc, i haven't looked at my documents for a long while.)
(Sorry for pessimism and offtopicism)
New things arise from the ashes.
https://en.m.wikipedia.org/wiki/Low-background_steel
For applications that need to avoid the background radiation (like physics research), pre atomic age steel is extracted, like from old shipwrecks.
https://en.m.wikipedia.org/wiki/Low-background_steel
> Low Background Steel (and lead) is a type of metal uncontaminated by radioactive isotopes from nuclear testing. That steel and lead is usually recovered from ships that sunk before the Trinity Test in 1945.
This is a desired quality, increasingly less present in IT work environments. People afraid of being shamed for stating knowledge gaps are not the folks you want to work with.
See a lot of people upset about AI still using AI image generation because it's not in their field so they feel less strongly about it and can't create art themselves anyway, hypocritical either use it or don't but don't fuss over it then use it for something thats convenient for you.
The new stuff generated does (and this is honestly already captured).
This author doesn't generate content. They analyze data from humans. That "from humans" is the part that can't be discerned enough and thus the project can't continue.
Their research and projects are great.
Making resources like wordfreq more visible won't exacerbate any of these concerns.
Another example is how data on humans after 2020 or so can't be separated by sex because gender activists fought to stop recording sex in statistics on crime, medicine, etc.
Edit: just the first one
Everything is “seamless” nowadays. Like I am seamlessly commenting here.
Arguably, the meaning of these words evolve due to misuse too.
I wish there were a way to check.
(I initially wanted to say 'paid for by the government' but that'd be socialising losses and we've had quite enough of that in the past.)
Next token-seeking is a solved problem. Novel thinking can be solved by humans and possibly by AI soon, but adding more garbage to the data won't improve things.
> As one example, Philip Shapira reports that ChatGPT (OpenAI's popular brand of generative language model circa 2024) is obsessed with the word "delve" in a way that people never have been, and caused its overall frequency to increase by an order of magnitude.
This will surely affect how we speak. It's possible that human language evolution could come to a halt, stuck in time as AI datasets stop being updated.
In the worst case, we will see a global "model collapse" with human languages devolving along with AI's, if future AIs are trained on their own outputs...
There are charts / graphs in the link, both since 2021, and since earlier.
The final graph suggests the phenomenon started earlier, possibly correlated in some way to Malaysian / Indian usages of English.
It does seem OpenAI's family of GPTs as implemented in ChatGPT unspool concepts in a blend of India-based-consultancy English with American freshmen essay structure, frosted with superficially approachable or upbeat blogger prose ingratiatingly selling you something.
Anthropic has clearly made efforts to steer this differently, Mistral and Meta as well but to a lesser degree.
I've wondered if this reflects training material (the SEO is ruining the Internet theory), or is more simply explained by selection of pools of Hs hired for RLHF.
It also made the web a less than ideal source for training. And yet LLMs were still fed articles written for Googlebot, not humans. ML/LLM is the second iteration of writing pollution. The first was humans writing for corporate bots, not other humans.
It seems to me that the fact it's so cheap and relatively easy for people with dreams of becoming wealthy influencers to put stuff out there has more to do with the flood of often mediocre content than AI does.
Of course the vast majority don't have much real success and get on with life and the crank turns and a new generation perpetuates the cycle.
LLMs etc. may make things marginally easier but there's no shortage of twenty somethings with lots of time imagining riches while making pennies.
Increasingly I find that for in-depth explanations or tutorials Youtube is the only place to go, but even there the search results can lead to loads of videos which just seem… off. But at least those are still made by humans.
Is it of your own coinage? When the AI sifts through the digital wreckage of the brief human empire, they may give you the credit.
https://en.wiktionary.org/wiki/nontent
It sucks, because sharing recipes seemed like one of those things the internet could be really good at.
The only remaining reliable source - now that many newspapers are axing the remaining staff in favour of LLMs - is pre-2020 print cookbooks. Anything online or printed later must be assumed to be tainted, full of untested sewage and potentially dangerous suggestions.
I joke with folks that my assumption with "one clove of garlic" is that they really mean "one head of garlic" if you want any flavour. (And if the recipe title has "garlic" in it and you are using one clove, you’re lying.)
I read Spanish and Italian fluently and stumble my way through Japanese (with translation). It's easier to find a good recipe in these languages, provided you can find the ingredients or substitutes.
That is, when baking, you can usually (again, exceptions for creaming the sugar in butter, etc.) take all of your dry ingredients and mix/sift them together, and then you pour your wet ingredients in a well you’ve made in the dry ingredients (these can also usually be mixed together).
But for shortbread or fork biscuits those three could indeed all go in the bowl in one go (but that one admittedly doesn't really need a bracket because the recipe is "put in bowl, mix with hands, bake").
When I saw them, they blew my mind. Short to store and easy to understand.
At least in Googles case, they're having so much difficulty keeping AI slop out of their search results that I don't have much faith in their ability to give it an appropriately low training weight. They're not even filtering the comically low-hanging fruit like those YouTube channels which post a new "product review" every 10 minutes, with an AI generated thumbnail and AI voice reading an AI script that was never graced by human eyes before being shat out onto the internet, and is of course always a glowing recommendation since the point is to get the viewer to click an affiliate link.
Google has been playing the SEO cat and mouse game forever, so can startups with a fraction of the experience be expected to do any better at filtering the noise out of fresh web scrapes?
Google has been _monetizing_ the SEO game forever. They chose not to act against many notorious actors because the metric they optimize for is ad revenue and and those sites were loaded with ads. As long as advertisers didn’t stop buying, they didn’t feel much pressure to make big changes.
A smaller company without that inherent conflict of interest in its business model can do better because they work on a fundamentally different problem.
A useful Search would ideally send a user to the site with the most signal and the fewest noise. Meanwhile, ads are inherently noise; they're extra pieces of information inserted into a webpage that at best tangentially correlate to the subject of a page.
Up until ~5 years ago, Google was able to strike a balance on keeping these two stable; you'd get results with some Ads but the signal generally outweighed the noise. Unfortunately from what I can tell from anecdotes and courtroom documents, the Ad team at Google has essentially hijacked every other aspect of the company by threatening that yearly bonuses won't be given out if they don't kowtow to the Ad teams wishes to optimize ad revenue somewhere in 2018-2019 and has no sign of stopping since there's no effective competition to Google. (There's like, Bing and Kagi? Nobody uses Bing though and Kagi is only used by tech enthusiasts. The problem with Google is that to copy it, you need a ton of computing resources upfront and are going up against a company with infinitely more money and ability to ensure users don't leave their ecosystem; go ahead and abandon Search, but good luck convincing others to give up say, their Gmail account, which keeps them locked to Google and Search will be there, enticing the average user.)
Google has absolutely zero incentive to filter out generative AI junk from their search results outside the amount of it that's damaging their PR since most of the SEO spam is also running Google Ads (since unless you're hosting adult content, Google's ad network is practically the only option). Their solution therefore isn't to remove the AI junk, but to instead reduce it enough to the degree where a user will not get the same type of AI junk twice.
A search engine isn't a two-sided market in itself but the ad network that supports it is. A better search engine is a technological problem, but a decently paying ad network is a technological problem and a hard marketing problem.
I've noticed that lately. It used to be the top google result was almost always what you needed. Now at the top is an AI summary that is pretty consistently wrong, often in ways that aren't immediately obvious if you aren't familiar with the topic.
The problem is that, of the signals you mention,
• the highly-informative ones (posting a new review every 10 minutes, having affiliate links in the description) are contextual — i.e. they're heuristics that only work on a site-specific basis. If the point is to create a training pipeline that consumes "every video on the Internet" while automatically rejecting the videos that are botspam, then contextual heuristics of this sort won't scale. (And Google "doesn't do things that don't scale.")
• and, conversely, the context-free signals you mention (thumbnail looks AI-generated, voice is synthesized) aren't actually highly correlated with the script being LLM-barf rather than something a human wrote.
Why? One of the primary causes is TikTok (because TikTok content gets cross-posted to YouTube a lot.) TikTok has a built-in voiceover tool; and many people don't like their voice, or don't have a good microphone, or can't speak fluent/unaccented English, or whatever else — so they choose to sit there typing out a script on their phone, and then have the AI read the script, rather than reading the script themselves.
And then, when these videos get cross-posted, usually they're being cross-posted in some kind of compilation, through some tool that picks an AI-generated thumbnail for the compilation.
Yet, all the content in these is real stuff that humans wrote, and so not something Google would want to throw away! (And in fact, such content is frequently a uniquely-good example of the "gen-alpha vernacular writing style", which otherwise doesn't often appear in the corpus due to people of that age not doing much writing in public-web-scrapeable places. So Google really wants to sample it.)
I’m guessing that the association between “pagers” and “Hezbollah” ended up creating the latter two tabs, but who knows. Maybe some AI video out there did a product review of Hezbollah.
The top of search results is consistently crowded by pages that obviously game ranking metrics instead of offering any value to humans.
How do we know what content LLMs were fed? Isn't that a highly guarded secret?
Won't the quality of the content be paramount to the quality of the generated output or does it not work that way?
For a while I thought email as a medium was doomed, but spammers mostly lost that arms race. One interesting difference is that with spam, the large tech companies were basically all fighting against it. But here, many of the large tech companies are either providing tools to spammers (LLMs) or actively encouraging spammy behaviors (by integrating LLMs in ways that encourage people to send out text that they didn't write).
I'm not saying this is impossible but that's going to be an uphill sell for me as a concept. According to some quick stats I checked I'm getting roughly 600 emails per day, about 550 of which go directly to spam filtering, and of the remaining 50, I'd say about 6 are actually emails I want to be receiving. That's an impressive amount overall for whoever built this particular filter, but it's also still a ton of chaff to sort wheat from and as a result I don't use email much for anything apart from when I have to.
Like, I guess that's technically usable, I'm much happier filtering 44 emails than 594 emails? But that's like saying I solved the problem of a flat tire by installing a wooden cart wheel.
It's also worth noting there that if I do have an email thats flagged as spam that shouldn't be, I then have to wade through a much deeper pond of shit to go find it as well. So again, better, but IMO not even remotely solved.
Looking at the last days’ spam¹ I have three 419-style scams (widows wanting to give away their dead husbands’ grand piano or multi-million euro estate) and three phishing attempts. There are duplicate messages in each category.
About fifteen years ago, I did a purge of mailing list subscriptions and there’s very little that comes in that I don’t want, most notably a writer who’s a nice guy, but who interpreted my question about a comment he made on a podcast as an invitation to be added to his manually managed email list and given that it’s only four or five messages a year, I guess I can live with that.
⸻
1. I cleaned out spam yesterday while checking for a confirmation message from a purchase.
However, the frontline of the email war has shifted lately. Now the most important part of the war is being fought over emails that look just like ham, but aren't. Business frauds where someone convinces you that they are the CEO or CFO or some VP and they need you to urgently buy this or that for them right now no time to talk is big business right now, and before you get too high-and-mighty about how immune you are to that, they are now extremely good at looking official. This war has not been won yet, and to a large degree, isn't something you necessarily win by AI either.
I think there's an analogy here to the war on content slop. Since what the content slop wants is just for you to see it so they can serve you ads, it doesn't need anything else that our algorithms could trip on, like links to malware or calls to action to be defrauded, or anything else. It looks just like the real stuff, and telling that it isn't could require a human rather vast amounts of input just to be mostly sure. Except we don't have the ability to authenticate where it came from. (There is no content authentication solution that will work at scale. No matter how you try to get humans to "sign their work" people will always work out how to automate it and then it's done.) So the one good and solid signal that helps in email is gone for general web content.
I don't judge this as a winning scenario for the defenders here. It's not a total victory for the attackers either, but I'd hesitate to even call an advantage for one side or the other. Fighting AI slop is not going to be easy.
Advertising in your mails isn't Google's.
There's probably an analogy to be made about the open decentralised internet in the age of AI here, if it gets to the point that search engines have to assume all sites are spam by default until proven otherwise, much like how an email server is assumed guilty until proven innocent.
I already find myself mentally filtering out audible releases after a certain date unless they're from an author I recognize.
Blog spam was generally written by humans. While it sucked for other reasons, it seemed fine for measuring basic word frequencies in human-written text. The frequencies are probably biased in some ways, but this is true for most text. A textbook on carburetor maintenance is going to have the word "carburetor" at way above the baseline. As long as you have a healthy mix of varied books, news articles, and blogs, you're fine.
In contrast, LLM content is just a serpent eating its own tail - you're trying to build a statistical model of word distribution off the output of a (more sophisticated) model of word distribution.
SEO text carefully tuned to tf-idf metrics and keyword stuffed to them empirically determined threshold Google just allows should have unnatural word frequencies.
LLM content should just enhance and cement the status quo word frequencies.
Outliers like the word "delve" could just be sentinels, carefully placed like trap streets on a map.
TFA mentions this hasn't been the case.
It’s the same origin. On Slashdot (the HN of the early 00’s) people would admonish others to RTFA. Then they started using it as a referent: TFA was the thing you were supposed to have read.
I miss the old Atomic MPC forums in the ~00s.
2. Given how LLMs work, a prompt is a bias — they're one-and-the-same. You can't ask an LLM to write you a mystery novel without it somewhat adopting the writing quirks common to the particular mystery novels it has "read." Even the writing style you use in your prompt influences this bias. (It's common advice among "AI character" chatbot authors, to write the "character card" describing a character, in the style that you want the character speaking in, for exactly this reason.) Whatever prompt the developer uses, is going to bias the bot away from the statistical norm, toward the writing-style elements that exist within whatever hypersphere of association-space contains plausible completions of the prompt.
3. Bot authors do SEO too! They take the tf-idf metrics and keyword stuffing, and turn it into training data to fine-tune models, in effect creating "automated SEO experts" that write in the SEO-compatible style by default. (And in so doing, they introduce unintentional further bias, given that the SEO-optimized training dataset likely is not an otherwise-perfect representative sampling of writing style for the target language.)
But, insofar as your use-case is against the TOUs of the big proprietary inference platforms, this second cost quickly swamps the first cost. They keep banning you, and you keep having to buy new dark-web credentials to come back.
Given this, it’s a lot cheaper and more reliable — you might summarize these as “more predictable costs” — to design a system around a substrate whose “immune system” won’t constantly be trying to kill the system. Which means either your own hardware, or a “being your own model” inference platform like RunPod/Vast/etc.
(Now consider that there are a bunch of fly-by-night BYO-model hosted inference platforms, that are charging unsustainable flat-rate subscription prices for use of their hardware. Why do these exist? Should be obvious now, given the facts already laid out: these are people doing TOU-violating things who decided to build their own cluster for doing them… and then realized that they had spare capacity on that cluster that they could sell.)
So it's classic positive feedback. LLM uses delve more, delve appears in training data more, LLM uses delve more...
Who knows what other semantic quirks are being amplified like this. It could be something much more subtle, like cadence or sentence structure. I already notice that GPT has a "tone" and Claude has a "tone" and they're all sort of "GPT-like." I've read comments online that stop and make me question whether they're coming from a bot, just because their word choice and structure echoes GPT. It will sink into human writing too, since everyone is learning in high school and college that the way you write is by asking GPT for a first draft and then tweaking it (or not).
Unfortunately, I think human and machine generated text are entirely miscible. There is no "baseline" outside the machines, other than from pre-2022 text. Like pre-atomic steel.
Some day we may view this as the beginnings of machine culture.
Have you ever seen someone use their smartphone? They're not "here," they are "there." Forming themselves in cyberspace -- or being formed, by the machine.
I don't think we'll have true machine culture until we have fully autonomous agents in the wild that are interacting with the world independently on its own terms. Right now the substrate is text which comes from a human mind -- it does not arise naturally from nothing. So the machine is a symbiote for now until we solve some difficult robotics problems.
I don't think you need to be independent or autonomous to develop a culture. And a lot of human culture was passed down over generations without understanding why it worked. We just imitate the behaviour and rituals from our most successful ancestors or role models.
If new LLMs can access the past generation's knowledge of how to please human evaluators, they will use it. It's not a deliberate decision by an "agent", it's just the best text source to copy from. This is a new feedback loop between generations of assistants, and it bypasses whatever the human designer had in mind. Phrases like "it is always best to ask an expert" will pop up just because you tuned the LLM to sound like a helpful assistant, and that's what helpful assistants sound like in the training data. You'd have to actively steer the new generation away from using their ancestral knowledge.
I guess it comes down to what your definition of "culture" is. There is no targeted teaching of the next generation, for example - but is this a requirement? I agree that talking about "machine culture" right now sounds like a stretch, but now I wonder what pieces are actually missing.
Far from saying the pool of language is now polluted, I think we now have a great data set to begin to discern authentic from inauthentic human language. Although sure, people on the fringes could get caught in a false positive for being bots, like you or I.
The biggest LLM of them all is the daily driver of all new linguistic innovation: Human society, in all its daily interactions. The quintillions of daily phrases exchanged and forever mutating around the globe - each mutation of phrase interacting with its interlocutor, and each drawing from not the last 500,000 tokens but the entire multi-modal, if you will, experience of each human to date in their entire lives - vastly eclipses anything any hardware could ever emulate given the current energy constraints. Software LLMs are just a state machine stuck in a moment in time. At best they will always lag, the way Stalinist language lagged years behind the patois of average Russians, who invented daily linguistic dodges to subvert and mock the regime. The same process takes place anywhere there is a dominant official or uncool accent or phrasing. The ghetto invents new words, new rhythm, and then it becomes cool in the middle class. The authorities never catch up, precisely because the use of subversive language is humanity's immune system against authority.
If there is one distinctly human trait, it's sniffing out anyone who sounds suspiciously inauthentic. (Sadly, it's also the trait that leads to every kind of conspiracy theorizing imaginable; but this too probably confers in some cases an evolutionary advantage). Sniffing out the sound of a few LLMs is already happening, and will accelerate geometrically, much faster than new models can be trained.
I'm a little more cautious though. I think GPT will be way more integrated, simply because it's useful. Stalinist language was artificial, in the sense that it was basically imposed on you from outside for no good reason. When you wanted to get real stuff done (either talking to close friends, being productive with colleagues, etc) you wouldn't use socialist newspeak because it got in the way. GPT will be imposed by the outside world, but it's actually a useful thing to be able to converse with a language model; you'll do it every day at work, when buying things, when using your phone/PC.
And also, unlike in USSR times, so much of our communication is online and visible. It would not surprise me if we develop a model that can train continuously on the firehose. Text is small. Data rate of every person on earth speaking simultaneously:
- 150 words per minute spoken
- 150 words × (5 characters/word + 1 space) = 150 × 6 = 900 characters per minute
- 1 byte per char = 900 bytes/min = 15 bytes/sec
- 15 bytes / sec * 8,000,000,000 people speaking continuously = 120 gigabytes/second
That's a lot but it's not even the bandwidth of a single consumer GPU.
GOGI.
I don't see how Google's SEO rules being written or unwritten has any bearing. Spammers will always find a way.
Based on the process above, naturally, the third iteration then is LLMs writing for corporate bots, neither for humans nor for other LLMs.
>Reddit also stopped providing public data archives, and now they sell their archives at a price that only OpenAI will pay.
>And given what's happening to the field, I don't blame them."
What beautiful doublethink.
Given just how many AI bots scrape up everything they can, oftentimes ignoring robots.txt or any rate limits (there have been a few complaint threads on HN about that), I can hardly blame the operators of large online services just cutting off data feeds.
Twitter however didn't stop their data feeds due to AI or because they wanted money, they stopped providing them because its new owner does everything he can to hinder researchers specializing in propaganda campaigns or public scrutiny.
In other words, why is it wrong for X but okay for Reddit? If you ignore one individual’s politics, the two services did the same thing.
Furthermore, while X had also only announced this in February, Reddit announced their API shutdown just 2 months later in April.
And, to further add to that, X was pretty upfront that they think they have access to a large and powerful dataset in X and didn't want to give it out for free. Reddit used very similar wording when announcing their changes.
[1] Or maybe Sheckley or Lem, now that I think about it.
...and then I realized you wrote 3 books a day. What the hell.
This is a self-inflicted problem, IMO.
Do you just have shitty friends that share all that crap? Or are you following shitty pages?
I use Facebook a decent amount, and I don't suffer from what you're complaining about. Your feed is made of what you make it. Unfollow the pages that make that crap. If you have friends that share it, consider unfriending or at the very least, unfollowing. Or just block the specific pages they're sharing posts from.
In my opinion the internet can be considered as the equivalent of a natural environment like the earth. it's a space where people share, meet, talk, etc.
I find it astonishing that after polluting our natural environment we know polluted the internet.
If we haven't already, we will be very soon. I'm sure there are people working on this problem, but I think we're starting to hit a very imminent feedback loop moment. Most of human's recorded information is digitized and most of that is generating non-human content at an incredible pace. We've injected a whole lot of noise into our usable data.
I don't know if the answer is more human content (I'm doing my part!) or novel generative content but this interim period is going to cause some medium-term challenges.
I like to think the LLM more-tokens-equals-better era is fading and we're getting into better use of existing data, but there's a very real inflection point we're facing.
They're the ones who post racist diatribes under green alt account and anti-vaxx conspiracy theories and complain about wokeness and talk out of their ass about subjects they have no actual expertise in. The ones who think flagging is just a super downvote.
I mean, the premise that simply having a minimalist layout would filter for a better class of people has always been a bit silly. Eternal September is an inevitable result of having new people join a community. Every new person, every new comment, increases entropy. It doesn't matter how technically proficient they are, they will change the culture simply by virtue of their existence and participation at scale.
Nice try
If it’s not clear, I’m joking.
Corporations did that, not humans.
"few people recognize that we already share our world with artificial creatures that participate as intelligent agents in our society: corporations" - https://arxiv.org/abs/1204.4116
You’re not edgy dude. Give it up.
compare the frequency of words to those used in human natural writings and you spot the computer from the human.
Brains aren't nearly as good at slightly adjusting the statistical properties of a text corpus as computers are.
Hmm I don’t disagree but I think it will be valuable skill going forward to write text that doesn’t read like it was written by an LLM
This is an arms race that I’m not sure we can win though. It’s almost like a GAN.
But that's a losing endeavor: if you can do that, you can immediately ask your LLM to fix its output so that it passes that test (and many others). It can introduce typos, make small errors on purpose, and anything you can think of to make it look human.
Two of the languages we support, Serbian and Chinese, are written in multiple scripts. To avoid spurious differences in word frequencies, we automatically transliterate the characters in these languages when looking up their words.
Serbian text written in Cyrillic letters is automatically converted to Latin letters, using standard Serbian transliteration, when the requested language is sr or sh."
I'd support keeping both scripts (српска ћирилица and latin script) , similarly to hiragana (ひらがな) and katakana (カタカナ) in Japanese.
We are going through a general uptick in authoritarian "discussions" online. It's interesting that you are only seeing it on the "left".
Yes, I see that the left has become more authoritarian, but it pales to the hard shift I see on the right.
There's a bunch of ways to measure political opinions. The Authoritarian-liberal one being one of many. The economic-left and the economic-right are becoming more separated from the social-left and the social-right.
Tribalism also causes people to take on the positions of their 'tribe' which may be distinct to what their own personalities might normally gravitate to.
In the past, it has been the economic-left and social-right that were more prone to authoritarianism with their proponents believing that their ideals should be enforced.
The economic-right and social-left was more of a logic vs empathy tension ('this works' vs 'this is right') and a lot of people seemed to reconcile the two for one flavour of centrism.
To me it is a little shocking how authoritarian elements of the social-left have become, an ideology that has long been characterized by empathy and supporting others seems to have become blended with opinions which are exclusionary or dogmatic, which seem counter to their own principles.
In some respects maybe this is just the march of time making the progressive opinions of one generation the orthodoxy of the next and these people are just finding a new conservatism rooted in a new orthodoxy.
It's rather more likely that the increase of left-authoritarianism is due to increasing resistance to moving the Overton Window, both the left-most edge and the right-most edge, leftwards. As resistance increases more forceful techniques are necessary.
Such as?
More tolerance is pretty much the definition of the social-left.
It may very well feel that way to people who live in strong bubbles, but it just isn't true across the general population -- which of course is how the Overton Window is defined.
I somewhat expect authoritarianism on the right and therefore would hold the left (to which I belong) at a higher standard.
One explanation is that now things have switched round, and people with left-wings beliefs, sometimes extremely life-wing beliefs, control a lot of institutions and structures. People who are my age (too old) grew up with the right being in that position, but I don't think that's a contemporary instinct to possess.
Regardless, the owner is well within his right to make an emotional decision based on his beliefs to stop anyway.
We've been past the tipping point when it comes to text for some time, but for video I feel we are living through the watershed moment right now.
Especially smaller children don't have a good intuition on what is real and what is not. When I get asked if the person in a video is real, I still feel pretty confident to answer but I get less and less confident every day.
The technology is certainly there, but the majority of video content is still not affected by it. I expect this to change very soon.
I see a lot of outrage around fake posts already. People want to believe bad things from the other tribes.
And we are going to feed them with it, endlessly.
It's relatively trivial to photoshop misinformation in a really powerful and undetectable way- but I don't see (legitimate) instances of groundbreaking news over a fake photo of the president or a CEO etc doing something nefarious. Why is AI different just because it's audio/video?
And it's not the grounbreaking the problem, it's the little constant lies.
Last week a photoshopped Musk tweet was going around, people getting all up in arms against it despite the fact it was very easy to spot as a fabricated one.
People didn't care, they hate the guy, they just wanted to fuel their hate more.
The whole planet run on fake content, magazin covers, food packaging, instagram pics of places that never looks that way...
And now, with AI, you can automate it and scale it up.
People are not ready. And in fact, they don't want to be.
It's by Language Jones, a YouTube linguist. Title: "The AI Apocalypse is Here"
https://youtu.be/XeQ-y5QFdB4
Even what's free & open source in the special effects community is astonishing lately.
I'm certain you'd be shocked to see the amount of CG that's in some of your favorite movies made in the last ~10-20 years that you didn't notice because it's undetectable
But, yeah, I do think it is some kind of bias. Maybe not survivorship, though… maybe it is a generalized sort of Malmquist bias? Like the measurement is not skewed by the tendency of movies with good CGI to go away. It is skewed by the fact that bad CGI sticks out.
https://news.ycombinator.com/item?id=41584276
https://www.youtube.com/watch?v=clnozSXyF4k
And it already happened, and no one pushed back while it was happening.
https://www.nytimes.com/interactive/2024/09/09/technology/ai...
https://www.nytimes.com/interactive/2024/01/19/technology/ar...
These are a little bit unfair, in that we're comparing handpicked examples, but I don't think many experts will pass a test like this. Technology only moves forward (and seemingly, at an accelerating pace).
What's a little shocking to me is the speed of progress. Humanity is almost 3 million years old. Homosapiens are around 300,000 years old. Cities, agriculture, and civilization is around 10,000. Metal is around 4000. Industrial revolution is 500. Democracy? 200. Computation? 50-100.
The revolutions shorten in time, seemingly exponentially.
Comparing the world of today to that of my childhood....
One revolution I'm still coming to grips with is automated manufacturing. Going on aliexpress, so much stuff is basically free. I bought a 5-port 120W (total) charger for less than 2 minutes of my time. It literally took less time to find it than to earn the money to buy it.
I'm not quite sure where this is all headed.
Flashlights? Sure, bring on aliexpress. USB cables with pop-off magnetically attached heads, no problem. But power supplies? Welp, to each their own!
But seriously, it's harder to accidentally make a USB cable that fries your equipment. The more common failure mode is it fails to work, or wears out too fast. Chargers on the other hand, handle a lot of voltage, generate a lot of heat, and output to sensitive equipment. More room to mess up, and more room for mistakes to cause damage.
Is there a big recent qualitative change here? Or is this a continuation of manufacturing trends (also shocking, not trying to minimize it all, just curious if there’s some new manufacturing tech I wasn’t aware of).
For some reason, your comment got me thinking of a fully automated system, like: you go to a website, pick and choose charger capabilities (ports, does it have a battery, that sort of stuff). Then an automated factor makes you a bespoke device (software picks an appropriate shell, regulators, etc). I bet we’ll see it in our lifetimes at least.
It really isn't. Have a look at daily median income statistics for the rest of the planet:
https://ourworldindata.org/grapher/daily-median-income?tab=t...
And more generally: I looked around on Ali, and the cheapest charger that doesn't look too dangerous costs around five bucks. So it's roughly equal to one day's income of at least half the population of our planet.The Technological Singularity - https://en.wikipedia.org/wiki/Technological_singularity
Progress isn't inevitable. It's possible for knowledge to be lost and for civilization to regress.
Athens had a democracy over 2500 years ago. A few Native American tribes had long-lasting democracies. Ukrainian cities were democratically self-governing 500 years ago, and Poland had elected kings.
Those were isolated examples. This was not a revolution. We also haven't regressed; isolated examples continued throughout history. If you point to a year, you can probably find some democracy somewhere. The only major regression I know in history was around 1000BC. Regressions are rare.
What changed was a revolution. From just before 1800 to just a little after 1900, virtually every country had a revolution which led to either being some form of democracy, or pretending to be one. Democracy was no longer isolated. We had the creation of a free world covering much of the world's population, and the creation of what was pretending to be a democracy (today, even the Democratic People's Republic of Korea pretends to be a democracy).
The number of countries which claim to not be a democracy, you can count on your fingers. Iran. Vatican City. Saudi Arabia. UAE. Oman. Eswanti. Did I miss any?
https://en.wikipedia.org/wiki/List_of_countries_by_system_of...
There is the very real possibility that everything just stalls and plateau where we are at. You know, like our population growth, it should have gone exponentially but it did not. Actually, quite the reverse.
I don't share your confidence in identifying real people anymore.
I often flag as "false-ish" a lot of things from genuinely real people, but who have adopted the behaviors of the TikTok/Insta/YouTube creator. Hell, my beard is grey and even I poked fun at "YouTube Thumbnail Face" back in 2020 in a video talk I gave. AI twigs into these "semi-human" behavioral patterns super fast and super hard.
There is a video floating around with pairs of young ladies with "This is real"/"This is not real" on signs. They could be completely lying about both, and I really can't tell the difference. All of them have behavioral patterns that seems a little "off" but are consistent with the small number of "influencer" videos I have exposure to.
I don't. I mean, I can identify the bad ones, sure, but how do I know I'm not getting fooled by the good ones?
I certainly have not encountered enough straight drivel where I would think it would have a significant effect on overall word statistics.
I suspect there may be some over-identification of AI content happening, a sort of Baader–Meinhof effect cognitive bias. People have their eye out for it and suddenly everything that reads a little weird logically "must be AI generated" and isn't just a bad human writer.
Maybe I am biased, about a decade ago I worked for an SEO company with a team of copywriters who pumped out mountains the most inane keyword packed text designed for literally no one but Google to read. It would rot your brain if you tried, and it was written by hand by a team of humans beings. This existed WELL before generative AI.
How confident are you in this assessment?
> straight drivel
We're past the point where what AI generates is "straight drivel"; every minute, it's harder to distinguish AI output from actual output unless you're approaching expertise in the subject being written about.
> a team of copywriters who pumped out mountains the most inane keyword packed text designed for literally no one but Google to read.
And now a machine can generate the same amount of output in 30 seconds. Scale matters.
So, then what really is the problem with just including LLM-generated text in wordfreq?
If quirky word distributions will remain a "problem", then I'd bet that human distributions for those words will follow shortly after (people are very quick to change their speech based on their environment, it's why language can change so quickly).
Why not just own the fact that LLMs are going to be affecting our speech?
> Why not just own the fact that LLMs are going to be affecting our speech?
The problem is that we cannot tell what's a result of LLMs affecting our speech, and what's just the output of LLMs.
If LLMs result in a 10% increase of the word "gimple" online, which then results in a 1% increase of humans using the word "gimple" online, how do we measure that? Simply continuing to use the web to update wordfreq would show a 10% increase, which is incorrect.