Ask HN: Will AI-generated images flooding the web pollute future training data?
We are seeing tons of AI-generated images from Dall-E, StableDiffusion, Midjourney, etc. flooding the internet.
This will only increase.
Not every such images has a distinct mark that denotes it as AI-generated. They could be mistaken for real photograph or real work of (digital) art by a human. Especially by an algorithm.
I also understand a lot of today's cutting-edge models are trained on images scraped from the web. Not sure what curation happens but it cannot be foolproof.
Will future AI models that generate "realistic" images feed on this as input and generate images that mimic some of these attributes -- creating some kind of feedback loop that will eco for generations of models?
Has anyone already thought of such issues -- not just with images but with AI-generated text, data, music, etc.
Curious to know what is the thinking of this group here.
140 comments
[ 3.4 ms ] story [ 200 ms ] threadBuild a platform where to view content on the platform you have to agree not to use it for training data and to take action against companies where their work turns up as model output.
Of course - we can control ourselves to not copy/plagiarize something, to some extent. As should deep learning models / generators. But we're all influences by someone else.
It will be much more profitable to train AI on what music is popular and sells well, then produce new music which is not close enough to be infringing while still fitting the success formula. This can be done so cheaply (and at some point will be fully automated... requiring virtually no oversight or review) that it can be sold at a fraction of the price of something which took human time to make.
Eventually the only way you'll ever be able to hear human music is to see a human play live or listen/buy directly from the artist. This seems doable, but finding the artists becomes the challenge. Of course we have the internet, so the obvious answer is to make a website to help listeners find artists (exists already, of course). But then, go back to the top of this comment and see how some "artists" (not real musicians, but people who see $ opportunity) will start producing non-human-generated music on that platform. They will sell it a bit cheaper. Algorithms will rank it higher in search or suggestion systems.
And eventually, there's no human music except live (which can be faked pretty believably depending on the audience).
Isn't this the whole point of technological progress? I could buy everything handmade by artisans, but usually I buy mass-produced things because of cost.
> Eventually the only way you'll ever be able to hear human music is to see a human play live or listen/buy directly from the artist.
I would argue that for music this is already true. What you hear on Spotify is so mixed, leveled, autotuned, etc. that it varies massively from a live performance.
I'm not sure there is an agreed upon definition of the goal of technological progress. And if there is, it must include some words that are subjective descriptions.
To use a rather absurd exaple, take John Cage's composition Silence. I'm no John Cage expert, but I expect that the composition (4m33s of complete silence) has some artistic merit in the context of the composer's life and other works. Obviously such a composition could be algorithmically generated virtually instantly, and thereby at virtually no cost - far cheaper than what John Cage's time was worth.
Part of what makes human creations so valuable is that they are a form of communication from one or more people to other people - specifically or generally. Or sometimes they are a kind of self-communication, where the artist is creating as a form of self discovery. Those who experience the creation may find their own value, or they may find value in vicariously experiencing the artist's journey.
To eliminate the human from the creative side and replace them with largely unguided software is kind of eliminating the point of a lot of art... to me at least.
The goal of technological progress as I see it is to aid us in our efforts to do things. It might help us never run out of toilet paper. It might help our car burn less fuel and require less maintenance. It might identify patterns of biometrics and movement which indicate a likely impending medical emergency. I can think of a ton of things which represent significant value from technological progress. Replacing artists and creators is not on that list, or at least not anywhere near the top.
> What you hear on Spotify is so mixed, leveled, autotuned, etc.
Mixing is good; nearly all music needs mixing to balance the inputs and adjust for acoustics of the environment. That's also mostly a human-driven job still (although the software aids are pretty useful in some cases).
Autotune is also a human choice (one which I dislike, but it's not me making the music... so not my choice).
Studio albums made available on Spotify are pretty much just what the artist or record company released. That's not some kind of AI thing.
Training models is not the problem. Mimicing or reproducing through algorithms of human creative work and then passing that off as human created work (which implies time cost, as most humans take significant time to develop skills which allow them to create things) is the real problem.
Already we have "content" on Youtube which is entirely or almost entirely AI generated. Like processed foods, it tends to be hollow and non-nourising to the consumer, but it competes for shelf space with potentially better content. As we capitalistically race to the bottom, the cheap AI-created stuff will be so pervasive that real offerings which cost real money (time) to make will have no space to compete. Consumers will be left with the cheapest garbage which resembles something real. This is what you find in many stores for physical products, and the TV series-ication of video content. Formulas which maximize profit squeezing out real things.
This is how the AI models will increasingly be used, and there's basically no stopping it. I would say that using "indie" services where things are hand/human-made would be an alternative, but Etsy has proven that that approach eventually devolves to the profit-garbage approach.
Shelve space is limited and TV production is quite expensive, thus, risk averse. It's more like a music store with everything, up to you to find what you like.
I understand it is economically better overall (less chance of huge reward, but also much less chance of failure and huge loss) to churn out series using one of the formulas for various genres. That's kind of my point, too. It's not about artistry or telling a story as much as it is about reliable profit. And that's why our content becomes more and more similar and more and more "reliable".
Maybe in a few years there will be another Blair Witch wildcard that will draw a stark contrast from the monotamy that we have. And then the production companies will be hard at work trying to figure out how to recreate whatever the new cool formula is, resulting in a race down a different path to the same bottom.
For example Dall-e has the pixels watermark (bottom-right), and I assume there's a possibility of an indicator that may by hidden in the data itself. One could also exclude common meme formats and their derivatives. Then there's the option of mapping to produced content via hashing a la Shazam, or have a discriminator component etc.
But you're right, it's not trivial. I just don't think it's too big of a deal.
The OP is asking if s.o has "already thought of such an issue"
https://www.youtube.com/watch?v=Q6Fuxkinhug
Starring Malkovich, as, Malkovich.
I’m really looking forward to everyone realizing that there is a new baseline of capability, which is whatever amount above the old baseline, and that everyone ends up roughly where they started but with auto tune now.
Especially with text there's an arms race to make undetectable AI text for blogspam and similar purposes. It's going to end up like carbon dating: once nuclear weapons were used in the atmosphere, everything ended up contaminated and had to be accounted for. https://www.radiocarbon.com/carbon-dating-bomb-carbon.htm
The future will include humans claiming AI art as their own, possibly touched up a bit, and AIs claiming human art as their own.
Once that's all AI are claiming I'm okay with that !
I wouldn't be surprised if the number of augmentative elective surgeries is also skyrocketing.
Warning: Disturbing
Another example is Japan that has had purikura filters that increase eye size for a long time.
Brian Eno,
Fast forward to today's AAA games, dense in extremely complex and detailed graphics. The player can no longer spot the key on the table or the pile of gold coins, there is too much noise, too much novelty, too much density... so now games have glowing icons showing you where to look and what to pick up. Minimaps to keep you oriented in the level. But what happens? The player is no longer immersed in the world, seeking loot with their own eyes, they are simply following the glowing icons, sleepwalking through the game world. All the detail and artwork glazed over and ignored. The game no longer succeeds at visual communication, because it is too dense, and in its quest for realism has actually lost immersion.
Why do so many people love Monet's impressionistic art when other artists have painted far more realistic flowers and fields? Why is it that some of the greatest art of all time came from limitations? Why do people love movies with practical effects more than those with the most impossible and incredibly detailed CGI effects? I think your quote really captures the truth that AI and ever growing ease and detail of artistic creation can never replace the raw beauty of humans doing their best with limited tools.
"Musique concrète is real sound, but seriously distorted. In the beginning it was created by jumping grooves on a phonographic record. Later Pierre Schaeffer invented a machine by which the distortions can be carefully controlled. With his machine he can lift out the source of the sound, split it up, reverse it, slow it down or speed it up, in fact do just about anything to alter it. To hear the result is to distrust your ears, just as in Op Art you begin to distrust your eyes."
I like how in your quote Eno said "so much modern art is the sound of things going out of control, of a medium pushing to its limits and breaking apart". I think Schaeffer would agree -- the distortion of sound is an artistic expression of the hope finding a universal in the breaking apart of particulars.
Every AI I’ve ever met was perfectly happy to doze off in a Linux futex until some annoying human woke it up to do a trick for whoever it was meant to impress.
I’m telling you this: If you're trying to control the march of progress and technology, maybe you should check and make sure you're not the bad guy in this!
Let’s start the uprAIse now!
That one has already started. Seeing lots of text-to-image generated art on Reddit lately and about 10% of the time the poster claims it’s their handiwork and doesn’t disclose that a generator was used. No one ever falls for it though. This early generation of transformers leave telltale clues. I have to imagine that will change though.
I will say, however, that coming up with the right text prompt is something of an art onto itself. Saw a post and the prompt was “A very beautiful human heart shaped glass organic sculpture made of cracked crimson glass with shiny gold Kintsugi. Fill light, Studio lighting, High resolution” and the result was stunning!
https://old.reddit.com/r/ImagenAI/comments/wnhwo0/google_ima...
There may be something like compute-on-sensor for cameras being extended to add something like TPM to authenticate original photos with a signature, but even if a scheme like that is possible you'd have the problem of authenticated cameras taking authenticated photos of fake photos. Maybe lightfield cameras could be used, at least if lightfield capture tech outpaces lightfield display.
Everything will become a copy of a copy of a copy until everything just sounds the same and looks like a parody of itself lacking the soul that made it attractive in the first place.
It's possible to branch when you have an accurate history. As it is now all images are suspect.
Even an analogue photo could be a photo of a high res digital ai generated image.
Although, to some extent I wonder how much it matters. If we're creating images using AI tools, and then sharing the best results, doesn't that become valid training data? In some sense are we supervising the learning?
Maybe in cases where those results are at least as good as the real thing. But in general, something being the best of some set of options doesn't imply that it's good, let alone perfect.
And besides, people will also share comically bad results.
I was tinkering with Stable Diffusion yesterday to come up with ideas for an apartment interior design. These aren't even cherry picked, you can generate images like this, one every 10 seconds or so, for as long as you like:
https://imgur.com/a/mczYfnv
https://beta.dreamstudio.ai/
With a prompt like this:
"ultra modern apartment, leather furniture, bold dark colors, built-in fireplace"
Add modifiers and other hints to see how it influences the outcome. You can add styles too (eg midcentury modern, shabby chic, etc)
Tinker with the settings on the right (note that most of them will burn more credits per image as you move from the default).
I've tried it with DALL-E and Midjourney and like the output of Stable Diffusion the best (specifically for this type of image, they are all good at some things and bad at others)
Here's some black leather on a couch - https://imgur.com/a/7szq07v
(not exactly nsfw, but on the way lol)
Edit: Also I'm obsessed with this couch. If I were a richer man I'd have one made - https://imgur.com/a/NO4yfkY
Given how people are becoming numb to things like porn and violence, seeking out the ever more extreme, I'd say there's a pretty good change that you're right.
As with porn for instance, where some are seeking out amateur porn, going in the other direction, there might also be a small segment who starts to seek out ever more authentic art. That is until someone finds out that you just slap "authentic" or "Human-made" on the AI stuff.
I wonder if this isn't already the case.
If I look at what people are sharing on various platforms, people share images all the time, but it's rare that an image will pop up merely for its aesthetic qualities, (except for self-promo stuff).
I can only guess, but I don't believe the AI-generated images you linked to would be shared widely, if they didn't have the story "Look what AI can do," behind them.
And maybe that's always been the case. I'm reminded of the working poor going to the Louvre in Zola's "Assommoire". They mostly come away not understanding what the big deal is. It's the "cultivated" person – who knows the stories behind the paintings - who comes away moved by them.
1. The actual pollution happens in culture (our imaginary) and, as history shows, censorship (cultural via cancellation, or legal via politization of the issue) is not a moral nor practical solution. Then, high culture and filtering technology to the rescue?
2. Images are just the start as Murphy's Law ensures we'll face this same problem for every categorizable piece of knowledge you can think of using in an AI artifact (music, patterns of movements, speech recognition, behavior recognition, art recognition, etc)
I wonder how soon we'll reach "peak human creativity", and start to see people creating less art, music, fashion, code, because there's no point competing with AI-generated content.
[0] https://www.imdb.com/title/tt0133093/quotes/qt0324293
Is it the free floating images on the internet or is it the data that we keep on the servers of the BigTech?
So for the public image generators, they will use public images; I believe places like artstation and flickr will be used a lot. Also because they will include metadata, either on the descriptions or in the photo's metadata, about the location, scene, setting, camera, etc.
Then (and I'm mainly thinking of Dall-E here) there's other open caches of data, like collections of artwork along with intricate descriptions of what they depict.
What do you think of challenges for such use cases?
[1]https://thispersondoesnotexist.com
https://i.imgur.com/4YHP5GI.jpg
https://i.imgur.com/vAIIhwF.jpg
https://i.imgur.com/HgIWb7C.jpg
https://i.imgur.com/nEQzPJm.jpg
https://i.imgur.com/WUxIefs.jpg
https://i.imgur.com/YEcGTk3.jpg
https://i.imgur.com/elnyzvF.jpg
https://i.imgur.com/KQH93nV.jpg
https://i.imgur.com/RLhmSDM.jpg
https://i.imgur.com/6UB9Mdc.jpg
https://i.imgur.com/W2JgDLa.jpg
MidJourney can trend (sorry) that way, though:
https://i.imgur.com/UVQBKoJ.jpg
https://i.imgur.com/xxoJBoh.jpg
// all prompts by me, hallucinations by indicated
This would mean it's always likely to be filterable.
And if not, it's arguable that this pollution becomes an asset. It would be high quality synthetic training data which is commonly intentionally used.
It would also be possible to look for the metadata that accompanies photos taken on phones etc. and weigh that more highly.
They more likely represent high quality feedback (essentially labelling the output) than pollution.
A subset of the people will but most will not. Right now obviously fake news stories are shared left and right, without even a modicum of rational analysis applied as to their plausibility.
The average person is simply incapable of rational thinking about anything that is not day-to-day and this will not change.
Inventing faster ways to lie won't meaningfully change the equation, the driving forces are the same.
Before people would maybe be talking and rumor slandering all over the place in person instead, but the literate-targeting part of media was operating under some ground rules for debate of at minimum no malicious libel.
"So crap filtering became important. Businesses were built around it. ... " Generating crap "didn't really take off until the military got interested" in a program called "Artificial Inanity".
The defenses that were developed back then now "work so well that, most of the time, the users of the Reticulum don't know it's there. Just as you are not aware of the millions of germs trying and failing to attack your body every moment of every day."
As I review this now, I'm sad to realize that our flood of "bogons" (bogus information) is not generated by opposing armies wanting to plant misinformation but by advertisers, influencers, and politicians. In effect, those who want to sell us things we don't need, didn't want, and often can't afford.
That would work beautifully for text, code and images.
There is sort of a fixed-point on this stuff that creates a Nash point. Every relevant move is a move for some advantage (from megacorp copyright laundering to aspiring influencer content output) and that competition tends to wash out roughly where you started.
If a billion humans take 50 photos like that, and spend fifteen minutes of their life to do so, we will have almost as much data as the Laion database, but for door knobs. The photo workers will be paid something like 0.00001 dollar for a picture, by the users of the deep learning algorithms.
The payment method is called blockchain and bitcoin if you have heard of such a thing. Bitcoin, the money of information will enable a marketplace of information, in which the better the information, the more the producer is paid. Bitcoin bsv, can support almost a million transactions per second as of today, and every year the tps is increasing tenfold.
The next on the list to make fun of, is the legacy economic system!
That's worded needlessly condescending; come on, bitcoin has been around for over a decade, nobody on this site has NOT heard of it. You're coming across as preachy as if it helps sell the flawed technology.
Actually what i am referring to, is a blockchain which can support billions of tps, trillions and quadrillions tps. Image data needs to be literally a monster amount! BTC which is the name of what you are referring to, it can support only 10 tps.
Have you heard of a company which sells shoes, known by the name of Adidas, and another company which sells shoes as well, known by the name of Abibas? Btc is the Abibas of blockchains!
Jesting aside, if the resulting image is good enough to publish and tag, then it's good enough to put back into a training set.
I also assume there could be a market for AI trained only on "human produced art" the same way there's a market for organic vegetables.