If these sorts of algorithms are ever able to perform animations, etc. then you're onto something here. If holodeck ever becomes a reality, it'll be built on the shoulders of this tech for sure.
Interesting thought! Not sure how to feel about it.
I believe this is how AMD and Nvidia do their "super sampling". The GPU runs the game at a super low resolution, and it gets ai upscaled to 4k. Might not be on individual textures, but I can imagine it being a pipeline in the future, where a game loads with 512px textures that then get upscaled to 1-2k, then the game runs at 720p on the GPU and is upscaled to 4k for the user.
Eh, this might be the least cool use of image-generating AI. Clearly-fantastical landscapes are one thing but the realism destroys it and all these look like shitty DeviantArt photoshops.
When you start seeing them on vinyl-wrapped van conversions next year, you will still say "Yup, still looks like shitty DeviantArt photoshops when writ large."
But it can still eliminate the livelihoods of a great many commercial artists.
Aren't said shitty DA Photoshops actually bread and butter for concept art? I can imagine that if it were able to do style transfer from prompts and a sketch it can be a very useful tool for concept artists
It's interesting how completely this fails at reflections. It keeps forgetting that the reflection of a mountain range can't just be a different mountain range with a somewhat similar shape.
How good is DALL-E at completing a picture of a face when (say) the left half of it has been masked out? Is the resulting face symmetrical? If so, why doesn't the same approach work with reflections?
I think faces are much easier than reflections for a few reasons.
1. faces aren't perfectly symmetric. There is a bunch of leeway on the small scale
2. faces can be oriented to make lack of symmetry harder to see (most photos aren't people perfectly facing the camera)
3. faces can be parametrized with relatively few parameters. You need correct position and sizing for eyes, nose, mouth, eyebrows and ears. If you get those right, everything else is doable based only on local information. For a mountain range reflection, you need to transfer a ton of exact information across the image (and that information can't easily be embedded in 2d. to know the reflection, you have to have a 3d map).
Eyeballs appearing to point in different directions and really asymmetric ears are among the common 'tells' of AI generation in a lot of models too.
I think the biggest challenge is that unlike facial symmetry, reflected mountains in training sets are often significantly perturbed by ripples and objects in the water. So tones in the water broadly matching the tones of the mountains above but with different texture is "good enough" for the model, and it doesn't catch finer detail mismatches in shapes of edges in the same way it catches finer detail mismatches in the shapes of eyes, which almost invariably have the same texture and similar lighting to their compatriot in images.
Faces aren’t perfectly symmetrical though, while mirror reflections are physics. The degree to which the landscape reflection symmetries are off in the article might be acceptable when generating faces.
One possible reason the same approach might not work in both cases could be that DALLE’s training set has more faces than landscapes with still water lakes, it’d have to learn these two symmetry constraints through enough clear examples in both cases, and no counter-examples. I can imagine counter-examples for both faces and landscapes that could potentially inhibit learning symmetry, like faces from an angle or side view, or landscapes with an occluder, but I have no idea how often these occur in DALLE’s training set. Getting this right means that DALLE somehow needs to learn the concept and the symmetry constraint from all the imagery, it has to understand the world a little, not just have pixels to borrow. It’s a hard problem, and at some level surprising how well it’s doing with the landscapes.
It's actually really interesting that the mirror image is that close though. The AI could have seen the water and just saw that there is mountainous shapes, but the AI at least noticed that it's similar to the above one - which tells you how deep these nets go. Perhaps for it to know that it's a nearly perfect mirror image it will need more depth.
Pretty much all the photos of water reflecting mountains are obviously fake because the reflections are wrong. Seeing the wrong mountains & clouds in the reflection kinda triggers me a little bit, I wouldn’t want to have prints of these.
Are there any other major tells that this is AI, especially for the photos that don’t show reflections?
I think the AI tech is really cool, and I’m sure they’ll figure out how to train accurate reflections if prints of AI landscapes start making money, but to me in this specific case, there’s something a little depressing about an auto-generated picture of a place you can’t go. The reason that good landscapes are wonderful is because they were captured, and it’s an aspirational image of somewhere you can visit. The story of the photo’s origin really matters in a lot of cases, which is true for art as well as photos, and is the main thing missing from AI imagery.
The one where waves in the sea look like long-exposure photos of a waterfall are also pretty clearly wrong to the observer. It's funny that there is such a fundamental disconnect between the lifetime of the waves and the scene, similarly to the wrong reflections.
Despite having read this comment prior to clicking the article I have to admit that it took a while before I noticed that the ones with reflections were off.
I’m probably extra sensitive to it; I’m both an amateur photographer and do computer graphics professionally (i.e. study the physics of reflections). Physically, reflections will always align vertically given a horizontal mirror surface, so the very first image in the article is a really pretty extreme example of incorrect reflections. It hit me immediately and pretty hard before I even had a chance to scroll and see the rest of the images. The cloud in the reflection is in the wrong place and the reflection has a big underside shadow that doesn’t exist. The mountain peaks don’t match, especially visible on the peak on the left, and the sun-lit unshadowed parts of the mountain don’t match. Once you start to see these things, it becomes second nature to recognize them and hard to unsee them. :P I used to have a kids book with an illustration that had a reflection of the moon clearly in the wrong place, and it drove me nuts.
I was actually surprised at how good the reflections are, given that the model has no "understanding" of what a reflection is or of the underlying three-dimensional space implied in each image. There are peaks that don't quite match up (the first example is probably the worst, with an entire extra peak), but overall, it's remarkably accurate for having no concrete ruleset for the domain it's working in.
FWIW, I totally agree with this. It’s quite good as a quick ‘impression’, and amazing that it knows to generate something similar in the reflection. This is just one of those things that separates fake from real. Even though it’s subjectively pretty good, it’s not good enough imagery from my perspective to use or to call it indistinguishable from a photo. It all depends on what the goal is, but the title of the piece does make the claim ‘these are not photos’, which from my perspective is establishing the goal posts and easy in this case to argue. The reasons that this is surprisingly good is an example of the story behind the image, it’s surprisingly good when you know how the image was made. In that sense I’m a little bit wrong above about the AI imagery lacking a story. ;)
It may have a vague understanding of things that have mirrored symmetry though, as many things do. And so this concept of reflections that sort of have mirrored symmetry is causing it to overassume that it's like other things.
Interesting. It looks basically like what you'd expect if a "reflection" pattern had evolved in nature (e.g. on a butterfly wing). It looks like a reflection, but logically it does not make sense. It just has the right construction of shapes and colors to fool a not-too-discerning observer.
>The story of the photo’s origin really matters in a lot of cases, which is true for art as well as photos, and is the main thing missing from AI imagery.
This is always what's left out of the discussion. When I purchase art, it's always from local artists. I like the art, but I also really like hearing about their process, or their inspiration, or whatever it is. It feels nice to be let into a part of the artists' lives.
I just can't be convinced that AI will ever replace that feeling.
It's not the art photography prints they have to replace. It's the millions of times a day a business reaches for an image to throw a brochure together or create a YouTube thumbnail or uses a matte painting for video compositing, or does a sky replacement. And especially in cases where the businesses want to have sole ownership of the image and not a stock license.
That’s one great reason to want AI generated imagery, and I even suspect will prove to be the primary business model for these kind of pixels, at least initially. But, speaking personally, I really hope our stock photos end up with better reflections than this. And I hope we don’t end up with a bunch of stock photos everywhere that are fake, it can be misleading in so many ways to use fake images, even for cheesy corporate brochures, and it’s disappointing and cheap for companies to want content without paying for it. Ultimately it does cheapen the original authors’ work and undermine the market for creative work if AI can pump out similar stuff that manages to skirt copyright while simultaneously copying.
>Are there any other major tells that this is AI, especially for the photos that don’t show reflections?
The lighting doesn't match on most of these pictures. Some mountains look like hills upscaled to a size of a mountain - erosion doesn't work like that. The long exposure ocean picture looks terrible because waves look static as if they were flowing over some rocks, but there doesn't seem to be any rocks. The waterfall and volcano picture is just terrible in every way, I'd call it a fantasy photobash if I didn't know it's an AI output.
I'm sure there will be models that are much better at imitating landscape photos, but all pictures in the article are pretty obviously generated, probably except for the ice cave which is pretty good I think.
It’s interesting that things like that are important to some and are completely ignored by the others. It reminds the eternal game engine enhancements race where people look for what’s most realistic with every release. Meanwhile, a dos game from the nineties has no realism conceived in it and expects either a pure power of imagination or in most cases just an acceptance of conventions from its player. (Same for eyes, noses and other proportions in anime.)
I see it as a similar phenomenon. Personally I don’t give a damn if something is off or not, clearly noticeable or hidden. Why would I? What’s the point of the art/painting/photo/etc following some arbitrary rules? All these pics would make a fantastic zero-cost background for a novel, a quest or whatever.
What's interesting to me is that the angle that the generated images are taken from are all at a human's head height or lower. That might seem obvious but it hides the fact that the AI can only copy and merge from it's source images. Is it capable of coming up with a new perspective on it's own?
That might be one first step on the road to solving the problem parent is asking about, but in the case of Nerf, the views (both input and output) are being created by people, not the AI. The nerf AI is being used to create a single 3d scene representation that could be rendered from multiple views, while the DALLE AI is trained to generate 2d images by being trained on a huge corpus of images & text; they are two very different setups.
What I desire in a landscape photograph is the belief, no matter how naïve this is, that I could experience that sight in person. A landscape photograph as proof that the world is beautiful. Fantasy pictures don't have this dimension and so quickly become boring. Generated images, even those made of a mush of real pictures, don't have it either. There is no _there_ there.
Here's an idea, free of charge. Someone, make Smart Goggles that take the landscape around you and make the same landscape particularly beautiful, striking, or stylized to the wearer, with customizable options, so you could walk around a national park and have it look like the most beautiful photo ever taken. You'll sell them like hotcakes.
Why would anyone pay money to be in nature and put a fake optimized/idealized version on top of it?
I just can't understand this need to remove reality from reality. But this may be due to the fact that I seem to become a grumpy old cynical white man.
On first thought, I would support a US/EU bill that requires all AI-generated imagery to have the creation date, model used, and some relevant prompt information embedded into it using steganography.
Any service generating imagery must keep an archive of such imagery for X years. FCC (or similar) penalty for publishing any image that was matched to an AI without attribution or with steganographic information missing would rival DCMA costs.
Why? Most users won't care, but news agencies will get caught red handed posting these as real / without thinking about the repercussions. By keeping an archive, a reverse image search would trivially and automatically catch them.
Cool - you’ve just created a black market for an AI that generates imperceptibly good images without such information embedded, and killed open-source AI because open-source AI can always be edited to remove the info.
But seriously - how would your system work with any open-source AI, or leaked proprietary AI? It wouldn't. The best you could do is sue someone and claim they had an unlabelled AI image, and hire an expert to try to make a case to rebut against their expert. It just falls apart and the legal system probably isn't interested in figuring out by minor artifacts whether that is AI or just compression, distortion, a bad photoshop job, a prank, etc. This is especially important as AI gets more imperceptible in the future. If I take an AI generated face, and shrink it by 50%, all artifacts are plausibly compression, not that the AI had bad hair edge generation, so you can't prove I had an AI. Too many loopholes.
A proper response would be, I argue, simply basing the penalties on the harm caused. Doesn't matter whether the image was made by Photoshop, AI, or an experienced photographer with camera tricks, the harm is the cost.
Edit: I keep adding more reasons, but here's another. This proposal would basically ban remixing AI images because embedding it inside another Photoshopped image, putting your face in the AI photo, resizing it, so forth might mess with the watermark. And naturally, any software capable of preserving the watermark could be easily altered to remove it, and many of these watermarks work solely by their secretive design, so good luck seeing the algorithms ever in your open-source package. Practically banning remixing of different types of photos in open-source software is a terrible idea.
These are excellent points. I'm convinced that my first-order gut feeling is off-base and not worth it. Shame they are lost below my now-invisible comment.
For god's sake why? What's the "problem" you'd be trying to solve with that regulatory nightmare of rules you claim to want? If someone renders a completely invented AI image and uses it for whatever they like, or if businesses and media use these creations of their own for all sorts of commercial purposes, who should care and why should punishment be involved?
I wonder how far we are away from AI creating realistic "alien" landscapes - places that don't look like anything on Earth, but look like they could exists somewhere in the universe.
What's fascinating to me is how so many of these images are "too" beautiful -- rather than actual photographs, they're more like overstuffed fantasy landscape paintings that are trying to combine mountains, moss, a volcano, sunset, and mist all in the same image (image #10 if I'm counting correctly).
But then maybe that's just a result a overstuffed prompts being fed in.
It's also very interesting to see both perspective and reflection inconsistencies, but how many of these aren't obvious at first glance -- only after you look for a couple of seconds.
Look up "Thomas Heaton Photography," among others. He is an actual photographer who has shot images very close to his in their dramatic scope. It is a legitimate photographic style.
Edit: You are best off looking this up in Google Images, his website is quite slow.
Would be great to have an AI web site that shows people their town / home landscape with twenty years on into 2.5 degree climate change. Maybe even their own house and garden.
I remember the 1990’s when morphing¹ was all the rage, and everyone was fascinated by the effect. (It was very common among those who could afford it, and could be seen in movies, TV shows, commercials and also, weirdly, on book covers.) Nowadays, the morph effect looks unbelievably crappy, and obviously fake. The same can be said to have happened to all older forms of special effects; we all now cringe at old movies’ stop-motion animation, rubber monster costumes, and plastic gadgets, but at the time, people thought they looked real.
I would bet that the same will happen with AI-generated images; people will learn to tell the signs, and artists will have to learn to avoid these.
I find it poetic that DALL-E also has trouble with symmetry. Anyone who has done much art can tell you about the time they got half of something perfect, and then just couldn’t get the opposing part to be as good.
64 comments
[ 2.6 ms ] story [ 140 ms ] threadInteresting thought! Not sure how to feel about it.
But it can still eliminate the livelihoods of a great many commercial artists.
1. faces aren't perfectly symmetric. There is a bunch of leeway on the small scale
2. faces can be oriented to make lack of symmetry harder to see (most photos aren't people perfectly facing the camera)
3. faces can be parametrized with relatively few parameters. You need correct position and sizing for eyes, nose, mouth, eyebrows and ears. If you get those right, everything else is doable based only on local information. For a mountain range reflection, you need to transfer a ton of exact information across the image (and that information can't easily be embedded in 2d. to know the reflection, you have to have a 3d map).
I think the biggest challenge is that unlike facial symmetry, reflected mountains in training sets are often significantly perturbed by ripples and objects in the water. So tones in the water broadly matching the tones of the mountains above but with different texture is "good enough" for the model, and it doesn't catch finer detail mismatches in shapes of edges in the same way it catches finer detail mismatches in the shapes of eyes, which almost invariably have the same texture and similar lighting to their compatriot in images.
One possible reason the same approach might not work in both cases could be that DALLE’s training set has more faces than landscapes with still water lakes, it’d have to learn these two symmetry constraints through enough clear examples in both cases, and no counter-examples. I can imagine counter-examples for both faces and landscapes that could potentially inhibit learning symmetry, like faces from an angle or side view, or landscapes with an occluder, but I have no idea how often these occur in DALLE’s training set. Getting this right means that DALLE somehow needs to learn the concept and the symmetry constraint from all the imagery, it has to understand the world a little, not just have pixels to borrow. It’s a hard problem, and at some level surprising how well it’s doing with the landscapes.
Are there any other major tells that this is AI, especially for the photos that don’t show reflections?
I think the AI tech is really cool, and I’m sure they’ll figure out how to train accurate reflections if prints of AI landscapes start making money, but to me in this specific case, there’s something a little depressing about an auto-generated picture of a place you can’t go. The reason that good landscapes are wonderful is because they were captured, and it’s an aspirational image of somewhere you can visit. The story of the photo’s origin really matters in a lot of cases, which is true for art as well as photos, and is the main thing missing from AI imagery.
And remember, these are the “best of the best” examples. I’m sure there are many more that have very obvious flaws.
But to include wrong reflections seems like an obvious error that the creators or author should have caught.
This is always what's left out of the discussion. When I purchase art, it's always from local artists. I like the art, but I also really like hearing about their process, or their inspiration, or whatever it is. It feels nice to be let into a part of the artists' lives.
I just can't be convinced that AI will ever replace that feeling.
The lighting doesn't match on most of these pictures. Some mountains look like hills upscaled to a size of a mountain - erosion doesn't work like that. The long exposure ocean picture looks terrible because waves look static as if they were flowing over some rocks, but there doesn't seem to be any rocks. The waterfall and volcano picture is just terrible in every way, I'd call it a fantasy photobash if I didn't know it's an AI output.
I'm sure there will be models that are much better at imitating landscape photos, but all pictures in the article are pretty obviously generated, probably except for the ice cave which is pretty good I think.
I see it as a similar phenomenon. Personally I don’t give a damn if something is off or not, clearly noticeable or hidden. Why would I? What’s the point of the art/painting/photo/etc following some arbitrary rules? All these pics would make a fantastic zero-cost background for a novel, a quest or whatever.
https://captures.lumalabs.ai/unbounded
Edit: For context, the original reply (now deleted) was about how magic mushrooms were easily obtainable.
https://youtu.be/cCeeTfsm8bk
I just can't understand this need to remove reality from reality. But this may be due to the fact that I seem to become a grumpy old cynical white man.
Any service generating imagery must keep an archive of such imagery for X years. FCC (or similar) penalty for publishing any image that was matched to an AI without attribution or with steganographic information missing would rival DCMA costs.
Why? Most users won't care, but news agencies will get caught red handed posting these as real / without thinking about the repercussions. By keeping an archive, a reverse image search would trivially and automatically catch them.
But seriously - how would your system work with any open-source AI, or leaked proprietary AI? It wouldn't. The best you could do is sue someone and claim they had an unlabelled AI image, and hire an expert to try to make a case to rebut against their expert. It just falls apart and the legal system probably isn't interested in figuring out by minor artifacts whether that is AI or just compression, distortion, a bad photoshop job, a prank, etc. This is especially important as AI gets more imperceptible in the future. If I take an AI generated face, and shrink it by 50%, all artifacts are plausibly compression, not that the AI had bad hair edge generation, so you can't prove I had an AI. Too many loopholes.
A proper response would be, I argue, simply basing the penalties on the harm caused. Doesn't matter whether the image was made by Photoshop, AI, or an experienced photographer with camera tricks, the harm is the cost.
Edit: I keep adding more reasons, but here's another. This proposal would basically ban remixing AI images because embedding it inside another Photoshopped image, putting your face in the AI photo, resizing it, so forth might mess with the watermark. And naturally, any software capable of preserving the watermark could be easily altered to remove it, and many of these watermarks work solely by their secretive design, so good luck seeing the algorithms ever in your open-source package. Practically banning remixing of different types of photos in open-source software is a terrible idea.
But then maybe that's just a result a overstuffed prompts being fed in.
It's also very interesting to see both perspective and reflection inconsistencies, but how many of these aren't obvious at first glance -- only after you look for a couple of seconds.
Edit: You are best off looking this up in Google Images, his website is quite slow.
For those looking for a video, see https://youtu.be/jF76LmzY7YY?t=468, which I would not have believed could be natural.
I would bet that the same will happen with AI-generated images; people will learn to tell the signs, and artists will have to learn to avoid these.
1. https://en.wikipedia.org/wiki/Morphing