This is neat! I was fairly surprised when a few that were instantly in my mind were actually correct - Back to the Future, and Wizard of Oz, Ghost Busters, Big Lebowski, to name a few.
And I don't think its simply because of the likelihood of guessing a popular movie, as many of the titles I've never heard of. Although some are a giveaway by their content and focusing on an obvious subject - Ghost Busters has clear ghosts; Hurt Locker has a bunch of bomb-disposal guys. Others like Edward Sissorhands, have no giveaways but just embody the style; I'd love to see the descriptions provided to the AI
> I'd love to see the descriptions provided to the AI
Unless the descriptions were particularly detailed, I would expect that a lot of this comes from the training data and the descriptions are just prompts for the model to recall which film it is.
For instance, Willy Wonka & the Chocolate Factory (1971) is clearly based on this real poster:
And of course, the most-common VQGAN was trained on ImageNet, which likely doesn't have every movie poster as training data. (it could be in CLIP though)
Would be interesting to see how well the newer CLIP guided diffusion model works. This is a collection of what it generates with the prompt 'mad max alien spacecraft landed in the desert'
What do you suppose the mechanism is for the Charlie and the Chocolate Factory image having a golden ticket held aloft by somebody’s right hand, with a person in a purple outfit and top hat? The page says:
> a brief text description of a movie
However apart from the existence of a golden ticket, I wouldn’t expect those details to make it into a brief description of the film. And yet there’s an original poster matching those details that the VQGAN + CLIP generated image seems to draw from.
Even more convincing to me is the face of John Malkovich being on the poster of Being John Malkovich. Unless the description includes a pretty accurate description of his face (hairstyle, gender, age, facial hair, skin color), the model must have encountered his appearance in its training set.
That's not enough for reconstructing the face of John Malkovich from text, you need minute facial feature parameters (eye shape, nose shape, eye-nose distances etc etc)
Because he is famous on the Internet, CLIP “knows” what John Malkovich looks like. Or, more accurately: what an image people would label “John Malkovich” feels like.
Wouldn't the most obvious explanation be a description which mentions Willy Wonka's chocolate factory, which doesn't really turn up anywhere in the training data except the original film media?
Star Wars is an interesting example because it appears to include elements lofted directly from the film (bits of stormtroopers body) alongside a princess who definitely isn't Leia. The algorithm might be creating things from scratch at a high level, but the constituent elements are pretty clearly close reproductions of parts of the source material
The same for me - I got 6 correct with some thought, but three of them were instant ideas (less than a second), and those ones were always right: Cast Away, Space Jam, and Monty Python and the Holy Grail.
For me those are not objects upon individual examination except the house - the witch is barely identifiable as a witch; its generally a triangle body with a hat not at all shaped like a witch's typical pointy one. There are cloud forms, but I'd be hard pressed to call them a tornado. Only when I take in the whole image in-general do I see them, but to focus on specific areas, definitely not.
The way people's brains are analyzing and picking apart these images differently is fascinating to me
These are very cool. I didn't get any of them, but paired with the titles, many could be art. I'd like to understand what the training set was, and what prompts were given. Some, like space jam and willy Wonka, seem like they must be accessing images from those movies that were in the training set.
It's very fascinating to watch and almost empathize with the computer as to how it joined these concepts together (i.e. "bullets fly" ends up like a kind of bullet-hummingbird type creature)
I have no idea how, but I actually guessed The Prestige... for some reason I just thought it reminded me of a Tesla Coil. Field of Dreams was a guess just based on it looking like a baseball field.
I'm pretty impressed by ones like Ferris Bueller's Day Off having what I'm pretty sure is the Cubs logo and Wrigley Field, plus Bill and Ted's Excellent Adventure having multiple phonebooths.
To my eye it looks like the Bill and Ted one is trying to include multiple blue police boxes. I'm assuming the movie description included "time travelling phone box" or something like that and the AI generated a bunch of Tardises.
I'm not sure why I got the ones I did, but I got Cast Away, Space Jam, Willy Wonka, National Treasure, Being John Malkovich, and Star Wars. Not sure if they got easier toward the bottom or if I started catching on after viewing the answers as I went down the page.
These images make me wonder what the "brief text description" is for each.
> I typed in movie titles and below is the art I got.
It's clearly impossible to end up with a white haired man and a Delorian from the words "Back to the Future", so it likely has way, way more information about the movie than the title. Possibly even movie posters. I think it's a bit misleading.
In the poster for Bill & Ted's Excellent Adventure, it looks like the AI caught "time travel" and "phone booth" and tried to draw a whole bunch of TARDISes.
Back to the Future has what looks like a TARDIS as well. Although… so does The Matrix, for some reason. Phone booths did play a minor role in that movie, but I kind of doubt that’s the reason. It would be interesting to see the exact text given as the prompt for each movie.
It was less fun for me because I haven't seen many of those movies. If it were 100% classics it would be more fun IMO. Not classic as in old, just most popular. On the other hand who gives a shit what I think. This person probably just generated the images for their favorite movies.
For the ones I did know, I almost always could see a similarity to the "feel" of the movie. But the Rear Window one?? What happened there? Didn't capture the "feel" whatsoever IMO. Of course there are windows but it looks more like a backalley in Japan. The color palette is way off.
Tangentially related, this AI image generation account on Twitter never ceases to fascinate me. Many of the photos are seeded with an initial image however, which reduces the magic/specialness for me a bit: https://twitter.com/images_ai
The color palette is definitely off, but it does capture the sense of looking out a window into other windows. Given that it was primed with a description, it's not surprising that it couldn't get the right colors.
The ambiguity between the bag of pills and typewriter against a backdrop of the strip is pretty uncanny. I would like to use this tool and models for blog post hero image art.
Pretty cool but no dice for me. First I was intrigued but then they all looked the same. I saw a lot of these movies and tried to relate to the poster, but I could not. Then I thought , would I watch the movie if this were the poster on Netflix or Prime or at a theater and I knew I would not. It feels very much like abstract art, which I don't particularly enjoy.
I also missed the personal connection and the prompt I get from seeing human faces, actors I like and enjoy.
I’ve seen most of the movies and could only guess a few, usually by identifying some specific image in the poster and then deducing the movie, like the ghosts and library for Ghostbusters. Some did a better job of getting the look and feel down, for example I said Forest Gump for Field of Dreams, which are pretty similar.
This was really cool. I was able to guess I think seven of them correctly. Some I didn’t even know why I thought of the movie, I just had the impression and it turned out to be right.
I got 5 right. Harold and Kumar, Space Jam, Cast Away, Willy Wonka, and Fear and Loathing. I feel like the prompts were generated off of movie frames? Space Jam for example was very obviously containing cartoons wearing Space Jam jerseys
An image summary of a movie maybe, not a movie poster though. Almost all posters feature the faces of the characters very prominently (presumably in order of screen time) that seems to have been skipped completely
Some of these are just bizarre in how effective they are.
I have never picked up the book or watched the movie Fear and Loathing in Las Vegas. I probably couldn't have told you anything about them beyond what was included in the text description the AI got. Yet I instantly knew that was the movie.
I could even envision a few of these as art work for a special edition blu-ray or something.
That was fun! I got 16 right, if I’m allowed to count the wrong Willy Wonka and Mad Max. Many of the movies I didn’t get I’ve not seen and only barely heard of, so I certainly relied on guessing on famous movies where the themes in the picture are prominent.
My best (frankly uneducated) guess would be that they trained the model with famous movie posters as the objective, and still frames of the movie as the training data. Then they gave frames of these movies to the ai to get a poster out.
I don't think they look as if movie posters were the training objective. No texts, no large faces of leading actors, wrong aspect ratio, unusual colour palette.
120 comments
[ 3.1 ms ] story [ 135 ms ] threadAnd I don't think its simply because of the likelihood of guessing a popular movie, as many of the titles I've never heard of. Although some are a giveaway by their content and focusing on an obvious subject - Ghost Busters has clear ghosts; Hurt Locker has a bunch of bomb-disposal guys. Others like Edward Sissorhands, have no giveaways but just embody the style; I'd love to see the descriptions provided to the AI
Unless the descriptions were particularly detailed, I would expect that a lot of this comes from the training data and the descriptions are just prompts for the model to recall which film it is.
For instance, Willy Wonka & the Chocolate Factory (1971) is clearly based on this real poster:
https://www.amazon.com/-/es/Póster-Online-Willy-Chocolate-Fa...
This seems less “generate a poster from a description” and more “recall half-memorised posters“.
The interesting thing to do, I think, would be to have data set of general images, and use a movie description to pull from those images.
VQGAN does the generation work, CLIP just says if it's good not, improve the latents, repeat. Here's a good technical writeup: https://ljvmiranda921.github.io/notebook/2021/08/08/clip-vqg...
And of course, the most-common VQGAN was trained on ImageNet, which likely doesn't have every movie poster as training data. (it could be in CLIP though)
https://i.imgur.com/A1sAaev.jpg
> a brief text description of a movie
However apart from the existence of a golden ticket, I wouldn’t expect those details to make it into a brief description of the film. And yet there’s an original poster matching those details that the VQGAN + CLIP generated image seems to draw from.
That's not enough for reconstructing the face of John Malkovich from text, you need minute facial feature parameters (eye shape, nose shape, eye-nose distances etc etc)
Star Wars is an interesting example because it appears to include elements lofted directly from the film (bits of stormtroopers body) alongside a princess who definitely isn't Leia. The algorithm might be creating things from scratch at a high level, but the constituent elements are pretty clearly close reproductions of parts of the source material
Thought process: "Making out on a beach? Oh that looks like an Italian flag!"
The matrix I answered as John Wick, which is pretty interesting.
Fear and Loathing was the other I got but I only tried 10.
The way people's brains are analyzing and picking apart these images differently is fascinating to me
I'm pretty impressed and amused by this.
When I looked at that one, I actually "saw" a figure holding distorted scissors imagery.
Trippy stuff!
It's very fascinating to watch and almost empathize with the computer as to how it joined these concepts together (i.e. "bullets fly" ends up like a kind of bullet-hummingbird type creature)
I'm pretty impressed by ones like Ferris Bueller's Day Off having what I'm pretty sure is the Cubs logo and Wrigley Field, plus Bill and Ted's Excellent Adventure having multiple phonebooths.
I would have appreciated some more recent ones though. I don’t quite remember all movies from the 80s.
These images make me wonder what the "brief text description" is for each.
https://thorsbyprojects.tumblr.com/post/660029925870518272/a...
It's clearly impossible to end up with a white haired man and a Delorian from the words "Back to the Future", so it likely has way, way more information about the movie than the title. Possibly even movie posters. I think it's a bit misleading.
For the ones I did know, I almost always could see a similarity to the "feel" of the movie. But the Rear Window one?? What happened there? Didn't capture the "feel" whatsoever IMO. Of course there are windows but it looks more like a backalley in Japan. The color palette is way off.
Tangentially related, this AI image generation account on Twitter never ceases to fascinate me. Many of the photos are seeded with an initial image however, which reduces the magic/specialness for me a bit: https://twitter.com/images_ai
Nonetheless, this was super fun.
I also missed the personal connection and the prompt I get from seeing human faces, actors I like and enjoy.
Yes, I think I detect a definate bias towards a Picasso/cartoonesque style :-)
Even Dr Stangelove is in colour.
I think our jobs are safe for the forseeable future.
I have never picked up the book or watched the movie Fear and Loathing in Las Vegas. I probably couldn't have told you anything about them beyond what was included in the text description the AI got. Yet I instantly knew that was the movie.
I could even envision a few of these as art work for a special edition blu-ray or something.