Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date.
Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike.
Movie Gen models and capabilities
Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt.
Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment.
Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes.
Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video.
We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.
Considering that Facebook/Meta releases blog posts titled "Open Source AI Is the Path Forward" but then refuses to actually release any Open Source AI, I'm guessing the answer is a hard "No".
They might release it under usage restrictions though, like they did with Llama, although probably only the smaller versions, to limit the output quality.
They have released a ton of open source? Llama 3 includes open training code, datasets, and models. Not to mention open-sourcing the foundation of most AI research today, pytorch.
Llama 3 is licensed under "Llama 3 Community License Agreement" which includes restrictions on usage, clearly not "Open Source" as we traditionally know it.
Just because pytorch is Open Source doesn't mean everything Meta AI releases is Open Source, not sure how that would make sense.
Datasets for Llama 3 is "A new mix of publicly available online data.", not exactly open or even very descriptive. That could be anything.
And no, the training code for Llama 3 isn't available, response from a Meta employee was: "However, at the moment-we haven't open sourced the pre-training scripts".
Sure, the Llama 3 Community License agreement isn't one of the standard open licenses and sucks that you can't use it for free if you're an entity the size of Google.
Correct me if I'm wrong, but that's the code for doing inference?
Meta employee told me just the other day: "However, at the moment-we haven't open sourced the pre-training scripts", can't imagine they would be wrong about it?
Personally, "Open" implies I can download them without signing an agreement with LLama, and I can do whatever I want with it. But I understand the community seems to think otherwise, especially considering the messaging Meta has around Llama, and how little the community is pushing back on it.
So Meta doesn't allow downloading the Llama weights without accepting the terms from them, doesn't allow unrestricted usage of those weights, doesn't share the training scripts nor the training data for creating the model.
The only thing that could be considered "open" would be that I can download the weights after signing the terms. Personally I wouldn't make the case that that's "open" as much as "possible to download", but again, I understand others understand it differently.
Doesn't the training script need to have a training loop at least? Loss calculation? A optimizer? The script you linked contains neither, pretty sure that's for inference only
Was my first reaction too when seeing the video at the top. But then after thinking about it, it makes sense as an example, you want to showcase things that aren't real but look realistic. A hippo swimming looks real, but it isn't as they don't swim.
I have watched some films recently, and they are full of weird mistakes. A bunch of balloons can't lift your house into the air. DeLoreans can't actually travel through time. Gamma rays don't give you superhuman strength. A 6502 CPU couldn't power an advanced AI for killer robots from the future. So unrealistic.
Haha, this is the first thing I thought of too. I knew adult hippos walk on the bottom, but from looking at existing videos it looks like small (baby/pygmy) hippos do too, they don't float at the surface like this.
All the vids have that instantly recognizable GenAI "sheen", for the lack of a better word. Also, I think the most obvious giveaway are all the micro-variations that happen along the edges, which give a fuzzy artifact.
…I mean, it was advancing slowly for linguistic tasks until late 2022, that’s fair. That’s why we’re in such a crazy unexpected rollercoaster of an era - we accidentally cracked intuitive computing while trying to build the best text autocomplete.
AI in general is from 1950, or more generally from whenever the abacus was invented. This very website runs on AI, and always has. I would implore us to speak more exactly if we’re criticizing stuff; “LLMs” came around (in force) in 2023, both for coherent language use (ChatGPT 3.5) and image use (DALLE2). The predecessors were an order of magnitude less capable, and going back 5 years puts us back in the era of “chatbots”, aka dumb toys that can barely string together a Reddit comment on /r/subredditsimulator.
AI so far has given us ability to mass produce shit content of no use to anybody and the next iteration of customer support phone menu trees that sound more convincingly yet remain just as useless. That and another round of IP theft and mass surveillance in the name of progress.
This is a consequence of a type of cognitive bias - bad examples of AI are more easily detectable than good examples of AI. Subsequently, when we recall examples of AI content, bad examples are more easily accessible. This leads to the faulty conclusion that.
> AI so far has given us ability to mass produce shit content of no use to anybody
Good AI goes largely undetected, for the simple reason that it closely matches the distribution of non-AI content.
Controversial aside: This is same bias that results in non-passing trans people being representative of the whole. Passing trans folk simply blend in.
You're making an unverifiable claim. How are we supposed to know that the undetected good AI exists at all? Everything I've seen explicitly produced by any of these models is in uncanny valley territory still, even the "good" stuff.
Verificationism[1] is a failed epistemology because it breaks under the Münchhausen trilemma. It's pseudo-scientific like astrology, four humors, and palm reading. Use better epistemologies.
This basic concept can be applied in many places. Do you ever wonder why social movements seem to never work out well and demands are never met? That’s because when they do work out, and demands are met, those people quickly become the “oppressor” or the powerful class from which others are fighting to receive more rights or money.
All criminals seem so incredibly stupid that you can’t understand why anyone would ever try since they all are caught? The smart ones don’t get caught and no one ever hears about them.
5 years ago there were no AI videos. A bit over a year ago the best AI videos were hilarious hallucinations of Will Smith eating spaghetti.
Today we have these realistic videos that are still in the uncanny valley. That's insane progress in the span of a year. Who knows what it will be like in another year.
Disco Diffusion was a (bad) thing in 2021 that lead to the spaghetti video / weird Burger Kind Ads level quality. But it ran on consumer GPUs / in Jupyter notebook.
2 years ago we had decent video generation for clips
With these things, like DALL-E 1 and GPT-3, the original release of the game changer often comes ca. 2 years before people can actually use it. I think that's what we're looking at.
So you're right to be excited, I agree. And I don't know, Meta, like OpenAI, seems to release conditionally, though yes, more. I doubt it before the election.
When the Will Smith one was released, it was kind of a parody though. Tech had already been able to produce that level of "quality" for about 2 years at the time of it's publishing. The Will Smith one is honestly something you could have created with Disco Diffusion in early 2021, I used to do this back then...
The subtle "errors" are all low hanging fruit. It reminds me of going to SIGGRAPH years back and realizing most of the presentations were covering things which were almost imperceptible when looking at the slides in front. The math and the tech was impressively, but qualitatively it might have not even mattered.
The only interesting questions now have nothing to do with capability but with economics and raw resources.
In a few years, or less, clearly we'll be able to take our favorite books and watch unabridged, word-for-word copies. The quality, acting, and cinematography will rival the biggest budget Hollywood films. The "special effects" won't look remotely CG like all of the newest Disney/Marvel movies -- unless you want them to. If publishers put up some sort of legal firewall to prevent it, their authors, characters, and stories will all be forgotten.
And if we can spend $100 of compute and get something I described above, why wouldn't Disney et al throw $500m at something to get even more out of it, and charge everyone $50? Or maybe we'll all just be zoo animals soon (Or the zoo animals will have neuralink implants and human level intelligence, then what?)
> In a few years, or less, clearly we'll be able to take our favorite books and watch unabridged, word-for-word copies. The quality, acting, and cinematography will rival the biggest budget Hollywood films. The "special effects" won't look remotely CG like all of the newest Disney/Marvel movies -- unless you want them to. If publishers put up some sort of legal firewall to prevent it, their authors, characters, and stories will all be forgotten.
I don't think so at all. You're thinking a movie is just the end result that we watch in theaters. Good directing is not a text prompt, good editing is not a text prompt, good acting is not a text prompt. What you'll see in a few years is more ads. Lots of ads. People who make movies aren't salivating at this stuff but advertising agencies are because it's just bullshit content meant to distract and be replaced by more distractions.
But at the same time, while it is indeed true that the end result is far more than simply just making good images, LLMs are weird interns at everything — with all the negative that implies as well as the positive, so they're not likely to produce genuinely award winning content all by themselves even though they can do better by asking them for something "award winning" — so it's certainly conceivable that we'll see AI indeed do all these things competently at some point.
> In a few years, or less, clearly we'll be able to take our favorite books and watch unabridged, word-for-word copies. The quality, acting, and cinematography will rival the biggest budget Hollywood films. The "special effects" won't look remotely CG like all of the newest Disney/Marvel movies -- unless you want them to. If publishers put up some sort of legal firewall to prevent it, their authors, characters, and stories will all be forgotten.
I'm also expecting, before 2030, that video game pipelines will be replaced entirely. No more polygons and textures, not as we understand the concepts now, just directly rendering any style you want, perfectly, on top of whatever the gameplay logic provided.
I might even get that photorealistic re-imagining of Marathon 2 that I've been wanting since 1997 or so.
I think the big blind spot people have with these models is that the release pages only show just the AI output. But anyone competently using these AI tools will be using them in step X of a hundred step creative process. And it's only going to get worse as both the AI tools improve and people find better ways to integrate them into their workflow.
Yeah exactly. Video pipelines that go into productions we only see the end results of have a lot of steps to them beyond just the raw video output/capture. Even Netflix/Hollywood productions without VFX have a lot of retouching and post processing to them.
Not even filters; every text2image model ever created thusfar, can be very easily nudged with a few keywords into generating outputs in a specific visual style (e.g. artwork matching the signature style of any artist it has seen the some works from.)
This isn't an intentional "feature" of these models; rather, it's kind of an inherent part of how such models work — they learn associations between tokens and structural details of images. Artists' names are tokens like any other, and artists' styles are structural details like any other.
So, unless the architecture and training of this model are very unusual, it's gonna at least be able to give you something that looks like e.g. a "pencil illustration."
At least all the humans in these videos seem to have the correct number of fingers, so that's progress. And Moo Deng seems to have a natural sheen for some reason so can't hold that against them. But your point about the edges is still a major issue.
An important contrast is that early VFX offered strong control with weak fidelity, and these prompt-based AI systems offer high fidelity with weak control. Intent matters if you want to make something more than a tech demo or throwaway B-roll and you can't communicate much intent in a 30 word prompt, assuming the model even follows the prompt accurately.
This is such an important problem of the entire genAI idea. It's absurd that people keep focusing on quality instead of talking about it.
But then, a lot of people have financial reasons to ignore the problem. What's too bad, because it's hindering the creation of stuff that are actually useful.
Yeah controlnet-style conditioning doesn't solve for consistent assets, or lighting, framing etc. Maybe its early but it seems hard to get around traditional 3D assets + rendering, at least for serious use-cases.
These models do seem like they could be great photorealism/stylization shaders. And they are also pretty good at stuff like realistic explosions, fluid renders etc. That stuff is really hard with CG.
> AI systems offer high fidelity with weak control
You are spot on. I've been involved in creating technologies used by film and video creators for decades, so I have some understanding of what would be useful to them. The best video AIs I've seen only seem capable of replacing some stock video clip creation because, so far, I haven't seen any ability to maintain robust consistency from shot to shot and scene to scene. There's also no granular control other than the text prompt. At first glance, these demos are very impressive but when you try to map the capability shown to a real production workflow for a movie, TV show or commercial, they're not even close because they aren't even trying to solve the problem.
To be clear, I think it's probably possible to create a video AI that would be truly useful in a real production workflow, it's just that I haven't seen anything working in that direction yet.
> You are spot on. I've been involved in creating technologies used by film and video creators for decades, so I have some understanding of what would be useful to them. The best video AIs I've seen only seem capable of replacing some stock video clip creation because, so far, I haven't seen any ability to maintain robust consistency from shot to shot and scene to scene. There's also no granular control other than the text prompt. At first glance, these demos are very impressive but when you try to map the capability shown to a real production workflow for a movie, TV show or commercial, they're not even close because they aren't even trying to solve the problem.
Yeah it's really hard to get across to a lot of folks that are really amped up about these tools that what they're focused on refining is not getting them any closer to their imagined goal in most professional workflows. This will be great right off the bat for what most developers would need images for-- making a hero image for a blog post, making a blurb of video for a background, or a joke, or making assets for their video game that would never cut it for a non-cheapo commercial project but are better than what they'd have been able to cobble together themselves. But those workflows are fundamentally so different from the very first steps in the process. It's a larger-scale version of trying to explain to no-compromise FOSS zealots 20 years ago that Gimp was nowhere near able to replace Photoshop in a professional toolkit because they're completely disinterested in taking feedback about professional use cases, and that being able to write your own filters in Perl doesn't really help graphic designers-- well 20 years later, the gap is as wide as it ever has been, and there are even more people, almost exclusively FOSS nerds with to professional visual work experience, that insist it's better.
That said, it's nearly as hard to get this across to ADs who are like "what do you mean this shot is going to take you 3 days? I just made these stills which are like 70% there in midjourney in 10 minutes."
> To be clear, I think it's probably possible to create a video AI that would be truly useful in a real production workflow, it's just that I haven't seen anything working in that direction yet.
I think that neural networks, generally, are already fantastically useful in tools like Nuke's Copycat node. Nobody misses masking frame-by-frame if they don't have to do it. But prompt-based tools? Nah. If even 200 words in a prompt was enough information to convey work that needed to be done, why do creative workflows need so many revisions and why are there so many meetings with sketches and mood boards and concept art among career professionals? Text prompts are great for people that are working with a medium they don't really know how to create in because the real artistic decisions are already made by the artists whose artwork was ingested into the models. If you don't understand that level of nuance, you don't see how unbelievably consequential it is to the final product, and not having granular control of it seems nearly inconsequential. Most professionals look at it and see a toy because they know it will never be capable of making what they want it to make.
> neural networks, generally, are already fantastically useful in tools
Yes, I agree. You've highlighted the distinction I should have included of "prompt-based".
There's a vast gulf between these AI-researcher-based concept demos on one side and the NN-based features slowly getting implemented in real production tools. Like you, I've found it challenging to have constructive conversations about AI tooling with anyone not versed in real production workflows. To anyone with real industry experience it's obvious that so far these demos don't represent a threat to real production workflows or the skilled career professionals making a good living. It's not that they're not threatening, they're just threatening to replace a different type of job entirely. If you're one of the poor souls in an off-shore locale doing remote low-end piece-work like manning a stock photo/video clip farm or doing >$100 per piece gigs on Fivver - then, yeah, you should feel "threatened".
A meta-point I try to make in these conversations is that, at least so far, every actual paying creative job I've seen AI threaten are, IMHO, work I wouldn't wish on my worst enemy. These are low-paid entry-level sweatshop gigs and everyone doing them aspires to do something else as soon as they can. The two analogies I use are: 1) How the "threat" of robotics to jobs is actually playing out. So far, in industrial applications robots are replacing Amazon warehouse and manufacturing assembly line workers, literally today's equivalent of 1920s sweatshop work. Much like the heart-wrenching videos of children in Calcutta earning pennies sifting through junk piles for metal scraps, it'll be a better world when robots replace those jobs and humans have jobs designing, installing, programming and servicing the robots. Likewise in consumer robotics applications, so far, the robots in our house only vacuum the floors, change the cat litter box, and wash the dishes/clothes. Growing up my family spent a couple years living in Asia in the 1970s and we actually had a "wash ama" who came twice a week and washed our clothes manually with a washboard and a tub. Sounds quaint but in reality it was grueling labor. She was a lovely lady but I'm glad Maytag replaced that job.
The second analogy I often use is observing that self-driving cars are mainly a threat to Uber and Lyft drivers who often barely earn minimum wage and have no job security to start with. Career professionals actually working in real video and film production workflows feel as "threatened" by prompt-based AIs as Formula 1 drivers feel about self-driving cars. Why does current F1 champion Max Verstappen never get asked how he feels about AI self-driving cars coming for his job? :-) As you observed, anyone who understands the thousands of creative choices which comprise any shot in a quality film doesn't even see these prompt-based AI demos as relevant. Once you've heard a skilled cinematographer, colorist or director of photography spend over an hour deconstructing and debating the creative choices made in single shot or scene from a film, it's hard to even imagine these demos as a threat to that level of creative skill. But being able to crudely copy the traits of a composite of a thousand exemplars of the craft without understanding any of the interactions between those thousands of creative choices does make for impressive demos. Even though the fidelity of the crude copy is amazing, the fact is such shots are a random puree of a thousand different creative choices pulled from a thousand different great shots. That's the root of what unskilled people call the "AI-clip sheen". It won't be easy to eliminate from prompt-based clip generators because the nature of the NN is it doesn't understand the interactions of all those subtle creative choices it's aping. Mashing together one cinematographer's lens choic...
I think the most vulnerable spots in the industry are in concept art and matte painting, though I also think companies are starting to realize it's not all its cracked up to be. A colleague that also contracts for [big famous FX and animation house we all know and love] said they fired their entire concept art department last year and replaced them with prompt jockeys.... for a few weeks. The prompters could bang out a million "great start" rough drafts in an hour, but then when their boss came around and inevitably said "oh, this one is the one to stick with. Just move this to the right and that to the left and make this bigger and that smaller and make this cloth purple" and they were cooked. They didn't even have the comparatively basic photoshop skills to do a hack job there, let alone make changes by hand-- so they'd struggle with control nets and inpainting and more prompts but the whole thing was one gigantic failure and they were begging the centuries of concept art expertise they unceremoniously booted out the door for forgiveness. And those workflows don't require anywhere near the control that, say, compositing does.
My biggest hope for the professional use of these things is in post-render pre-comp polishing for simulations and pyro. They're so good at understanding patterns and having smooth transitions that they can make a nonsense, physically absurd combination of images blend together perfectly... one of my favorites was a background guy's nose in a sepia toned video was neatly melded into a distant oncoming train. I think that could be really great for smoothing out volume textures and things like that. Given, that probably has more to do with my specialty than anything.
My main problem is that I'm just starting out my career in this field after switching from a decade of python dev work, and then doing some visual design before going to art school where I graduated at the top of my program having mostly concentrated in making cool shit at the Houdini/UE confluence. Two years ago everyone was saying "holy crap you've got the golden skillset," and now everyone's like "oof... hang in there... I guess..." Even aside from the strike aftermath, nobody in the market has any idea what to do right now, especially with juniors, let alone a really weird mixture of junior + senior dev that I am with a few contracts under my belt and a ton of really solid coding experience, but nothing really impressive in the industry itself. Who fucking knows. I think a lot of people in charge of hiring are waiting for a moment where it's going to just be sort of obvious what they need to do, and don't want to hire people into FTEs that are going to be eliminated through ai efficiency gains in 6 months. I don't have a lot of insight into the hiring side of the business though.
Wow, your story about the "FX and animation house" is funny, sad and unsurprising - all at the same time. I'm just surprised they didn't actually test the full workflow before leaping. It reminds me of this tale from actual production people working with Sora https://www.fxguide.com/fxfeatured/actually-using-sora/ which I also found completely unsurprising. It still took a team of three experience pros around two weeks to complete a very modest 90 second video and they needed to reduce their expectations to "making something out of the clips the AI gave us" instead of what they actually wanted. And even that reduced goal required using their entire toolbox of traditional VFX tools to modify the clips the AI generated to match each other well enough. Sure, it's early days and Sora is still pre-alpha but, while some of these problems are solvable with fine-tuning, retraining and adding extensive features for more granular control, some other aspects of these workflow gaps are fundamental to the nature of how NNs work. I suspect the bottom line is that solving some key parts of real-world high-end film/video workflows with the current prompt-based NNs is a case of "you can't get there from here."
For sure. Tooling on top of the core model functionality will absolutely increase the utility of the existing prompt-based workflows, too, but my gut says the diminishing returns on model training is going to keep the "good enough" goalposts much much further into the future with video than with text and still images.
I wonder how much RLHF or other human tweaking of the models contributes to this sort of overstauration / excess contrast in the first place. The average consumer seems to prefer such features when comparing images/video, and use it as a heuristic for quality. And there have been some text-to-image comparisons of older gen models to newer gen, purporting that the older, more hands-off models didn't skew towards kitschy and overblown output the way newer ones do.
I assure you that's not enough. These are high quality videos. Once they get uploaded to social media, compression mostly makes imperfections go away. And it's been shown that when people are not expecting AI content, they are much less likely to realize they are looking at AI. I would 100% believe most of these videos were real if caught off guard.
A surprising amount of it is really popular too. I recently figured out that the Movie Recaps channel was all AI when the generated voice slipped and mispronounced a word in a really unnatural way. They post videos almost daily and they get millions of views. Must be making bank.
I was watching UFC recaps on Youtube and the algorithm got me onto AI generated MMA content, I watched for a while before realizing it. They were using old videos which were "enhanced" using AI and had an AI narrator. I only realized it when the fight footage got so old, and the AI had to do so much work to touch it up, that artifacts started appearing in the video. Once I realized it I rewatched the earlier clips in the video and could see the artifacts there too, but not until I was looking for them.
There's already rabbit holes of fake MMA fighting you can fall into online? Even if you're a "fan" and relatively aware of what to look for ... still difficult to spot? Horribly, had the same sensation while watching UFC at a bar. "Haven't I seen this match where they fall on the ground and hug for hours before?" Mostly empty background audience with limited reactions.
Somebody took AI video editing, and in a year or two, we're already at entire MMA rabbit holes of fake videos.
Commenting mostly as a personal evidence reference of how crazy the World Wide Web has gotten from anecdotal sources.
Most probably they employ overseas, underpaid workers with non-standard English accents and so they include text-to-speach in the production process to smoothen the end result.
I won't argue wether text to speech qualifies as an AI but I agree they must be making bank.
> Most probably they employ overseas, underpaid workers with non-standard English accents and so they include text-to-speach in the production process to smoothen the end result.
Might also be an AI voice-changer (i.e. speech2speech) model.
These models are most well-known for being used to create "if [famous singer] performed [famous song not by them]" covers — you sing the song yourself, then run your recording through the model to convert the recording into an equivalent performance in the singer's voice; and then you composite that onto a vocal-less version of the track.
But you can just as well use such a model to have overseas workers read a script, and then convert that recording into an "equivalent performance" in a fluent English speaker's voice.
Such models just slip up when they hit input phonemes they can't quite understand the meaning of.
(If you were setting this up for your own personal use, you could fine-tune the speech2speech model like a translation model, so it understands how your specific accent should map to the target. [I.e., take a bunch of known sample outputs, and create paired inputs by recording your own performances of them.] This wouldn't be tenable for a big low-cost operation, of course, as the recordings would come from temp workers all over the world with high churn.)
Right, content creators are in a race to the bottom.
But the people who position themselves to profit from the energy consumption of the hardware will profit from all of it: the LLMs, the image generators, the video generators, etc. See discussion yesterday: https://news.ycombinator.com/item?id=41733311
Imagine the number of worthless images being generated as people try to find one they like. Slop content creators iterate on a prompt, or maybe create hundreds of video clips hoping to find one that gets views. This is a compute-intensive process that consumes an enormous amount of energy.
The market for chips will fragment, margins will shrink. It's just matrix multiplication and the user interface is PyTorch or similar. Nvidia will keep some of its business, Google's TPUs will capture some, other players like Tenstorrent (https://tenstorrent.com/hardware/grayskull) and Groq and Cerebras will capture some, etc.
But at the root of it all is the electricity demand. That's where the money will be made. Data centers need baseload power, preferably clean baseload power.
Unless hydro is available, the only clean baseload power source is nuclear fission. As we emerge from the Fukushima bear market where many uranium mining companies went out of business, the bottleneck is the fuel: uranium.
You spent a lot of words to conclude that energy is the difference maker between modern western standards of living and whatever else there is and has been.
Continuing that thought, higher uranium prices and real demand will lead to unshuttering and exploiting known and proven deposits that are currently idle and increase exploration activity of known resources to advance their status to measured and modelled for economic feasiblity, along with revisiting radiometric maps to flag raw prospects for basic investigation.
More supply and lower prices will result.
Not unlike the recent few years in (say) lithium, anticipated demand surged exploration and development, actual demand didn't meet anticipated demand and a number of developed economicly feasible resources were shuttered .. still waiting in the wings for a future pickup in demand.
Spend a few months studying existing demand (https://en.wikipedia.org/wiki/List_of_commercial_nuclear_rea...), existing supply (mines in operation, mines in care and maintenance, undeveloped mines), and the time it takes to develop a mine. Once you know the facts we can talk again.
Look at how long NexGen's Rook 1 Arrow is taking to develop (https://s28.q4cdn.com/891672792/files/doc_downloads/2022/03/...). Spend an hour listening to what Cameco said in its most recent conference call. Look at Kazatomprom's persistent inability to deliver the promised pounds of uranium, their sulfuric acid shortages and construction delays.
Uranium mining is slow and difficult. Existing demand and existing supply are fully visible. There's a gap of 20-40 million pounds per year, with nothing to fill the gap. New mines take a decade or more to develop.
Weird .. and to think I spent several million line kms in radiometric surveys, worked multiple uranium mines, made bank on the 2007 price spike and that we published the definite industry uranium resources maps in 2006-2010.
Clearly you're a better expert.
> when someone compares uranium to lithium, I know I'm not talking to a uranium expert.
It's about boom bust and shuttering cycles that apply in all resource exploration and production domains.
Perhaps you're a little too literal for analogies? Maybe I'm thinking in longer time cycles than yourself and don't a few years of lag as anything other than a few years.
Once again, allow me to offer my sincere apologies.
You are well-prepared to familiarize yourself with the current supply/demand situation. It's time to "make bank", just like you did in 2007... only more so. The 2007 spike was during an oversupplied uranium market and mainly driven by financial actors.
I invite you to begin by listening to any recent interview with Mike Alkin.
I think it's unusual to assume they are based in the US and employ/underpay foreigners. A lot of people making the content are just from somewhere else.
But it uses AI only for audio, right? Script for the vid seems to be written by human, given the unusual humor type of this channel. I started watching this channel some time ago.
It's hard to tell whether they use AI for script generation. After having seen enough of those recaps, the humor seems to be rather mechanical and basic humor is relatively easy to get from an LLM if prompted correctly. The video titles also seem as if they were generated.
That said, this channel has been producing videos well before ChatGPT3.5/4 so at the very least they probably started with human written scripts.
I thought it was just text to speech when I happen to saw some of those videos. And it seems to have been consistently similar since before ChatGPT etc. Why do you think titles are AI generated?
I feel like it might actually be quite complex for AI to pull up the perfect clips and edit them with the script, including timing and everything. Maybe it could be made automatic, but nonetheless it would be a complex process and I don't think possible few years ago. I know Gemini and possibly some others can analyze video if fed to them, but I'm still skeptical that this channel in particular would have done it, when they have always had this frequency of uploads and similar tone.
Also I think there's far better TTS now with ElevenLabs and others so it could be made much more human like.
A group I follow about hobby/miniatures (as in wargaming miniatures and dioramas) recently shared an "awesome" image of a diorama from another "hobby" group.
The image had all the telltale signs of being AI generated (too much detail, the lights & shadows were the wrong scale, the focus of the lens was odd for the kind of photo, etc). I checked that other group, and sure enough, they claim to be about sharing "miniature dioramas" but all they share is AI-generated crap.
And in the original group, which I'm a member of and is full of people who actually create dioramas -- let's say they are "subject matter experts" -- nobody suspected anything! To them, who are unfamiliar with AI art, the photo was of a real hand-made diorama.
The only generation that ever knew how to set the clock on a VCR: our parents needed our help; our kids won't have even seen a VCR outside of a museum, much less used one.
Very interesting point. Wonder about the generation before and what skills they had to share with their parents who were most likely traumatised from a world war or two.
I remember setting the vcr clock and tuning the new tv with the remote. I’m sure the adults could of figured it out but they probably got more from seeing their ‘smart’ kids figuring it out in time for the cartoons!
A friend who lives in North Carolina sent me a video of the raging floodwaters in his state- at least that's what the superimposed text claimed it was. When I looked closer, it was clearly an Indian city filled with Indian people and Indian cars. He hadn't noticed anything except the flood water. It reminded me of that famous selective attention test video[1]. I won't ruin it for those who haven't seen it, but it's amazing what details we can miss when we aren't looking for them. I suspect this is made even worse when we're casually viewing videos in a disjointed way as on social media and we're not even giving one part of the video our full attention.
It probably doesn't work if you're primed to look for hidden details. I took the test along with my Psychology 101 class of about 30 people and no one noticed anything amiss.
Once you see it you can indeed not imagine how you couldn't. Some people see it the first time, but it's a small amount of people. This video just demonstrates how humans can only focus at one thing at a time, and when we're multitasking, we're actually doing little parts of different tasks one at a time but very quickly after each other, kind of like a single CPU core. And if we tightly focus our attention to one point, we are not aware of other things that might be relatively close to that point.
That is also how magicians work, drawing your attention to one particular thing, hiding the secret of the trick from you, sometimes even in plain sight, like in the video.
Or pickpockets, who might bump into you and picking your pocket at the same time, where your attention is focussed on the sudden impact, keeping your attention away from your walled being taken.
I do not see how the examples you mentioned are related to the topic? What does selective attention have to do with the video looking AI generated in all the frames?
Their argument is that if someone is affected by confirmation bias, they likely won’t notice these kinds of details.
Essentially, send me a video of something I care about and I will only look for that thing. Most people are not detectives, and even most would-be detectives aren’t yet experts.
For the entire duration of the Russia/Ukraine war "combat footage" that is actually from the video game ARMA 3 has gone viral fairly regularly, and now exactly the same thing is happening with Israel/Iran.
And which YouTube happily promotes straight to the top, of course -- thanks to the efforts of its rocket-science algorithm team. (Not sure whether the ones I've been seeing were generated by that particular platform, but YT does seem to promote obviously fake and deceptively labelled "combat" footage with depressing regularity).
The willingness of people to believe that combatants are wearing cinematic body cams for no tactical reason can only be matched by their willingness to assume people meticulously record every minute of their lives just so they can post a once-in-a-lifetime event on TikTok.
Who even needs AI generated videos when you can just act out absurdity and pretend it's real?
As far as I know, most of the viral stuff has been active air defence CWIS and the like which can be hard to discern.
There's a morbid path from the grainy Iraq war and earlier shaky footage, through IS propaganda which at the time had basically the most intense combat footage ever released to the Ukraine war.
Which took it to the morbid end conclusion of endless drone video deaths and edited clips 30+ mins long with day long engagements and defending.
And yes, to answer your belief that there is none - there is loads of "cinematic body cam footage out there now".
Thousands of combatants are wearing bodycams, and pretty regularly, there are videos released by Russians of a dead Ukrainian's last moments taken from their corpse and the same happens vice versa.
Dude I clicked on some random Youtube accounts that were streaming the world cup live, and it took me a while to realize that they were actually just streaming video games replica of the actual game (at least, I think they were simulating the actual game with a video game, but I'm not sure as I didn't compare closely)
Someone once did the opposite - streamed a real pay-per-view UFC match on Twitch and pretended it was a game he was playing. It actually worked for a while before the Twitch mods realized what was going on.
Indeed watching Reels or Tiktok videos is an exercise in testing your bullshit meter and commenting accordingly to let the uninformed know hey this is most likely fake.
Facebook is mostly this now too. Long comment threads of boomers thanking AI images for their military service or congratulating it on a long marriage.
The way I see it, it won’t take long before human eyes won’t be able to distinguish AI generated content from original.
The only regret I have about that is losing video as a form of evidence. CCTV footage and the like are a valuable tool for solving crimes. That’s going to be out the window soon.
Trust can be preserved by adding PKI at the hardware level. What you said about CCTV is true; once the market realises and demand appears, camera manufacturers will start making camera modules that, e.g., sign each frame with the manufacturer's private key, enabling Joe Public to verify that that frame came from a camera made by that manufacturer. Reputational risk makes the manufacturer store the private key in the device in a secure, tamper-proof way (like TPMs do now), which (mostly) prevents those private keys from leaking.
Does this create difficulties if you want to modify the raw video data in any way? Yes it does, even if you just want to save it in a different lossy compression level or format. But these problems aren't insurmountable. Essentially, provenance info can be added for each modification, signed by the entity that made the change, and the end viewer can then decide if they trust the full certificate chain (just as they do now with HTTPS).
Oh wow, that's a great idea. Isn't this already happening maybe?
Recently someone said here that it's noticable that videos from CCTV cameras are often filmed with a phone or camera on a screen instead of using the original video, and people were speculating that it might be hard or impossible to get access to the original recording because of bureaucracy or something, but that recording a playback on a screen with a phone or camera or something is then often allowed. Maybe they also do this partly so that the original can't be easily messed with by other people.
But yeah if you can verify that a certain video was filmed at a certain time by a certain camera, that is great. Of course the companies providing these cameras should be trustworthy, and that the camera's are actually really sending what they actually record, and that the company itself doesn't mess with the original recordings.
I recall an article posted 1-2 years ago about a camera company (Kodak? Can't remember) which was starting to offer something along these lines.
>the companies providing these cameras should be trustworthy, and that the camera's are actually really sending what they actually record, and that the company itself doesn't mess with the original recordings.
I agree. We can't guarantee any of these things, but on the bright side, the incentives are pointing in the right direction to make self-interested companies choose to behave the right way.
It will complicate things and make the hardware more expensive, so I doubt it will sweep through all consumer camera tech unless the "Is this photo real?" question becomes a crisis. There's also the fact that it would be possible to give individual cameras different private keys, with certificates signed by the manufacturer: This would enable non-repudiation (you would not be able to plausibly deny that you had taken a particular photo/video), which has potentially big upsides but also privacy downsides. I think that could be solved by giving the user the option of signing with their unique camera private key (when the user wants to prove to others that they took the photo themselves) or with the manufacturer's key (when they want to remain anonymous).
It's sad that almost AS SOON as we acquired the ability to record real-life moments (with the promise of being able to share undeniable evidence of events with one another), we also acquired the ability to doctor it, negating that promise.
I'm not sure we should have been trusting images for the previous decades either. Photoshop has been a thing for a long time already. I mean, there's those famous photos that Stalin had people removed from.
Your mention of Stalin is I think stronger as an argument that there’s been a significant change. Those fakes took lots of time by skilled humans and were notoriously obvious - what made them effective was the crushing political power preventing them from receiving critical analysis in public.
Similarly, while Photoshop made it easier it happened at a time where technical advances made the problem harder because everyone’s standards for photos went up dramatically, and so producing a realistic fake was still a slow process for a skilled worker.
Now, it’s increasingly available to everyone and that means that we’re going to see a lot more scams and hoaxes as people without artistic talent or willingness to invest time can make realistic fakes even for minor things. That availability is transformative enough to merit the concern we’ve been seeing here.
The glass half-full in me feels that the advantage to this is that in a few years the average person will know better than to trust anything that could be faked like that, instead of the old situation where someone who was willing to put in that effort could actually trick a lot of people.
I think that’s true, but it’s kind of like the trade offs during the pandemic where we knew it would eventually settle into a stable state but still wanted to reduce the harm getting there. We basically need some large fraction of the global population to level up in media literacy at all once.
I don't think it goes out the window completely. You need just the owner of the CCTV to stand up in court and say "yes this is the CCTV footage I personally copied from storage and I did not manipulate it".
To me they seem off, but off in the same sense real humans in ads always seem off. E.g. the fake smile of the smiling girl. That's what people look like in ads.
It's my understanding that the AI sheen is done on purpose to give people a "tell". It is totally possible right now to at least generate images with no discernible tell.
In the linked webpage, the following videos would be good enough to trick me:
- The monkey in hotspring video, if not for its weird beard...
- The koala video I would have mistaken for hollywood-quality studio CGI (although I would know it's not real because koalas don't surf... do they?)
- The pumpkin video if played at 1/4 resolution and 2x speed
- The dog-at-Versailles video edit
If the videos are that good, I'm sure I already can't distinguish between real photos and the best AI images. For example, ThisPersonDoesNotExist isn't even very recent, but I wouldn't be able to tell whether most of its output is real or not, although it's limited to a certain style of close-up portrait photography.
> limited to a certain style of close-up portrait photography
Not to take away from your point but it's more limited than one might think from this phrase. As an exercise, open that page and scroll so the full image is on your screen, then hover your mouse cursor within the iris of one of the eyes, refresh and scroll again. (Edit: I just noticed there's a delayed refresh button on the page, so one can click that and then move their mouse over the eye to skip a full page refresh.) I've yet to see a case where my mouse cursor is not still in line with the iris of the next not-person.
That sheen looks (to me) like some of the filters that are used by people who copy videos from TV and movie and post them on (for example) facebook reels.
There's an entire pattern of reels that are basically just ripped-off-content with enough added noise to (I presume) avoid content detection filters. Then the comments have links to scam sites (but are labelled as "the IMDB page for this content").
Yes, but thats just a hypothesis, have we seen any evidence that shows the cause of the "AI sheen" is bad training data, or more likly, just a shortcomming of generating realistic photos from text at this early stage.
It is maybe recognizable in most cases, but definitely not instantly nor easily. I could definitely see nobody noticing one of those clips used in an otherwise non-AI video production.
The bubble released into the air is also pretty good until at the end where bubbles appear out of thin air.
But overall the physics are surprisingly good. In the videos from text we a person moving covered in a bedsheet, a mirror doing vaguely mirror-like things, a monkey moving in water and creating plausible waves, shadows moving over a 3d object with the sloth in the pool and plausible fire. Those are all classic topics to tackle in computer-generated graphics, all casually handled by a model that isn't explicitly trained on physical simulation.
In a twist of irony it's the simplest of those (the mirror) that's the most obviously wrong.
Always important to bear in mind that the examples they show are likely the best examples they were able to produce.
Many times over the past few years a new AI release has "wowed" me, but none of them resulted in any sudden overnight changes to the world as we know it.
VFX artists: You can sleep well tonight, just keep an eye on things!
Tbf, the biggest private infrastructure project in the history of humanity is now underway (Microsoft GPU centers), the fastest app to reach #1 on the App Store was released (ChatGPT), and it’s dominating online discourse. Many companies have used LLMs to justify layoffs, and /r/writers and many, many fanart subreddits already talk of significant changes to their niches. All of this was basically at 0 in 2022, and 100 by early 2023. It’s not normal.
Everyone should sleep well tonight, but only because we’ll look out for each other and fight for just distribution of resources, not because the current job market is stable. IMO :)
I am not trying to be negative, however it is the reality that ML/LLM has eliminated entire industries. Medical transcription for example is essentially gone.
I don't see it as that much of a problem. It's like washing machines taking away people's job of washing clothes, what are they gonna do with their time now? Maybe something more productive.
We really have a problem once there are no more jobs left for us humans, and only the people who own capital (stocks, real estate etc) will be able to earn money from dividends.
The amount required to pay rent on their continued survival, which in a capitalist society, and excluding members of the capitalist class, will never be zero.
That thread you linked doesn’t seem to align at all with your claims though? The majority of comments do not make the claim that they’re using any GenAI elements.
As someone who’s worked in the industry previously and am quite involved still, very few studios are using it because of the lack of direction it can take and the copyright quagmire. There are lots of uses of ML in VFX but those aren’t necessarily GenAI.
GenAI hasn’t had an effect on the industry yet. It’s unlikely it will for a while longer. Bad business moves from clients are the bigger drain, including not negotiating with unions and a marked decline in streaming to cover lost profits.
Yes, and like pretty much every AI release I've seen, even these cherry-picked examples mostly do not quite match the given prompt. The outputs are genuinely incredible, but if you imagine actually trying to use this for work, it would be very frustrating. A few examples from this page:
Pumpkin patch - Not sitting on the grass, not wearing a scarf, no rows of pumpkins the way most people would imagine.
Sloth - that's not really a tropical drink, and we can't see enough of the background to call it a "tropical world".
Fire spinner - not wearing a green cloth around his waist
Ghost - Not facing the mirror, obviously not reflected the way the prompter intended. No old beams, no cloth-covered furniture, not what I would call "cool and natural light". This is probably the most impressively realistic-looking example, but it almost certainly doesn't come close to matching what the prompter was imagining.
Monkey - boat doesn't have a rudder, no trees or lush greenery
Science lab - no rainbow wallpaper
This seems like nitpicking, and again I can't underestimate how unbelievable the technology is, but the process of making any kind of video or movie involves translating a very specific vision from your brain to reality. I can't think of many applications where "anything that looks good and vaguely matches the assignment" is the goal. I guess stock footage videographers should be concerned.
This all matches my experience using any kind of AI tool. Once I get past my astonishment at the quality of the results, I find it's almost always impossible to get the output I'm looking for. The details matter, and in most cases they are the only thing that matters.
The one thing that immediately stood out to me in the ghost example was how the face of the ghost had "wobbly geometry" and didn't appear physically coupled to the sheet. This and the way the fruit in the sloth's drink magically rested on top of the drink without being wedged onto the edge of the glass as that would require were actually some of the more immediate "this isn't real" moments for me.
I think those types of visual glitches can probably be fixed with more or better training, and I have no doubt that future versions of this type of system will produce outputs that are indistinguishable from real videos.
But better training can't fix the more general problem that I'm describing. Perfect-looking videos aren't useful if you can't get it to follow your instructions.
The ghost is insanely impressive, it's the example that gave me a "wow" effect. The cloth physic looks stunning, I never thought we would reach such a level of temporal coherence so fast.
On the contrary I dont think theres anything special with their examples. It probably represent well most output. Think of image generation and the insane stuff people can produce with it. There's no "oh yeah this is just cherry picked"
I find the edit video with text the most fascinating aspect. I can see this being used for indie films that doesn’t have a CGI budget. Like the scene with the movie theater, you can film them on lounge chairs first and then edit it to seem like a movie theater.
Why bother? Actors cost money and scheduling is difficult. Do the whole thing in AI - the model will be trained on better actors than your indie cast, anyway.
It will be a loong time before AI can produce lead actors that are believable, act exactly the way you as a director want and tell the story you want to tell, so I think at least for now you'll still need the actors for the lead roles. But I can totally this being used for generating people/stuff in the background of certain shots in a low budget movie.
Isn't that a core problem now. Getting actors to act exactly how you want was never a solved problem.
But this limits promotion where actors do interviews and sell the movie to the public. It also limits an actor doing something crazy that tanks a movie like a tweet.
The answer is that it depends on the director. For David Fincher or the Coen brothers, having this level of exactitude and precision is what their craft is all about.
But for plenty of other masters - think Cassavetes, Mike Leigh, even PTA - the actor's outstanding talent and instincts bring something to the script and vision that is outside of their prescriptive powers. Their focus is essentially setting up a framework for magic to happen inside of.
> Getting actors to act exactly how you want was never a solved problem.
Also, some great lines in movies came from actors ad libs.
I hope there will always be some space for mild hallucination; without improvisation we wouldn't even have jazz.
> Getting actors to act exactly how you want was never a solved problem.
As a choreographer myself, that's not necessarily a problem but rather a feature: it depends on how the director creates. Often you want what's unique to the performer, you don't want them to do something that's exactly like what you envisioned but whatever their interpretation/vision of it is, the "imperfectness" is what makes it interesting and rich.
> It will be a loong time before AI can produce lead actors that are believable.
A "loong time" will be sooner that most of us think.
The way this is done currently is similar to motion capture except that the tools are gradually becoming democratized: A single actor can act all the roles you need (You could even act the scenes and roles yourself). That footage is then fed into a model that generates an actor with the appearance and voice that you desire.
As a random on the internet, my prediction is that within a year, you'll be able to produce lead actors that are believable using movie generation plus smartphone footage of yourself acting the scene.
Initially it will be expensive to make a feature length film. But from 2025 onward, the cost will come down as the tools improve. These will be a different type of movie for sure, but every advancement in film technology has always led to films that seem strange compared to what came before.
You're getting downvoted, but I agree with you except for your timeline. This won't be possible in a year. What's here is a concept demo, but the gulf between "that's neat" and "you can make a decent 10 minute short film" is pretty vast.
> But the gulf between "that's neat" and "you can make a decent 10 minute short film" is pretty vast.
Agreed. I expect the tools I described to be prohibitively expensive for the average person for some time. By the end of 2025, probably only well-funded studios will be able to use such tools and probably not economically.
But I'd be quite surprised if Hollywood studios/publishers aren't using their immense back catalogue to train private models right now. I don't think they'll ever allow royalty-free movie generation tools to be used that were trained on their catalogues. So perhaps there'll be a cottage industry of stock footage by amateur and professional actors for training/augmenting the movie generation models and tools that will be available for general use, royalty free.
Or perhaps these tools will simply emerge as just another TikTok filter and we'll see goofy couples who filmed a skit in their living room presented as gladiators arguing on the surface of Mars while their dog runs back and forth between them unable to choose a favourite.
100% agree, the background replace that puts the guy into a stadium would be fully usable as a cut in a movie/tv show, and the background is believable enough that no one would bat an eye. If you use it properly, I expect a quality uplift on indie films/shorts. Your limit is your creativity
You’re not wrong, but the comment somewhat misses the point. A shot like that would require you to rent a stadium (generally not cheap) along with paying for a believable number of extras. That would put the shot out of budget for most indie filmmakers. Spending $20 on tokens to get “good enough” is totally worth it, and allows you to get shots that were previously out of reach.
In the professional creative tools business, "Now the only limit is your creativity" has been a popular marketing tagline for decades, especially for products based on new enabling technologies. It's common enough that a wry corollary has developed in response, which goes: "Unfortunately, for a lot of people that's a pretty big limit."
I personally expect a decrease in quality. Without limits people tend to get less creative. Sure, there is some balance here in that tools also enable new things to be done which are not possible without tools but working around limits has often inspired some of the most creative works.
And without being forced to interact with other people. The movie made by one creative and 100 automatons does not ai all compare to the one where there are multiple brilliant creatives butting heads and personalities and choosing never to work with each other again but the show must go on.
How many movie lines have been adlibbed but are absolute classics? Sonofabitch, he stole my line!
What a bizarre statement in an age when the phrase "executive meddling" can describe the sameness of so much content output, and most of the greatest flops have a story which goes "yeah there were too many people involved".
Like the second Avengers movie had this problem in spades.
It's not a bizarre statement at all. An executive meddling with something because X or Y element of a piece of art doesn't align with A or B market trend is not the same thing as people working together and sometimes clashing due to creative differences. You'll find that most works that you, or other people, like weren't the result of a sole individual's creative decisions going completely unchallenged. Others suggested, or revised, or fought. There can be too many cooks in the kitchen, of course, but that's an entirely different issue from executive meddling.
I'm not sure why you jump to executive involvement when I specifically stated two creatives butting heads. There's all sorts of stories where execs or someone came in and forced a movie to have, eg a giant robot spider in a wild west settint that didn't make sense but they really wanted one in some movie so they forced it to happen in one of the projects they were overseeing. but the sum of us is better than individual, so while there are solo artists out there, they're the exception rather than the rule.
I don't think that is necessarily true. Right now movies are so expensive that they can be created only by a few handfuls of people. But those people might not be the most creative people around. If thousands of people can create movies, we might find out that some people we didn't know of are far more creative.
Also "creation by committee" isn't a thing when somebody can produce a movie in their basement.
Anyway, I look forward to people using this tech to create alternative endings of existing movies.
So expensive? It has never been cheaper to create movies thanks to digital cameras and non-linear editors, digital audio workstations, etc. You are no longer encumbered by the costs of film, development, renting an edit bay, requiring an audio editing studio to mix audio and maintain a tape library of special effects or hire foley artists, no need for an optical printer to layer visual effects, etc.
You already can produce a movie in your basement, many of which can be found on YouTube.
Like with all generative AI this comes down to how specific you can get, and how consistent you can be. If you can art direct each element of the frame, down to the design of the individual props and scenic items, and have those items remain consistent from shot to shot. Then do the same with lighting, actors, camera characteristics (eg lens, focus, position in the scene, framing), etc, etc, etc, then maybe you’ve got a chance of making a ‘high budget movie from the early 2010s’. But I haven’t seen any generative ai that comes even close to this level of control or consistency… They’re closer to slot machines than anything consistent…
Yup, you can notice some issues even in their picked example. e.g. the prompt for the video of the painting woman says "there is a bear cub at her feet" and it quite clearly is not "at her feet" in the video.
The era of easily available game engines have brought to live hundreds of thousands of garbage games, but that doesn't matter. What matters are hundreds of really innovative ones that we wouldn't get otherwise.
Website doesn't work on Firefox and videos don't play on Edge.
They should consider asking the AI to make a correct website before having it make hippos swim.
Yeah it doesn't play the video for me on S10+. I can't imagine what they're doing to break that. It's just another disposable consumerist craze anyway.
Interesting perspective, considering a paper ByteDance just released yesterday [1] has much worse video quality. If your comparison is to real videos, then for sure the quality isn't great. If instead you compare to other released research, the this model is one of the best released thus far.
Yeah, some were impressive, but others looked quite bad. The guy running in the desert looked like a guy floating over the ground only sporadically making contact with the sand. The footfalls in a lot of these videos look pretty janky or "soft".
The clothing changes also have pretty rough edges, or just look like they're floating over the original model. The 3D glasses one looked atrocious. The lighting changes are also pretty lacking.
It's only going to get better, faster, cheaper, easier.[a]
Sooner than anyone could have expected, we'll be able to ask the machines: "Turn this book into a two-hour movie with the likeness of [your favorite actor/actress] in the lead role."
Sooner than anyone could have expected, we'll be able to have immersive VR experiences that are crafted to each person.
Sooner than anyone could have expected, we won't be able to identify deepfakes anymore.
Are we only a few years away from one person/small group made movies where the dialog, acting, location and special effects can be tweaked endlessly for a relatively low cost. If I was a studio exec I'd be worried.
> I wouldn't be. How is any of this going to lead to meaningful art?
Local art, local actors, local animations telling stories about local culture. A netflix for every city, even neighborhoods. That's going to be crazy fun.
There are plenty of great outsider storytellers and artists. Youtube is proof of this. People mostly do comedy on youtube because that's what the medium supports on a low budget, but AI is going to change that.
I'm really not seeing how that would happen from these examples. It would seem like achieving an adequate, directorial level of control would require writing a novel -- or, anyway, more than a conventional screenplay -- to get the AI to make the movie you wanted.
There is so much that has to be conveyed in making a film, if you want it to say something particular.
> It would seem like achieving an adequate, directorial level of control would require writing a novel -- or, anyway, more than a conventional screenplay -- to get the AI to make the movie you wanted.
And? What's the problem with that? You seem to be locked in a "prompt to get a movie" mindset.
Those are the examples provided. When they deliver pro tools for generating movie clips, I will be more convinced, but that hasn't remotely happened yet.
I'd love to see what'd happen if someone dumped the entire text of the Silmarillion or the Hobbit into one of these models. Assuming context windows and output capacity become large enough.
Especially primed by all the lord of the rings movies, for example; I could see the studio taking all the archived footage, camera angles, all the extra data that was generated in the creation of the films and feeding that into something like this model to create all kinds of interesting additional material.
Extremely-heavily-CG movies already mostly look like shit compared to ones where they build sets and have props and location shoot, even if somewhat assisted by computer compositing and such (everything is, nowadays). [edit: I don’t even mean that the graphics look bad, but the creative and artistic choices tend to be poor]
The limitations of reality seem to have a positive effect on the overall process of film making, for whatever reason. I expect generative AI film will be at least as bad. Gonna be hard to get an entire well-crafted film out of them.
You're unlikely to get an AI that wins accolades for the same reason that's unlikely with humans: they represent the absolute pinnacle of achievement.
The same AI can still raise the minimum bar for quality. Or replace YouTubers and similar while they're still learning how to be good in the first place.
No idea where we are in this whole process yet, but it's a continuum not a boolean.
What accolades? The Hollywood self-congratulatory conspicuous consumption festivals they use to show how good they are at producing "art" every year? The film festivals where billions of dollars are spent on clothing and jewelry to show off the "class" of everyone attending, which people like Weinstein used to pick victims, and everyone else uses as conspicuous consumption and "marketing" media?
Pinnacle is not the word I'd use. Race to the bottom, least possible effort, plausibly deniable quality, gross exploitation, capitalist bottom line - those are all things I'd use to describe current "art" awards like Grammy, Oscars, Cannes, etc.
The media industry is run by exploiting artists for licensing rights. The middle men and publishers add absolutely nothing to the mix. Google or Spotify or platforms arguably add value by surfacing, searching, categorizing, and so on, but not anywhere near the level of revenue capture they rationalize as their due.
When anyone and everyone can produce a film series or set of stories or song or artistic image that matches their inner artistic vision, and they're given the tools to do so without restriction or being beholden to anyone, then we're going to see high quality art and media that couldn't possibly be made in the grotesquely commercial environment we have now. These tools are as raw and rough and bad performing as they ever will be, and are only going to get better.
Shared universes of prompts and storylines and media styles and things that bring generative art and storytelling together to allow coherent social sharing and interactive media will be a thing. Kids in 10 years will be able to click and create their own cartoons and stories. Parents will be able to engage by establishing cultural parameters and maybe sneak in educational, ethical, and moral content designed around what they think is important. Artists are going to be able to produce every form of digital media and tune and tweak their vision using sophisticated tools and processes, and they're not going to be limited by budgets, politics, studio constraints, State Department limitations, wink/nod geopolitical agreements with nation states, and so on.
Art's going to get weird, and censorship will be nigh on impossible. People will create a lot of garbage, a lot of spam, low effort gifs and video memes, but more artists will be empowered than ever before, and I'm here for it.
Any accolades, be that professional groups, people's awards, rotten tomatoes or IMDB ratings.
> Race to the bottom, least possible effort, plausibly deniable quality, gross exploitation, capitalist bottom line - those are all things I'd use to describe current "art" awards like Grammy, Oscars, Cannes, etc.
I find them ridiculous in many ways, but no, one thing they're definitely not is a race to the bottom.
If you want to see what a race to the bottom looks like, The Room has a reputation for being generally terrible, "bad movie nights" are a thing, and Mystery Science Theater 3000's schtick is to poke fun at bad movies.
> The media industry is run by exploiting artists for licensing rights.
Yes
> The middle men and publishers add absolutely nothing to the mix. Google or Spotify or platforms arguably add value by surfacing, searching, categorizing, and so on, but not anywhere near the level of revenue capture they rationalize as their due.
I disagree. I think that every tech since a medium became subject to mass reproduction (different for video and audio, as early films were famously silent) has pushed things from a position close to egalitarianism towards a winner-takes-all. This includes Google: already-popular things become more popular, because Google knows you're more likely to engage with the more popular thing than the less popular thing. This dynamic also means that while anyone will be free to make their own personal vision (although most of us will have all the artistic talent of an inexperienced Tommy Wiseau), almost everyone will still only watch a handful of them.
> Art's going to get weird, and censorship will be nigh on impossible.
Bad news there, I'm afraid. AI you can run on your personal device, is quite capable of being used by the state to drive censorship at the level of your screen or your headphones.
We already exist in a world where most of the revenue for film companies comes from formulaic productions. Studio execs certainly worry about how they are going to create profits in addition to any concerns about the qualitative cultural value of the films.
> How is any of this going to lead to meaningful art?
Nearly all the movies that go to theaters aren't "meaningful art". Not only that but what's meaningful to you isn't necessarily what's meaningful to others.
If someone can get their own personal "Godzilla VS The Iron Giant" crossover made into a feature-length film it will be meaningful to them.
> Nearly all the movies that go to theaters aren't "meaningful art".
No but what they are is expensive, flashy, impressive productions which is the only reason people are comfortable paying upwards of $25 each to see them. And there's no way in the world that an AI movie is going to come anywhere close to the production quality of Godzilla vs Kong.
And like, yeah, their example videos at the posted link are impressive. How many attempts did those take? Are they going to be able to maintain continuity of a character's appearance from one shot to the next to form a coherent visual structure? How long can these shots be before the AI starts tripping over itself and forgetting how arms work?
My suspicion is that, if AI moviemaking actually becomes common, there will be a younger generation of folks who will grow up on it and become used to its peculiarities.
We will be the old ones going "back in my day, you had to actually shoot movies on a camera! And background objects had perfect continuity!" And they will roll their eyes at us and retort that nobody pays attention to background objects anyway.
But I have faith that people will notice the difference. The current generation may not care about autotune, but that doesn't mean another generation won't. People rediscover differences and decide what matters to them.
When superhero movies were new, almost everyone loved them. I was entranced. After being saturated with them... the audience dropped off. We started being dissatisfied with witty one-liners and meaningless action. Can you still sell a super-hero movie? Sure. Like all action movies, they internationalize well. But the domestic audiences are declining. It makes me think of Westerns. At one time, they were a hollywood staple. Now, not so much. Yes, they still make them, and a good one will do fine, but a mediocre one... maybe not.
> The current generation may not care about autotune
The previous generation's care about autotune was also flatly wrong. Autotune was used by a few prominent artists then and is more widely used now as an aesthetic choice, for the sound it creates which is distinctly not natural singing, as the effect was performed by running the autotune plugin at a much, much higher setting than was expected in regular use.
Tone correction occurs in basically every song production now, and you never hear it. Hell, newer tech can perform tone correction on the fly for live performances, and the actual singing being done on the stage can be swapped out on the fly with pre-recorded singing to let the performer rest, or even just lipsync the entire thing but still allow the performer to jump in when they want to and ad-lib or tweak delivery of certain parts of songs.
The autotune controversey was just wrong from end to end. When audio engineers don't want you to hear them correcting vocals, you don't hear it. I'd be willing to buy another engineer being able to hear tone correction in music, but if a layman says they do, sorry but I assume that person's full of shit.
My suspicion (and fear) is that poor members of the younger generation are going to grow up reading AI kids books and watching AI TV shows, and playing AI generated iPad games, and be less literate, less experienced, less rounded and interesting people as a result. This is already kind of a problem where under-served kids access less, experience less, and are able to do less and I see AI doing nothing but absolutely slamming the gas on that process and causing already under-served kids to be even more under-served. That human created art will be yet another luxury only afforded to the children of the advantaged classes.
And maybe they won't have a problem with it, like you say, maybe that'll just be their "normal" but that seems so fucking sad to me.
If poor kids of the future grow up reading AI book-slop instead of classic books that's going to be due to complicated factors of culture and habit rather than economic necessity. Most of the traditional Western canon of "great literature" is already in the public domain, available for free.
It gives anyone with a participating-system library card free electronic access to books and magazines. And it's unlikely that librarians themselves will be adding AI book-slop to the title selection.
> If poor kids of the future grow up reading AI book-slop instead of classic books that's going to be due to complicated factors of culture and habit rather than economic necessity.
To be clear, I'm not talking great literature. I'm talking Clifford the Big Red Dog type stuff.
That said I still have a number of problems with this assertion:
It will absolutely be down, in part, to economic necessity. Amazon's platform is already dealing with a glut of shitty AI books and the key way they get ahead in rankings is being cheaper than human-created alternatives, and they can be cheaper because having an AI slop something out is way less expensive and time consuming than someone writing/illustrating a kid's book.
Moreover, our economy runs on the notion that the easier something is to do, the more likely people are to do it at scale, and vetting your kids media is hard and annoying as a parent at the best of times: if you come home from working your second job and are ready to collapse, are you going to prepare a nutritious meal for your child and set them up with insightful, interesting media? No you're going to heat up chicken nuggets and put them in front of the iPad. That's not good, but like, what do you expect poor parents to do here? Invent more time in the day so they can better raise their child while they're in the societal fuckbarrel?
And yes, before it goes into that direction, yes this is all down to the choices of these parents, both to have children they don't really have the resources to raise (though recent changes to US law complicates that choice but that's a whole other can of worms) and them not taking the time to do it and all the rest, yes, all of these parents could and arguably should be making better choices. But ALSO, I do not see how it is a positive for our society to let people be fucked over like this constantly. What do we GAIN from this? As far as I can tell, the only people who gain anything from the exhausted-lower-classes-industrial-complex are the same rich assholes who gain from everything else being terrible, and I dunno, maybe they could just take one for the team? Maybe we build a society focused on helping people instead of giving the rich yet another leg up they don't need?
...if you come home from working your second job and are ready to collapse, are you going to prepare a nutritious meal for your child and set them up with insightful, interesting media? No you're going to heat up chicken nuggets and put them in front of the iPad.
This is what I mean by "complicated factors of culture and habit." An iPad costs more than an assortment of paper books. Frozen chicken nuggets cost more than basic ingredients. But the iPad and nuggets are faster and more convenient. The kids-get-iPad-and-nuggets habit is popular with middle-income American families too, not just poor families where parents work two jobs. The economic explanation is too reductive.
I'm not trying to say that this is the "fault" of parents or of anyone in particular. When the iPad came out I doubt that Apple engineers or executives thought "now parents can spend less time engaging with children" or that parents thought of it as "a way to keep the kids quiet while I browse Pinterest" but here we are.
I mean, that's the thing though. We now have had kids parked in front of iPads for a good amount of time, along with other technical innovations like social media, and we have documented scientific proof of the harms they do to children's self-esteem, focus and mental acuity. I don't think the designers of the iPad or even the engineers at Facebook set out to cause these issues, but. they. did. And now we have a fresh technology in the form of AI that whole swathes of "entrepreneurs" are ready to toss into more children's brains as these previous ones were.
Is it too much to ask for a hint of caution with regard to our most vulnerable populations brains?
As a former iPad (OS) designer, and former Facebook feed engineer, of course we're upset about what happened. Most of us fought valiantly, with awareness, against what became the dark forces and antisocial antipatterns. But the promo-culture performance incentive system instituted by HR being based on growth metrics at all costs made all of us powerless to stop it. Do something good for the world, miss your promo or get fired.
Circa 2020 a huge number of fed-up good-intentioned engineers and designers quit. It had no effect, at all.
I'm genuinely sorry that happened to you. That had to be an absolute nightmare of an experience.
To be clear: I am not saying that engineers need to be better at preventing this stuff. I am saying regulators need to demand that companies be careful, and study how this stuff is going to affect people, not just yeet it into the culture and see what happens.
I was there (Apple) at the time. Absolutely did NOT expect this thing that Steve thought was a neat way to see the whole NYT front page at once, was going to be the defining MacGuffin of an entire generation of children.
There's already conversation in AI art about how "Y'all will miss all these weird AI glitches when they're gone!" It will become the new tape hiss. Something people will nostalgically simulate in later media that doesn't have it naturally.
They are art compared to getting uncontrolled choices. Who decides what the actors look like? How they move? What emotions their faces are to convey? How the blocking works for a scene? What the color scheme is for the movie? How each shot is taken? Etc.
There is a vast difference between a formulaic hollywood movie and some guy with a camera. If I say "Godzilla vs. The Iron Giant" what is the plot? Who is the good guy? Why does the conflict take place?
AI will come up with something. Will it be compelling even to the audience of one?
As a toy, maybe. As an artistic experience... not convinced.
Some of it is done that way, but by no means all of it. You can easily see the differences, because, say, Wes Anderson movies are not the same as Martin Scorsese movies.
If it were really all just market decisions, directors would have no influence. This is not remotely the case. Nor are they paid as though that were true.
When you fabricate points and statements that work better with your counterpoint than what they actually said, it's a straw man argument. That's what you've got there. That man is definitely made of straw.
This is a common perspective among people that don't realize how much goes into making a movie. That stuff informs which movies get approved and it certainly can inform broader script changes, casting changes, and in some cases editing decisions, but there's a UNIVERSE of other artistic decisions that need to be made. Implying that the people involved are mere technicians implementing a marketing strategy is exponentially more reductive than saying developers and designers aren't relevant to making software because marketing surveys dictate the feature development timeline. A developer's input is far more fungible than an artist working on a feature film.
I assure you, they don't do surveys on the punchiness and strategy used by foley artists; the slope and toe of the film stock chosen for cut scenes by the DP or that those cut scenes should be shot like cut scenes instead of dream sequences; the kind of cars they use; how energetic the explosions are; clothing selection and how the costumes change situationally or throughout the film; indescribably nuanced changes in the actor's delivery; what fonts go on the signs; which props they use in all of the sets and the strategies they use to weather things; what specific locations they shoot at within an area and which direction they point the camera, how the grading might change the mood and imply thematic connections, subtle symbolism used, the specifics of camera movements, focus, and depth of field, and then there's the deeeep world of lighting... All of those things and a million others are contributions from individual artists contributing their own art in one big collaborative project.
> Implying that the people involved are mere technicians implementing a marketing strategy
Well no. Instead, I am implying that they are as much of a "technician" so someone who is putting in a huge amount of effort into making AI videos.
If you want to say that it is perfectly possible for someone to put a high amount of vision and make a large amount of creative decisions into AI videos, then I agree.
> All of those things and a million others are contributions
Yes I agree that there can be a million other contributions to making AI videos. Glad you agree too.
Your sarcastic, bad faith, I-know-you-are-but-what-am-I-level statements attack positions I dont hold, yet avoid addressing anything I said— all of which is based on my professional experience with using generative AI in media creation, and also film production pipelines and workflows. Your retort might have made you feel better about making baseless statements but it certainly didn't lend credibility to those statements or you in anybody else's eyes.
Never in my entire life have I said it's not possible for the AI image generation process to include a lot of creative decisions. In fact, I've repeatedly said the opposite. Like I said, you're attacking positions I don't have.
I use prompt-based generative AI in creative ways as part of my professional processes all the time. And you know what? They are nowhere close to being useful for generating anything that plays a significant on-screen role in high-end media creation. Anybody who says they are does not know how different the requirements for high-end use cases are.
You're using me as a proxy to toe the line of a ridiculous ideological polemic that I have nothing to do with. If you want to argue with someone that has the uniform, standard-issue set of anti-AI opinions you expect to encounter so you don't have to consider your arguments too much, there's like thousands of them right over on reddit. Easy.
You seem to have forgotten what this conversation is about in your herculean effort to still feel right about saying something utterly ridiculous. You said this:
> You still aren't getting it. Movie directors aren't making these decisions either.
> What they are doing, is listening to market focus groups and checking off boxes based on the data from that.
> A market focus group driven decisions for a movie is just as much, if not more so, of an "algorithm" than when a literal computer makes the decision.
> Thats not art. Its the same as if a human manually did an algorithm by hand and used that to make a movie.
I responded as a professional in the field pointing out how ridiculous that take is, and then everything else that you said is putting words in my mouth based on opinions you assumed I had, but do not. I'm not arguing a "side" here-- I'm pointing out that something you said about practices in my field is entirely baseless. I have too much of a professional stake in this to pick a "side" because I actually have to deliver great work to spec, on time, and can't be bothered to field a whole bunch of people either with dunning-kruger confidence in their understanding technology or dunning-kruger confidence in understanding art having the nerve to be condescending while making entirely baseless, glib comments about what I do for a living, and acting like the righteousness of their cause makes it ok to be full of shit. If you want to be able to argue a "side" where you're just vaguely responsible for your grand idea and it doesn't really matter if you're full of it because nobody else there knows what they're talking about, either, then reddit is a tiny little ascii string down the street.
> Never in my entire life have I said it's not possible for the AI image generation process to include a lot of creative decisions. In fact, I've repeatedly said the opposite.
Hey well you said this. This is agreeing with what my point is, and I am glad we were able to clear up any possible misunderstandings, or I convinced you or similar.
As far as I am concerned you don't have any disagreements with my central point, which is good enough for me.
I am glad we cleared up the main miscommunication.
Cool. Nothing says intellectual confidence like refusing to acknowledge someone pointing out that you're spouting complete nonsense. Glad you were able to protect your ego by deciding I was talking about something that was more convenient for you. I sure hope you decide not to challenge your Dunning-Kruger confidence in backing up your ideas with "facts" and "information" based on the "content-aware fill" your brain uses in lieu of actual knowledge of movies as an artform and professional media production. I'm really happy that you think your "central point" means you can use naive assumptions lacking requisite information by orders of magnitude to condescendingly disparage real people's jobs and artistic practices. Surely, lacking knowledge of commercial art production doesn't negatively affect your ability to reason about the usage and effects of generative AI in commercial art production on both a practical and philosophical level. Surely. I hope you'll continue to pontificate about the finer points of this topic while refusing to consider dramatically more informed sources if they don't completely match your line of reasoning.
But what they're describing is a case where someone with the storytelling ability but not the money or technical skills could create something that looks solid.
You're imagining "pls write film" but the case of being able to film something and then adjust and tweak it, easily change backdrops etc could lead to much higher polish on creations from smaller producers.
Would the green mile be any less hard hitting if the lights flickering were caused by an AI alteration to a scene? If the mouse was created purely by a machine?
I don't have a problem with adjusting small elements of the film, but that isn't going to make it a tool for youtubers (or home users off the grid) to tell their own stories.
Looking back on history I think this will lead to meaningful art (and tons and tons of absolute garbage!).
The printing press led to publishing works being reachable by more people so we got tons of garbage but we also got those few individual geniuses that previously wouldn't have been able to get their works out.
I see similarities in indie video/PC games recently too. Once the tech got to the point that an individual or small group could create a game, we got tons of absolute garbage but also games like Cave Story and Stardew Valley (both single creators IIRC).
Anything that pushes the bar down on the money and effort needed to make something will result in way more of it being made. It also hopefully makes it possible for those rare geniuses to give us their output without the dilution of having to go through bigger groups first.
I'm also excited from the perspective that this decouples skills in the creative process. There have to be people out there with tremendous story telling and movie making skills who don't have the resources/connections to produce what they're capable of.
The printing press enabled the artistic visions of single individuals (the writers) to find a wide audience.
To do something similar, this has to allow the director (or whomever is prompting the AI) to control all meaningful choices so that they get more or less the movie the intend. That seems far away from what is demonstrated.
You don't get "The Green Mile" from this, because it's a tool. You get "The Green Mile" with artistic vision. The tool has to be told what to do. But now a director can shoot a film with actors who don't match the physical description of a character in a story, and then correct their race/gender/figure/whatever with AI. That probably means they save money on casting. A director can shoot a scene inside a blank set and turn it into a palace with AI. That saves money from shooting on location or saves money from having to pay for expensive sets.
So now a director with a limited budget but with a good vision and understanding of the tools available has a better chance to realize their vision. There will be tons of crap put out by this tool as well. But I think/hope that at least one person uses it well.
But because it will make shooting a movie more accessible to people with limited budgets, the movie studios, who literally gatekeep access to their sets and moviemaking equipment, are going to have a smaller moat. The distribution channels will still need to select good films to show in theaters, TV, and streaming, but the industry will probably be changing in a few years if this development keeps pace.
This is the best answer I've seen, but I think what was demonstrated is miles away from this. A lot would need to be able to be specified (and honored) from the prompts, far more than any examples have demonstrated.
I'm not against tools for directors, but the thing is, directors tell actors things and get results. Directors hire cinematographers and work with them to get the shots they want. Etc. How does that happen here?
Also, as someone else mentioned, there is the general problem that heavily CG movies tend to look... fake and uncompelling. The real world is somehow just realer than CG. So that also has to be factored into this.
I think it starts simple. Have you ever been watching a movie or tv show, and it shows the people walking up to the helicopter or Lamborghini and then cut to "they've arrived at their destination no transportation in sight"?
It will start out with more believable green screen backgrounds and b roll. Used judiciously, it will improve immersion and cost <$10 instead of thousands. The actors and normal shots will still be the focus, but the elements that make things more believable will be cheaper to add.
Have you ever noticed that explosions look good? Even in hobby films? At some point it became easy to add a surprisingly good looking explosion in post. The same thing will happen here, but for an increasing amount of stuff.
That I could believe, although... there is quite a bit of commentary from film buffs that lots of the stuff done in post doesn't quite look right, compared to older films.
Which doesn't mean it won't keep happening (economics), but it doesn't necessarily mean any improvement in movie quality.
Interesting that you pick that example in particular. Due to the sheer depth of behinds the scenes takes HBO has provided for Game of Thrones and House of the Dragon, it seems to be the consensus view among effects folks that CG fire and explosions are nearly impossible to get right and real fire is still the way to go.
My guess is the art form will evolve. When YouTube started, some people thought it would not be able to compete with heavily produced video content. Instead, YouTube spawned a different type of "movie". It was short-form, filmed on phone cameras, lightly scripted, etc. The medium changed the content. I suspect we will see new genres of video content show up once this tech is widely available.
The first real movies made 100 years ago looked like something someone today could put together in their garage on a shoestring budget. AI-generated movies have existed for just two years, and are only going to get better. This is bleeding edge research, and I haven't seen any sign yet of AI models hitting a quality ceiling.
If it becomes easy to make "Deadpool & Wolverine", it will no longer be where the money is. Everything that becomes a commodity attracts competition and ceases to be special. (You can see some of that in super-hero movies, which have started to be generic and lost some of their audience.)
But, in reality, even making that kind of film is miles away from these examples.
> If it becomes easy to make "Deadpool & Wolverine", it will no longer be where the money is. Everything that becomes a commodity attracts competition and ceases to be special. (You can see some of that in super-hero movies, which have started to be generic and lost some of their audience.)
Well, given the studios still hold the copyright, they can severely constrain supply to keep profits up.
My suspicion is that this kind of stuff gradually reduces some of the labor involved in making films and allows studios to continue padding their margins.
> How is any of this going to lead to meaningful art?
It's a powerful tool. A painting isn't better because the artist made their own paint. A movie made with IRL camera may not be better than one made with an AI camera.
From this and other comments, I get the impression that most on HN assume the tool will be used exclusively by people without any sort of artistic talent, either plagiarizing existing works and/or producing absolute dreck.
However, I see an interesting middle ground appear: a talented writer could utilize the AI tooling to produce a movie based upon their own works without having to involve Hollywood, both giving more writers a chance to put their works in front of an audience as well as ensuring what's produced more closely matches their material (for better or worse).
Shameless plug: I just created a short AI film (1) and tried to tell an actual story and trigger emotions. I spent countless hours crafting the script, choosing the shots, refining my prompts, generating images, animating images, generating music, sounds, and so on... For me AI tools are just that - tools. True, you have to yield them some "control", but at the end of the day you are still the one guiding and directing them.
Similarily, a film director "just" gives guidance to a bunch of people: actors, camera operator, etc. Do you consider the movie is his creation, even if he didn't directly perform any action?
A photographer just has to push a button and the camera captures an image. Is the output still considered his creation?
Yes and Yes, so I think we should consider the same with AI assisted art forms.
Maybe the real topic is the level of depth and sophistication in the art (just like the difference between your iPhone pictures and a professional photographer's) but in my opinion this is orthogonal to it being human or AI generated.
To be honest so far we have mostly seen AI video demos which were indeed quite uninteresting and shallow, but now filmmakers are busy learning how to harness these tools, so my prediction is that in no time you will see high quality and captivating AI generated films.
this is an excellent example that despite all the technical limitations (the ugly image artifacts, the lack of exact image consistency, etc), it's _already possible_ to create something that connects. The "format purists" currently dismiss AI tooling the same way they used to dismiss computer graphics animation back when Toy Story 1 came out.
I, uh, gave some more nuance because i had some free time as a sibling comment to this. I hope we don't get downvoted because someone has to call a spade a spade.
good comment, haha. agree with those points and would add, since im thinking about it again now, that the entire work feels like a fairly (deeply) shallow riff on the opening sequence in pixar's "up". but of course with no stakes or emotional impact whatsoever.
sure it is. that's my critical evaluation of this work. if you liked it, i highly recommend the hallmark channel, lifetime, family channel originals, the netflix straight-to-vod swimlane, and a frontal lobotomy.
I'm in agreement with scudsworth here, but i have a little more nuance, i think. I know how long this took, and how many compromises were made. The only reason this works, at all, is because humans have a massive list of cultural memes and tropes that shorthand "experience" for us. it has the "AI can only generate 2-5 seconds of video before it goes completely off the rails" vibe; which allows it to fit in with the ADHD nature of most video production of the last 30 years - something a lot of people do not like. For an example where this was jarring in the video, when they're drawing or painting, you cut the scene slightly too late, you can see the AI was about to do some wild nonsense.
What happened to the mom? Why does the kid get older and younger looking? why does the city flicker in the beginning? which kid is his in the ballet performance? why do they randomly have "lazy eye"? i could keep going but i think we all get my point.
I can intuit the tropes used by the AI to convey meaning, and i'd be willing to list them all with relevant links for the paltry sum of $50. Be warned, it will be a very large list. Tropes and "memes" are doing 100% of the heavy lifting of this "art".
Sorry, human. As someone who stopped creating art on a daily basis due to market dilution (read: it's too hard to build a fanbase that i care about), i am very critical of most "art" produced anyhow.
to be fair, that's exactly what the asset class EXISTS for - betting on huge outcomes, no matter the odds. People misunderstand that due to how much of tech is "VC funded" when building stuff that would fare better as a bootstrapped company (or funded by other means)
If we judge from AI writing, we can extrapolate what an AI movie would be like. I cannot imagine reading an AI book. It would look and smell like a book but nothing of value or new insight would be inside. Michael Bay might be very interested.
You look back at old movies, and on a technical level they really aren’t as good as contemporary trash productions. But they knew how to weave the camera and a script into something amazing back then even if they didn’t have resolve and aftereffects to polish every shot. A good script writer, editor and cinematographer have a huge impact on the quality of a movie. But these roles are only a small portion of the operating budget of a movie. Filming every single scene is an exhausting undertaking and this constitutes the bulk of a movie production’s budget. If you can get good quality footage without leaving your garage then you can have a small team make a great movie. Maybe not the extent where you simply click a button but to the level that you would launch straight to a streaming service.
Yes, AI will probably fail miserably for a while at least, at making the sort of well written artistic, clever movie that nobody watches. The only ones that need to be worried are the studios making churning out massive blockbusters…
I don’t know, for a car the last 10% has a direct relation with "people die" that is obvious to everyone. With the movie made in anyone basement, the risks are far less likely to create such a vivid perception of dramatic end result.
Not that cyber-bullying and usurpation schemes escalating a whole new level being less of a concern in the aftermaths, to be clear.
Less about risk parallels and more about control parallels. The last 10% of fine grained control over a system is hard. Like every time I’ve done text to image prompting and it gives me a great starting point, but cannot get certain details i want, no matter how i ask.
If you look at the majority of their catalogue these days, they really aren't trying to squeeze that last 10% out of the movie quality these days anyways, so I doubt it will matter.
The average person spends 9-11 hours per day consuming media depending on what source you look at. When people are playing games or browsing social media at the same time that they have the latest Netflix show on their TV, you can't tell me that this is really valuable time spent to deepen one's understanding of the human experience; it's a replacement for the human experience.
Most people will not notice if the soundtrack to a new TV show is made by a 5 word AI prompt of "exciting build-up suspense scene music" while they're playing pouring money into their mobile gacha game to get the "cute girl, anime, {color} {outfit}" prompt picture that is SSS rank.
You or I might not care for AI slop, but it's a lot cheaper to produce for Netflix or Zinga or Spotify or whatever, and if they go this route, they don't have to pay for writers, actors, illustrators, songwriters, or licensing for someone else's product. They'll just put their own AI content on autoplay after what you're currently watching, and hope most people don't care enough to stop it and choose something else.
Counterpoint: home "studio" recording has been feasible for decades, but music execs are not ruffled. Sure, you get a Billie Eilish debut album once a generation, but the other 99.99% of charting music is from the old guard. The media/entertainment machine is so much bigger than just creating raw material.
I think AI porn is overhyped. We've had the ability to create realistic photos and short vids for over a year now and onlyfans creators are still doing fine. AI porn is just a niche for stuff that humans don't want to perform.
I think AI porn is chocked full of fascinating moral quandaries. It kind of transcends all other types of GenAI for the amount of hard questions it asks society.
As they say, porn is always the leading spear of technology. It's something to keep an eye on (no pun intended) to understand how society will accept/integrate generated content.
It's definitely interesting from a moral and tech perspective.
However, commercially it seems like a niche within the existing structures of porn. Mostly competing with the market for animated stuff. At least that's where it is right now, and its already at photorealistic parity with human content creators.
We have not had the ability to create interesting ai porn vids yet. How would we? Meta just showed movie gen.
But i'm pretty sure the short images of moving woman very subtle might gotten the one or other of. Just wait a little bit until you can really create wat you are looking for.
You obviously have not spent any time on civit.ai if you're saying that.
What scares me most is that in my opinion, by far the best prompt writers are the ones who are deeply "motivated" and "experienced" with prompting. Often the best prompters have only one hand on the keyboard at a time.
You have no idea how much butthurt their is from specifically artists who draw at the AI NSFW models which exist.
I can trivially fine-tune and create more art from certain artists in an hour than they have produced themselves in their whole careers. This makes a lot of people very upset.
In enthusiasm perhaps, but when I play that in my own head with the voice of Dr. Károly Zsolnai-Fehér, he doesn't script his videos like that. I can't recall a single instance of him using triple repetition like that.
This sounds gimmicky and worth watching once or twice, then forgetting about. Worthwhile art will continue to be created from a specific person's/group's vision, not an algorithmically generated sum of personal preferences.
"Turn this book into a two-hour movie with the likeness of [your favorite actor/actress] in the lead role."
This sounds marginally above fanfiction, so I do think it'll be very easy to tell them apart. "Terminator, except with Adam Sandler and set on Mars" is a cute, gimmicky idea, not a competitor to serious work.
Maybe, but I guarantee you this is going to get banned in the US for "safety" or "misinformation" reasons eventually (with large backing from Hollywood).
Well, yeah. If you explicitly try to come up with a cute, gimmicky idea, it's not going to be serious. Taste still matters regardless of paint, cameras or computers.
I think that’s an extremely short sited perspective. There isn’t much that separates a “fun and wacky” movie from something impactful from a cinematography perspective. With the right music, ambience and script you could absolutely do any genre of movie you wish to.
I disagree, I believe your perspective is short-sighted. If you really think what makes a movie "impactful" is the music, ambience, and script then I don't think you have much media literacy.
It's no more ridiculous than saying what makes a painting impactful is the brush strokes. But if I copy Picasso's work stroke for stroke, why am I not Picasso? After all, the dumbass paints like a child, admittedly! How could someone like him ever be considered a great painter?
If they are stitching then I would consider that a form of art.
However, merely describing something is not doing the thing. Otherwise, the business analysists at my company would be software engineers. No, I make the software, and they describe it.
The end-goal here is humanless automation, no? Then I'm not sure your assumption holds up. If there's no human, I question the value.
You may question the value but if it’s anything like rugs you won’t be in the majority. People pay a significant premium for artisanal handmade rugs but that being said, more than 95% of the rugs people use are machine made because they’re essentially indistinguishable from a handmade one and are much, much cheaper and just as functional.
I'm sure eventually you can, but I don't think triggering emotions is the correct "KPI" so to speak.
On social media platforms, typically the most popular content triggers the strongest emotions. It's rage-bait however, or sadness bait, or any other kind of emotional manipulation. It tricks the human mind and drives up engagement, but I don't think that is indicative of its value.
To be clear, I'm certain that's not what you're doing, and the music is good. But I think it's complicated enough that triggering emotions isn't enough data to ascertain value.
I don't know, exactly, what combination of measurements are needed to ascertain value. But I'm confident human-ness is part of the equation. I think if people are even aware of the fact a human didn't make something they lose interest. That makes the future of AI in entertainment dicey, and I think that's what fuels the constant dishonesty around AI we're seeing right now. Art is funny in how it works because, I think, intention does matter. And knowledge about the intention matters, too. It maybe doesn't make much sense, but that's how I see it.
Right now there is a ton of stigma around AI art. That stigma fuels a ton of poorly-informed rhetoric against it. There is also tons and tons of casual use of AI art being shitposted for funsies everywhere that reinforces that rhetoric that AI art means "Push button. Receive crap. Repeat."
Meanwhile, as someone who has been engaged with the AI art community for years, and spent years volunteering part-time as a content moderator for Midjourney, the process of creating art via AI with intentionality is deeply human.
As an MJ mod, I have seeeeeeen things.... It's like browsing though people's psyche. Even in public portfolios people bare their souls because they assume no one will bother to look. People use AI to process the world, their lives, their desires, their trauma. So much of it is straight-up self-directed art therapy. Pages and pages, thousands of images stretching over weeks, sometimes months, of digging into the depths of their selves.
Now go through that process to make something you intend to speak publicly from the depth of your own soul. You don't see much of that day to day because it is difficult. It's risky at a deeply personal level to expose yourself like that.
But, be honest: How much deeply personal art do you see day to day? You see tons of ads and memes. But, to find "real art" you have to explicitly dig for it. Shitposting AI images is as fun and easy as shitposting images from meme generators. So, no surprise you see floods of shitposts everywhere. But, when was the last time you explicitly searched out meaningful AI art?
> But, be honest: How much deeply personal art do you see day to day?
You bring up a good point - very little. But, to be fair, those people aren't necessarily trying to convince me it's art.
I think you're mostly right but I am a little caught up on the details. I think it's mostly a thing of where the process is so different, and involves no physical strokes or manipulation, that I doubt it. And maybe that's incorrect.
However, I will also see a lot of people who don't know how to do art pretending like they've figured it all out. I also see the problem with that. It wouldn't be such a problem if people didn't take such an overly-confident stance in their abilities. I mean, it's a little offensive for that guy mucking around for an hour to act like he's DiVinci. And maybe he's a minority, I wouldn't know, I don't have that kind of visibility into the space.
I think a lot of the friction comes from that. Shitposts are shitposts, but I mean... we call them shitposts, you know? They, the people that make them, call them shitposts. There's a level of humility there I haven't necessarily seen with "AI Bros".
I think, if you really love art, AI can be a means to create a product but it can also be a starting point to explore the space. Explore styles, explore technique, explore the history. And I think that might be missing in some cases.
For a personal example, I'm really into fashion and style. I love clothes and always have. But it's really been an inspiration to me to create clothes, to sew. I've done hand sewing, many machine stitches too. And I don't need to - I could explore this in a more "high-level" context, and just curate clothing. But I think there's value in learning the smaller actions, including the obsolete ones.
Not really. A vision implies a particular kind of project, presumably created by someone with expertise and some well-thought through ideas about what it ought to be. Personal preferences just mean that someone likes X qualities.
To use a real-world example: if the Renaissance-era patrons had merely written down their preferences and had work made to match those preferences, it's highly unlikely that you'd have gotten the Mona Lisa or David.
Which is to say that, there will definitely be some interesting and compelling art made with AI tools. But it will be made by a specific person with an artistic vision in mind, and not merely an algorithm checking boxes.
I rarely watch movies or read books twice anymore. There's too much content already. The challenge with purely human art at this point is that it will be silenced by the perpetual flood of half-assed generated work. There will be room in elite art circles for more, but at some point the generated stuff will be so ubiquitous (and even meaningful) that anyone without connections is going to have a tough time building an audience for their handcrafted work, unless it happens to be particularly controversial or 'difficult' to make. The demand for visual stimulus will be satisfied by hypertuned AI models. Generative AI is not there quite yet but there's no reason to think it won't be better than 90%+ of purely human content within a decade given the pace of development over the last few years.
I don't buy this narrative at all. People like people and increasingly follow artists because of their personality and overall "brand." No one cares about generated AI art or its creator(s), because it's not interesting. It's also not sharable with other humans; see, for example, the frenzy around going to a Taylor Swift concert. The mass appeal and shared interest is part of the draw.
At best, you'll get something like a generic sitcom. The idea that "all visual stimulus will be satisfied by hypertuned AI models" doesn't line up with how people experience the arts, at all.
I fully agree here. I want to be part of an audience, and as part of that audience I always look at the human development of the things to share - artifacts in the case of fine art, or experiences in the case of performative art. The artist will always be more important than their work to me.
I don't want to carry mechanical solutions labelled culture - deterministic enough, despite hallucinations - into the next generation that follows my own. It's an impressive advancement for automation, sure, but just not a value worth sharing as human development.
That being said, I think GenAI could be a valuable addition in any blueprint-/prototype-/wireframing phase. But, ironically, it positions itself in stark contrast to what I would consider my standards to contemporary brainstorming, considering the current Zeitgeist:
- truthful to history and research (GenAI is marketing and propaganda)
- aware of resources (GenAI is wasteful computing)
- materialistic beyond mere capitalistic gains (GenAI produces short-lived digital data output and isn't really worth anything)
That may be the case today but kids are starting to grow up with this stuff as part of their lives, and I don't think we can anticipate the reaction as both they and the models grow in tandem. I think human creativity is much deeper than LLMs, but that is from my human perspective and I can't fully rule out that the LLMs may become better at it at some point in the future. I actually think they're already smarter and more creative than most people (though not more than the potential of any given human if they practiced/trained thoroughly).
> Sooner than anyone could have expected, we'll be able to ask the machines: "Turn this book into a two-hour movie with the likeness of [your favorite actor/actress] in the lead role."
I've been doing this with ChatGPT, except it's more of a "turn into a screenplay" then "create a graphic of each scene" and telling it how I want each character to look. It's works pretty well but results in more of a graphic novel than a movie. I'm definitely been waiting for the video version to be available!
Does anyone other than PMs when thinking up user stories do shit like this or finds this kinda stuff desirable? It just sounds like a business person who doesn't have a life other than selling their product trying to think up "real user" usecases every time.
Nope. This is just like when cryptobros would regularly insist that cryptocurrencies would replace banks by the end of the decade. It's safe to assume that anyone who makes such wild predictions is a bagholder who stands to gain financially from said wild predictions coming true, even though they never will.
Many creative works these days require the effort and input of so many people, so much time, and so much money that they can't have a specific creative vision. Mediums like book, comics, indie movies, and very low budget indie games, where the the end product was created by the smallest number of people, have the most potential to be interesting and creative. They can take risks. This doesn't mean they will be good, most aren't, but it means that the range of quality is much broader, with some having a chance to shine in ways which big budget projects just can't. The issue with small teams and small budgets is that they are inherently limited in what they can create. Better tools allows smaller groups of people to make things that previously would have required an entire studio but without diluting the creative vision.
Will this also result in a tidal wave of low effort garbage? Of course it will. But that can be ignored.
Incredible, simply incredible. You know a paper is seminal when all the methods seem obvious in hindsight! Though I’m not caught up on SOTA, so maybe some of this is obvious in normal-sight, too.
RIP Pika and ElevenLabs… tho I guess they always can offer convenience and top tier UX. Still, gotta imagine they’re panicking this morning!
Upload an image of yourself and transform it into a personalized video. Movie Gen’s cutting-edge model lets you create personalized videos that preserve human identity and motion.
Given how effective the still images of Trump saving people in floodwater and fixing electrical poles have been despite being identifiable as AI if you look closely (or think…), this is going to be nuts. 16 seconds is more than enough to convince people, I’m guessing the average video watch time is much less than that on social media.
Also, YouTube shorts (and whatever Meta’s version is) is about to get even worse, yet also probably more addicting! It would be hard to explain to an alien why we got so unreasonably good at optimal content to keep people scrolling. Imagine an automated YouTube channel running 24/7 A/B experiments for some set of audiences…
Absolutely agree. It's very terrifying and will likely cause mass disruption because it will disintegrate the social fabric that is held together by people needing other people for stuff.
The luddites wanted sane working conditions and to be trained on the use of newly introduced, dangerous machinery that was maiming and killing people and was meant to replace swathes of them overnight to make some rich capitalists richer.
The end result for them wanting humane conditions was getting murdered by the machine owners & the state.
Perhaps the rabidly pro-AI people shouldn't be on the side of the murderous psychopaths who wanted to extract maximum profit by employing children and displacing thousands of people if they don't want to be viewed as similarly psychotic.
Off topic but some day you could live off grid with your own solar fusion mini reactor powering your own hardware that enables creating your own stories, movies and tales. No more need of streaming services. Internet would be to obtain news, goods and buy greatest and latest (or not) data to update your models.
Decentralization could be for once not as painful as it is now; however, I still believe every single hardware vendor would try to hook to the internet and make you install an app. Looking forward to this AI revolution for sure.
I'm not. The likelihood that such movies (for example) would have anything significant to say about being human seems very low.
If one watches movies, reads books, etc. just to pass the time, maybe this would be some kind of boon. But for those of us looking for meaningful commentary on life, looking to connect with other human beings, this would be some circle of hell. It's some kind of solipsism.
I don't have anything against stitching together clips to tell your story, but I'm unconvinced that these demonstrate anything like that. As I said in another comment, it seems like you'd need to write a screenplay PLUS all the information the director, cinematographer, etc. use to create an actual movie -- everything from direction for how actors portray scenes to decisions on exactly how shots are constructed, to blocking for multiple actors in a scene, to color schemes...
There are a LOT of choices in making a movie, and if you just let the AI make them, you are getting "random" (uncontrolled) choices. I don't think that is going to compare favorably to the real thing.
If you can specify all that, then it's just a tool. Cool. But it's still going to take pro-level skills to use it.
If La Jetee was just some photos stitched together plus meaningful narration, then of course, you could use AI-generated photos.
But would AI be able to quote Vertigo, like La Jetee does? Doesn't art, at least to some degree, require intent (including all intentional subversions of that intent dogma, of course)?
I wouldn't be so sure. AI can ingest far more information about humans than a human ever could. It has read our stories and understands our languages. AI might have more to say about humans than we do ourselves.
Of course AI can never truly experience being human, it has no emotions, but it is excellent at mimicry and it can certainly provide a meaningful outside perspective.
Is there anything to say about humanity that is not in the training corpus already?
Every new novel of any merit shows that there is. And the world keeps changing. The experience of being human keeps changing.
Nothing AI has yet done has demonstrated anything at the level of art or mastery. I guess I'm unconvinced that throwing a million stories into the blender and synthesizing is going to produce a compelling one.
Maybe people with good story literacy and cultural comprehension will be able to tell the difference for much longer, maybe even indefinitely. But the majority of people, and I dread that includes me, won't, at some point. I've already fallen for some AI generated music and thought "hey, that sounds pretty good, I'll bookmark it". It's genuinely scary.
95% of television and movies, to me, are completely uninteresting and not worth watching. the property of being human-made has a pretty low success rate for basically anyone
To me, this feels like a very dystopian take. I watch movies, read books, and listen to music because they are a way to connect with fellow human beings. Taking the human out of the equation also removes any meaning for me.
I get that this is kind of a fundamental line in the sand for most of the "AI art" going around, and it seems like most people fall on one side or the other. "I consume art for entertainment" vs "I interact with art to experience the human condition".
I also don't want to say that AI Art has no value, because I think as a tool to help artists realize their vision it can be very useful! I just don't think that art entirely made by AI is interesting.
Surely if you’ve watched an amazing show you’re likely to share it with a friend, no? I see this bringing likeminded people together in tight, niche communities.
Are any image / video generation tools giving just the output or the layers, timelines, transitions, audio as things to work with in our old fashioned toolsets?
The problem:
In my limited playing of these tools they don't quite make the mark and I would easily be able to tweak something if I had all the layers used. I imagine in the future products could be used to tweak this to match what I think the output should be....
At least the code generation tools are providing source code. Imagine them only giving compiled bytecode.
Keep in mind that these technologies produce more stuff like what they've been trained on, and they need tremendous amounts of training data to pull that off.
It so happens that there are innumerable samples of prose and source code and rendered songs and videos and images to use as this training data.
But that's not so much the case for professional workflows (outside of software development).
If the tools can evolve to generating usefully detailed and coherent media projects instead of just perceptually convincing media assets, it's going to be a while before they get there.
They definitely do not give you an Adobe After Effects project. This is because of the way they are trained. I suspect a vast proportion of its training data is not annotated with the corresponding layers, timelines, etc so the model is unable to reproduce it like that. You basically just get video AFAIK.
If you have experience as a graphic designer, you can get very far with any layer based graphic tools like Krita or Affinity in conjunction with proper inpainting against generative image models - in fact that's InvokeAI's entire target user base.
There are some approaches that use an LLM to generate “scripts” (you can think of them as a DSL) for composing/arranging media, essentially driving other models to generate parts of the media. One example is WavJourney: https://audio-agi.github.io/WavJourney_demopage/
We live in the future. I just hope we consumers get easy access to these video tools at some point. I want to make personal movies from my favorite books
Yet another one-shot, single-clip Instagram machine that can't do a follow-on shot natively.
As it stands, the only chance you have of depicting a consistent story across a series of shots is image-to-video, presuming you can use LoRAs or similar techniques to get the seed photos consistent in themselves.
I made a silly 1-hour long movie with friends +/- 20 years ago, on DV tape.
I would love to use this to actually be able to implement all the things we wanted to achieve back then
1,057 comments
[ 1.1 ms ] story [ 433 ms ] threadFrom Twitter/X:
Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date.
Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike.
More details and examples of what Movie Gen can do https://go.fb.me/kx1nqm
Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt.
Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment.
Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes.
Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video.
We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.
It being 30B gives me hope.
[0] https://ai.meta.com/blog/generative-ai-text-images-cm3leon/
Considering that Facebook/Meta releases blog posts titled "Open Source AI Is the Path Forward" but then refuses to actually release any Open Source AI, I'm guessing the answer is a hard "No".
They might release it under usage restrictions though, like they did with Llama, although probably only the smaller versions, to limit the output quality.
Just because pytorch is Open Source doesn't mean everything Meta AI releases is Open Source, not sure how that would make sense.
Datasets for Llama 3 is "A new mix of publicly available online data.", not exactly open or even very descriptive. That could be anything.
And no, the training code for Llama 3 isn't available, response from a Meta employee was: "However, at the moment-we haven't open sourced the pre-training scripts".
Here is the Llama source code, you can start training more epochs with it today if you like: https://github.com/meta-llama/llama3/blob/main/llama/model.p...
It's rumored Llama 3 used FineWeb, but you're right that they at least haven't been transparent about that: https://huggingface.co/datasets/HuggingFaceFW/fineweb
For models I prefer the term "open weight", but to assert they haven't open sourced models at all is plainly incorrect.
Correct me if I'm wrong, but that's the code for doing inference?
Meta employee told me just the other day: "However, at the moment-we haven't open sourced the pre-training scripts", can't imagine they would be wrong about it?
https://github.com/meta-llama/llama-recipes/issues/693
> For models I prefer the term "open weight"
Personally, "Open" implies I can download them without signing an agreement with LLama, and I can do whatever I want with it. But I understand the community seems to think otherwise, especially considering the messaging Meta has around Llama, and how little the community is pushing back on it.
So Meta doesn't allow downloading the Llama weights without accepting the terms from them, doesn't allow unrestricted usage of those weights, doesn't share the training scripts nor the training data for creating the model.
The only thing that could be considered "open" would be that I can download the weights after signing the terms. Personally I wouldn't make the case that that's "open" as much as "possible to download", but again, I understand others understand it differently.
Apologies - you are right and I was wrong. I would edit my comments but they're past the edit window, will leave a comment accordingly.
In fact, the more realistic the deepfakes become, the less harmful actual revenge porn and stolen sex videos can be, because of plausible deniability.
This "good deepfakes will prevent harm because of plausible deniability" is absurd copium, and utterly divorced from reality.
Speak to victims some time. You are not helping them.
That's something that can be fixed in a future release or you can fix it right now with some filters in post in your pipeline.
This has been the default excuse for the last 5+ years. I won't hold my breath.
AI in general is from 1950, or more generally from whenever the abacus was invented. This very website runs on AI, and always has. I would implore us to speak more exactly if we’re criticizing stuff; “LLMs” came around (in force) in 2023, both for coherent language use (ChatGPT 3.5) and image use (DALLE2). The predecessors were an order of magnitude less capable, and going back 5 years puts us back in the era of “chatbots”, aka dumb toys that can barely string together a Reddit comment on /r/subredditsimulator.
> AI so far has given us ability to mass produce shit content of no use to anybody
Good AI goes largely undetected, for the simple reason that it closely matches the distribution of non-AI content.
Controversial aside: This is same bias that results in non-passing trans people being representative of the whole. Passing trans folk simply blend in.
https://en.wikipedia.org/wiki/Verificationism
All criminals seem so incredibly stupid that you can’t understand why anyone would ever try since they all are caught? The smart ones don’t get caught and no one ever hears about them.
Today we have these realistic videos that are still in the uncanny valley. That's insane progress in the span of a year. Who knows what it will be like in another year.
Let'em cook.
2 years ago we had decent video generation for clips
7 months ago we have Sora https://news.ycombinator.com/item?id=39393252 (still silence since then)
With these things, like DALL-E 1 and GPT-3, the original release of the game changer often comes ca. 2 years before people can actually use it. I think that's what we're looking at.
I.e. it's not as fast as you think.
And isn’t this model open source…? So we get access to it, like, momentarily? Or did I miss something?
When the Will Smith one was released, it was kind of a parody though. Tech had already been able to produce that level of "quality" for about 2 years at the time of it's publishing. The Will Smith one is honestly something you could have created with Disco Diffusion in early 2021, I used to do this back then...
2022 saw: https://makeavideo.studio/ (coherent, but low res - it was possible to upscale at extreme expense) https://sites.research.google/phenaki/ https://lumiere-video.github.io/
It was more like 18-20 months ago sorry so early 2023, but https://runwayml.com/research/gen-1 was getting there as was https://pika.art/home - Sora obviously changed the game, but I would say these two were great.
The only interesting questions now have nothing to do with capability but with economics and raw resources.
In a few years, or less, clearly we'll be able to take our favorite books and watch unabridged, word-for-word copies. The quality, acting, and cinematography will rival the biggest budget Hollywood films. The "special effects" won't look remotely CG like all of the newest Disney/Marvel movies -- unless you want them to. If publishers put up some sort of legal firewall to prevent it, their authors, characters, and stories will all be forgotten.
And if we can spend $100 of compute and get something I described above, why wouldn't Disney et al throw $500m at something to get even more out of it, and charge everyone $50? Or maybe we'll all just be zoo animals soon (Or the zoo animals will have neuralink implants and human level intelligence, then what?)
That would be a boring movie.
I don't think so at all. You're thinking a movie is just the end result that we watch in theaters. Good directing is not a text prompt, good editing is not a text prompt, good acting is not a text prompt. What you'll see in a few years is more ads. Lots of ads. People who make movies aren't salivating at this stuff but advertising agencies are because it's just bullshit content meant to distract and be replaced by more distractions.
But at the same time, while it is indeed true that the end result is far more than simply just making good images, LLMs are weird interns at everything — with all the negative that implies as well as the positive, so they're not likely to produce genuinely award winning content all by themselves even though they can do better by asking them for something "award winning" — so it's certainly conceivable that we'll see AI indeed do all these things competently at some point.
I'm also expecting, before 2030, that video game pipelines will be replaced entirely. No more polygons and textures, not as we understand the concepts now, just directly rendering any style you want, perfectly, on top of whatever the gameplay logic provided.
I might even get that photorealistic re-imagining of Marathon 2 that I've been wanting since 1997 or so.
This isn't an intentional "feature" of these models; rather, it's kind of an inherent part of how such models work — they learn associations between tokens and structural details of images. Artists' names are tokens like any other, and artists' styles are structural details like any other.
So, unless the architecture and training of this model are very unusual, it's gonna at least be able to give you something that looks like e.g. a "pencil illustration."
There were movies with horrible VFX that still sold perfectly well at the time.
But then, a lot of people have financial reasons to ignore the problem. What's too bad, because it's hindering the creation of stuff that are actually useful.
These models do seem like they could be great photorealism/stylization shaders. And they are also pretty good at stuff like realistic explosions, fluid renders etc. That stuff is really hard with CG.
You are spot on. I've been involved in creating technologies used by film and video creators for decades, so I have some understanding of what would be useful to them. The best video AIs I've seen only seem capable of replacing some stock video clip creation because, so far, I haven't seen any ability to maintain robust consistency from shot to shot and scene to scene. There's also no granular control other than the text prompt. At first glance, these demos are very impressive but when you try to map the capability shown to a real production workflow for a movie, TV show or commercial, they're not even close because they aren't even trying to solve the problem.
To be clear, I think it's probably possible to create a video AI that would be truly useful in a real production workflow, it's just that I haven't seen anything working in that direction yet.
Yeah it's really hard to get across to a lot of folks that are really amped up about these tools that what they're focused on refining is not getting them any closer to their imagined goal in most professional workflows. This will be great right off the bat for what most developers would need images for-- making a hero image for a blog post, making a blurb of video for a background, or a joke, or making assets for their video game that would never cut it for a non-cheapo commercial project but are better than what they'd have been able to cobble together themselves. But those workflows are fundamentally so different from the very first steps in the process. It's a larger-scale version of trying to explain to no-compromise FOSS zealots 20 years ago that Gimp was nowhere near able to replace Photoshop in a professional toolkit because they're completely disinterested in taking feedback about professional use cases, and that being able to write your own filters in Perl doesn't really help graphic designers-- well 20 years later, the gap is as wide as it ever has been, and there are even more people, almost exclusively FOSS nerds with to professional visual work experience, that insist it's better.
That said, it's nearly as hard to get this across to ADs who are like "what do you mean this shot is going to take you 3 days? I just made these stills which are like 70% there in midjourney in 10 minutes."
> To be clear, I think it's probably possible to create a video AI that would be truly useful in a real production workflow, it's just that I haven't seen anything working in that direction yet.
I think that neural networks, generally, are already fantastically useful in tools like Nuke's Copycat node. Nobody misses masking frame-by-frame if they don't have to do it. But prompt-based tools? Nah. If even 200 words in a prompt was enough information to convey work that needed to be done, why do creative workflows need so many revisions and why are there so many meetings with sketches and mood boards and concept art among career professionals? Text prompts are great for people that are working with a medium they don't really know how to create in because the real artistic decisions are already made by the artists whose artwork was ingested into the models. If you don't understand that level of nuance, you don't see how unbelievably consequential it is to the final product, and not having granular control of it seems nearly inconsequential. Most professionals look at it and see a toy because they know it will never be capable of making what they want it to make.
Yes, I agree. You've highlighted the distinction I should have included of "prompt-based".
There's a vast gulf between these AI-researcher-based concept demos on one side and the NN-based features slowly getting implemented in real production tools. Like you, I've found it challenging to have constructive conversations about AI tooling with anyone not versed in real production workflows. To anyone with real industry experience it's obvious that so far these demos don't represent a threat to real production workflows or the skilled career professionals making a good living. It's not that they're not threatening, they're just threatening to replace a different type of job entirely. If you're one of the poor souls in an off-shore locale doing remote low-end piece-work like manning a stock photo/video clip farm or doing >$100 per piece gigs on Fivver - then, yeah, you should feel "threatened".
A meta-point I try to make in these conversations is that, at least so far, every actual paying creative job I've seen AI threaten are, IMHO, work I wouldn't wish on my worst enemy. These are low-paid entry-level sweatshop gigs and everyone doing them aspires to do something else as soon as they can. The two analogies I use are: 1) How the "threat" of robotics to jobs is actually playing out. So far, in industrial applications robots are replacing Amazon warehouse and manufacturing assembly line workers, literally today's equivalent of 1920s sweatshop work. Much like the heart-wrenching videos of children in Calcutta earning pennies sifting through junk piles for metal scraps, it'll be a better world when robots replace those jobs and humans have jobs designing, installing, programming and servicing the robots. Likewise in consumer robotics applications, so far, the robots in our house only vacuum the floors, change the cat litter box, and wash the dishes/clothes. Growing up my family spent a couple years living in Asia in the 1970s and we actually had a "wash ama" who came twice a week and washed our clothes manually with a washboard and a tub. Sounds quaint but in reality it was grueling labor. She was a lovely lady but I'm glad Maytag replaced that job.
The second analogy I often use is observing that self-driving cars are mainly a threat to Uber and Lyft drivers who often barely earn minimum wage and have no job security to start with. Career professionals actually working in real video and film production workflows feel as "threatened" by prompt-based AIs as Formula 1 drivers feel about self-driving cars. Why does current F1 champion Max Verstappen never get asked how he feels about AI self-driving cars coming for his job? :-) As you observed, anyone who understands the thousands of creative choices which comprise any shot in a quality film doesn't even see these prompt-based AI demos as relevant. Once you've heard a skilled cinematographer, colorist or director of photography spend over an hour deconstructing and debating the creative choices made in single shot or scene from a film, it's hard to even imagine these demos as a threat to that level of creative skill. But being able to crudely copy the traits of a composite of a thousand exemplars of the craft without understanding any of the interactions between those thousands of creative choices does make for impressive demos. Even though the fidelity of the crude copy is amazing, the fact is such shots are a random puree of a thousand different creative choices pulled from a thousand different great shots. That's the root of what unskilled people call the "AI-clip sheen". It won't be easy to eliminate from prompt-based clip generators because the nature of the NN is it doesn't understand the interactions of all those subtle creative choices it's aping. Mashing together one cinematographer's lens choic...
My biggest hope for the professional use of these things is in post-render pre-comp polishing for simulations and pyro. They're so good at understanding patterns and having smooth transitions that they can make a nonsense, physically absurd combination of images blend together perfectly... one of my favorites was a background guy's nose in a sepia toned video was neatly melded into a distant oncoming train. I think that could be really great for smoothing out volume textures and things like that. Given, that probably has more to do with my specialty than anything.
My main problem is that I'm just starting out my career in this field after switching from a decade of python dev work, and then doing some visual design before going to art school where I graduated at the top of my program having mostly concentrated in making cool shit at the Houdini/UE confluence. Two years ago everyone was saying "holy crap you've got the golden skillset," and now everyone's like "oof... hang in there... I guess..." Even aside from the strike aftermath, nobody in the market has any idea what to do right now, especially with juniors, let alone a really weird mixture of junior + senior dev that I am with a few contracts under my belt and a ton of really solid coding experience, but nothing really impressive in the industry itself. Who fucking knows. I think a lot of people in charge of hiring are waiting for a moment where it's going to just be sort of obvious what they need to do, and don't want to hire people into FTEs that are going to be eliminated through ai efficiency gains in 6 months. I don't have a lot of insight into the hiring side of the business though.
Somebody took AI video editing, and in a year or two, we're already at entire MMA rabbit holes of fake videos.
Commenting mostly as a personal evidence reference of how crazy the World Wide Web has gotten from anecdotal sources.
I won't argue wether text to speech qualifies as an AI but I agree they must be making bank.
Might also be an AI voice-changer (i.e. speech2speech) model.
These models are most well-known for being used to create "if [famous singer] performed [famous song not by them]" covers — you sing the song yourself, then run your recording through the model to convert the recording into an equivalent performance in the singer's voice; and then you composite that onto a vocal-less version of the track.
But you can just as well use such a model to have overseas workers read a script, and then convert that recording into an "equivalent performance" in a fluent English speaker's voice.
Such models just slip up when they hit input phonemes they can't quite understand the meaning of.
(If you were setting this up for your own personal use, you could fine-tune the speech2speech model like a translation model, so it understands how your specific accent should map to the target. [I.e., take a bunch of known sample outputs, and create paired inputs by recording your own performances of them.] This wouldn't be tenable for a big low-cost operation, of course, as the recordings would come from temp workers all over the world with high churn.)
But the people who position themselves to profit from the energy consumption of the hardware will profit from all of it: the LLMs, the image generators, the video generators, etc. See discussion yesterday: https://news.ycombinator.com/item?id=41733311
Imagine the number of worthless images being generated as people try to find one they like. Slop content creators iterate on a prompt, or maybe create hundreds of video clips hoping to find one that gets views. This is a compute-intensive process that consumes an enormous amount of energy.
The market for chips will fragment, margins will shrink. It's just matrix multiplication and the user interface is PyTorch or similar. Nvidia will keep some of its business, Google's TPUs will capture some, other players like Tenstorrent (https://tenstorrent.com/hardware/grayskull) and Groq and Cerebras will capture some, etc.
But at the root of it all is the electricity demand. That's where the money will be made. Data centers need baseload power, preferably clean baseload power.
Unless hydro is available, the only clean baseload power source is nuclear fission. As we emerge from the Fukushima bear market where many uranium mining companies went out of business, the bottleneck is the fuel: uranium.
Trial and error content-creation using generative AI, whether or not it creates any real-world value, consumes a lot of electricity.
This electricity demand is likely to translate into demand for nuclear power.
When this demand for nuclear power meets the undersupply of uranium post-Fukushima, higher uranium prices will result.
More supply and lower prices will result.
Not unlike the recent few years in (say) lithium, anticipated demand surged exploration and development, actual demand didn't meet anticipated demand and a number of developed economicly feasible resources were shuttered .. still waiting in the wings for a future pickup in demand.
Look at how long NexGen's Rook 1 Arrow is taking to develop (https://s28.q4cdn.com/891672792/files/doc_downloads/2022/03/...). Spend an hour listening to what Cameco said in its most recent conference call. Look at Kazatomprom's persistent inability to deliver the promised pounds of uranium, their sulfuric acid shortages and construction delays.
Uranium mining is slow and difficult. Existing demand and existing supply are fully visible. There's a gap of 20-40 million pounds per year, with nothing to fill the gap. New mines take a decade or more to develop.
It is not in the slightest like lithium.
Would two decades in global exploration geophysics and being behind the original incarnation of https://www.spglobal.com/market-intelligence/en/industries/m... count?
> Once you know the facts we can talk again.
Gosh - that does come across badly.
When someone compares uranium to lithium, I know I'm not talking to a uranium expert.
All the best to you, and I'll try to be more polite in the future.
Clearly you're a better expert.
> when someone compares uranium to lithium, I know I'm not talking to a uranium expert.
It's about boom bust and shuttering cycles that apply in all resource exploration and production domains.
Perhaps you're a little too literal for analogies? Maybe I'm thinking in longer time cycles than yourself and don't a few years of lag as anything other than a few years.
You are well-prepared to familiarize yourself with the current supply/demand situation. It's time to "make bank", just like you did in 2007... only more so. The 2007 spike was during an oversupplied uranium market and mainly driven by financial actors.
I invite you to begin by listening to any recent interview with Mike Alkin.
Good night and enjoy your weekend.
That said, this channel has been producing videos well before ChatGPT3.5/4 so at the very least they probably started with human written scripts.
I feel like it might actually be quite complex for AI to pull up the perfect clips and edit them with the script, including timing and everything. Maybe it could be made automatic, but nonetheless it would be a complex process and I don't think possible few years ago. I know Gemini and possibly some others can analyze video if fed to them, but I'm still skeptical that this channel in particular would have done it, when they have always had this frequency of uploads and similar tone.
Also I think there's far better TTS now with ElevenLabs and others so it could be made much more human like.
The image had all the telltale signs of being AI generated (too much detail, the lights & shadows were the wrong scale, the focus of the lens was odd for the kind of photo, etc). I checked that other group, and sure enough, they claim to be about sharing "miniature dioramas" but all they share is AI-generated crap.
And in the original group, which I'm a member of and is full of people who actually create dioramas -- let's say they are "subject matter experts" -- nobody suspected anything! To them, who are unfamiliar with AI art, the photo was of a real hand-made diorama.
At this point, looking at a big tech SoMe feed I would expect that everything is, or at least could be, gen AI content.
1. https://www.youtube.com/watch?v=vJG698U2Mvo
I could see it being pretty shocking if I hadn't, but I honestly can't imagine how I'd miss that.
> I could see it being pretty shocking if I hadn't, but I honestly can't imagine how I'd miss that.
The point of the video wasn't to count correctly, but to see the gorilla
That is also how magicians work, drawing your attention to one particular thing, hiding the secret of the trick from you, sometimes even in plain sight, like in the video.
Or pickpockets, who might bump into you and picking your pocket at the same time, where your attention is focussed on the sudden impact, keeping your attention away from your walled being taken.
Essentially, send me a video of something I care about and I will only look for that thing. Most people are not detectives, and even most would-be detectives aren’t yet experts.
Who even needs AI generated videos when you can just act out absurdity and pretend it's real?
There's a morbid path from the grainy Iraq war and earlier shaky footage, through IS propaganda which at the time had basically the most intense combat footage ever released to the Ukraine war. Which took it to the morbid end conclusion of endless drone video deaths and edited clips 30+ mins long with day long engagements and defending.
And yes, to answer your belief that there is none - there is loads of "cinematic body cam footage out there now".
I still don't know if it's autogenerated from the original video or recreated manually but yeah it's pretty realistic for the first few seconds.
https://www.theverge.com/2017/12/4/16732912/ufc-video-stream...
Fake images play into that, but they don't need to be AI generated for that to be true, it's been true forever.
The only regret I have about that is losing video as a form of evidence. CCTV footage and the like are a valuable tool for solving crimes. That’s going to be out the window soon.
Does this create difficulties if you want to modify the raw video data in any way? Yes it does, even if you just want to save it in a different lossy compression level or format. But these problems aren't insurmountable. Essentially, provenance info can be added for each modification, signed by the entity that made the change, and the end viewer can then decide if they trust the full certificate chain (just as they do now with HTTPS).
Recently someone said here that it's noticable that videos from CCTV cameras are often filmed with a phone or camera on a screen instead of using the original video, and people were speculating that it might be hard or impossible to get access to the original recording because of bureaucracy or something, but that recording a playback on a screen with a phone or camera or something is then often allowed. Maybe they also do this partly so that the original can't be easily messed with by other people.
But yeah if you can verify that a certain video was filmed at a certain time by a certain camera, that is great. Of course the companies providing these cameras should be trustworthy, and that the camera's are actually really sending what they actually record, and that the company itself doesn't mess with the original recordings.
I recall an article posted 1-2 years ago about a camera company (Kodak? Can't remember) which was starting to offer something along these lines.
>the companies providing these cameras should be trustworthy, and that the camera's are actually really sending what they actually record, and that the company itself doesn't mess with the original recordings.
I agree. We can't guarantee any of these things, but on the bright side, the incentives are pointing in the right direction to make self-interested companies choose to behave the right way.
It will complicate things and make the hardware more expensive, so I doubt it will sweep through all consumer camera tech unless the "Is this photo real?" question becomes a crisis. There's also the fact that it would be possible to give individual cameras different private keys, with certificates signed by the manufacturer: This would enable non-repudiation (you would not be able to plausibly deny that you had taken a particular photo/video), which has potentially big upsides but also privacy downsides. I think that could be solved by giving the user the option of signing with their unique camera private key (when the user wants to prove to others that they took the photo themselves) or with the manufacturer's key (when they want to remain anonymous).
Similarly, while Photoshop made it easier it happened at a time where technical advances made the problem harder because everyone’s standards for photos went up dramatically, and so producing a realistic fake was still a slow process for a skilled worker.
Now, it’s increasingly available to everyone and that means that we’re going to see a lot more scams and hoaxes as people without artistic talent or willingness to invest time can make realistic fakes even for minor things. That availability is transformative enough to merit the concern we’ve been seeing here.
The ultimate compression is to reduce the video clip to a latent space vector representation to be rendered on device. :)
Just give us a few more revs of Moore’s law for that to be reasonable.
edit: found a patent… https://patents.google.com/patent/US11388416B2/en
I have yet to find examples of this
* https://www.reddit.com/gallery/1fvs0e1
* https://old.reddit.com/r/StableDiffusion/comments/1fak0jl/fi...
- The monkey in hotspring video, if not for its weird beard...
- The koala video I would have mistaken for hollywood-quality studio CGI (although I would know it's not real because koalas don't surf... do they?)
- The pumpkin video if played at 1/4 resolution and 2x speed
- The dog-at-Versailles video edit
If the videos are that good, I'm sure I already can't distinguish between real photos and the best AI images. For example, ThisPersonDoesNotExist isn't even very recent, but I wouldn't be able to tell whether most of its output is real or not, although it's limited to a certain style of close-up portrait photography.
https://this-person-does-not-exist.com/en
Not to take away from your point but it's more limited than one might think from this phrase. As an exercise, open that page and scroll so the full image is on your screen, then hover your mouse cursor within the iris of one of the eyes, refresh and scroll again. (Edit: I just noticed there's a delayed refresh button on the page, so one can click that and then move their mouse over the eye to skip a full page refresh.) I've yet to see a case where my mouse cursor is not still in line with the iris of the next not-person.
There's an entire pattern of reels that are basically just ripped-off-content with enough added noise to (I presume) avoid content detection filters. Then the comments have links to scam sites (but are labelled as "the IMDB page for this content").
But overall the physics are surprisingly good. In the videos from text we a person moving covered in a bedsheet, a mirror doing vaguely mirror-like things, a monkey moving in water and creating plausible waves, shadows moving over a 3d object with the sloth in the pool and plausible fire. Those are all classic topics to tackle in computer-generated graphics, all casually handled by a model that isn't explicitly trained on physical simulation.
In a twist of irony it's the simplest of those (the mirror) that's the most obviously wrong.
Always important to bear in mind that the examples they show are likely the best examples they were able to produce.
Many times over the past few years a new AI release has "wowed" me, but none of them resulted in any sudden overnight changes to the world as we know it.
VFX artists: You can sleep well tonight, just keep an eye on things!
Everyone should sleep well tonight, but only because we’ll look out for each other and fight for just distribution of resources, not because the current job market is stable. IMO :)
they are not allowed to call the recession a recession until January 20
Here's an example thread: https://www.reddit.com/r/vfx/comments/1e4zdj7/in_the_climate...
I am not trying to be negative, however it is the reality that ML/LLM has eliminated entire industries. Medical transcription for example is essentially gone.
What is the required amount of labor humans should have to do?
that’s quite a productive thing. art has tremendous value to society.
why don’t we automate the washing machine more instead of automating the artist?
Automating more in the real world is much (much) harder than grabbing the low-hanging fruits in the digital world.
As someone who’s worked in the industry previously and am quite involved still, very few studios are using it because of the lack of direction it can take and the copyright quagmire. There are lots of uses of ML in VFX but those aren’t necessarily GenAI.
GenAI hasn’t had an effect on the industry yet. It’s unlikely it will for a while longer. Bad business moves from clients are the bigger drain, including not negotiating with unions and a marked decline in streaming to cover lost profits.
Pumpkin patch - Not sitting on the grass, not wearing a scarf, no rows of pumpkins the way most people would imagine.
Sloth - that's not really a tropical drink, and we can't see enough of the background to call it a "tropical world".
Fire spinner - not wearing a green cloth around his waist
Ghost - Not facing the mirror, obviously not reflected the way the prompter intended. No old beams, no cloth-covered furniture, not what I would call "cool and natural light". This is probably the most impressively realistic-looking example, but it almost certainly doesn't come close to matching what the prompter was imagining.
Monkey - boat doesn't have a rudder, no trees or lush greenery
Science lab - no rainbow wallpaper
This seems like nitpicking, and again I can't underestimate how unbelievable the technology is, but the process of making any kind of video or movie involves translating a very specific vision from your brain to reality. I can't think of many applications where "anything that looks good and vaguely matches the assignment" is the goal. I guess stock footage videographers should be concerned.
This all matches my experience using any kind of AI tool. Once I get past my astonishment at the quality of the results, I find it's almost always impossible to get the output I'm looking for. The details matter, and in most cases they are the only thing that matters.
But better training can't fix the more general problem that I'm describing. Perfect-looking videos aren't useful if you can't get it to follow your instructions.
But this limits promotion where actors do interviews and sell the movie to the public. It also limits an actor doing something crazy that tanks a movie like a tweet.
But for plenty of other masters - think Cassavetes, Mike Leigh, even PTA - the actor's outstanding talent and instincts bring something to the script and vision that is outside of their prescriptive powers. Their focus is essentially setting up a framework for magic to happen inside of.
Also, some great lines in movies came from actors ad libs. I hope there will always be some space for mild hallucination; without improvisation we wouldn't even have jazz.
https://www.snopes.com/fact-check/harrison-ford-i-know/
As a choreographer myself, that's not necessarily a problem but rather a feature: it depends on how the director creates. Often you want what's unique to the performer, you don't want them to do something that's exactly like what you envisioned but whatever their interpretation/vision of it is, the "imperfectness" is what makes it interesting and rich.
A "loong time" will be sooner that most of us think.
The way this is done currently is similar to motion capture except that the tools are gradually becoming democratized: A single actor can act all the roles you need (You could even act the scenes and roles yourself). That footage is then fed into a model that generates an actor with the appearance and voice that you desire.
As a random on the internet, my prediction is that within a year, you'll be able to produce lead actors that are believable using movie generation plus smartphone footage of yourself acting the scene.
Initially it will be expensive to make a feature length film. But from 2025 onward, the cost will come down as the tools improve. These will be a different type of movie for sure, but every advancement in film technology has always led to films that seem strange compared to what came before.
Agreed. I expect the tools I described to be prohibitively expensive for the average person for some time. By the end of 2025, probably only well-funded studios will be able to use such tools and probably not economically.
But I'd be quite surprised if Hollywood studios/publishers aren't using their immense back catalogue to train private models right now. I don't think they'll ever allow royalty-free movie generation tools to be used that were trained on their catalogues. So perhaps there'll be a cottage industry of stock footage by amateur and professional actors for training/augmenting the movie generation models and tools that will be available for general use, royalty free.
Or perhaps these tools will simply emerge as just another TikTok filter and we'll see goofy couples who filmed a skit in their living room presented as gladiators arguing on the surface of Mars while their dog runs back and forth between them unable to choose a favourite.
How has LLMs actually developed in the last two years?
Have you noticed movies and TV series use multiple cameras to capture a scene from different angles?
When it comes to video these things can't get consistency between angles or scenes.
Add to that that the results are full of glitches and the resolution is equivalent to a CRT screen in the year 2000. Same resolution as s
Fixing these limitations would equate to a revolution rather than a steady evolution.
And let's not forget that these systems are also hugely unprofitable at this stage.
And how many tokens you can afford.
In the professional creative tools business, "Now the only limit is your creativity" has been a popular marketing tagline for decades, especially for products based on new enabling technologies. It's common enough that a wry corollary has developed in response, which goes: "Unfortunately, for a lot of people that's a pretty big limit."
How many movie lines have been adlibbed but are absolute classics? Sonofabitch, he stole my line!
Like the second Avengers movie had this problem in spades.
Also "creation by committee" isn't a thing when somebody can produce a movie in their basement.
Anyway, I look forward to people using this tech to create alternative endings of existing movies.
You already can produce a movie in your basement, many of which can be found on YouTube.
But lowering those limits allows for more people to get creative
How many beautiful stories never left their author's head because their author couldn't afford it? Either the monetary cost or the opportunity cost
Considering how many movies come from one place in one country (Hollywood), we haven't even scratched the surface of human creativity
I avoid updating to each new Firefox version, because from time to time they break some features important for me.
I use Firefox on my personal device: the website worked fine though took an extra "hiccup" to load compared to Edge (version 131.0 on Windows).
Are you ready to become a penguin in all of your posts to maximise aquatic engagement? I am.
[1]: https://epiphqny.github.io/Loong-video/
The clothing changes also have pretty rough edges, or just look like they're floating over the original model. The 3D glasses one looked atrocious. The lighting changes are also pretty lacking.
Are you still impressed though?
It's only going to get better, faster, cheaper, easier.[a]
Sooner than anyone could have expected, we'll be able to ask the machines: "Turn this book into a two-hour movie with the likeness of [your favorite actor/actress] in the lead role."
Sooner than anyone could have expected, we'll be able to have immersive VR experiences that are crafted to each person.
Sooner than anyone could have expected, we won't be able to identify deepfakes anymore.
We sure live in interesting times!
---
[a] With apologies to Daft Punk: https://www.youtube.com/watch?v=gAjR4_CbPpQ
I don't think you get "The Green Mile" from something like this.
Local art, local actors, local animations telling stories about local culture. A netflix for every city, even neighborhoods. That's going to be crazy fun.
There is so much that has to be conveyed in making a film, if you want it to say something particular.
And? What's the problem with that? You seem to be locked in a "prompt to get a movie" mindset.
...yet
The limitations of reality seem to have a positive effect on the overall process of film making, for whatever reason. I expect generative AI film will be at least as bad. Gonna be hard to get an entire well-crafted film out of them.
The same AI can still raise the minimum bar for quality. Or replace YouTubers and similar while they're still learning how to be good in the first place.
No idea where we are in this whole process yet, but it's a continuum not a boolean.
Pinnacle is not the word I'd use. Race to the bottom, least possible effort, plausibly deniable quality, gross exploitation, capitalist bottom line - those are all things I'd use to describe current "art" awards like Grammy, Oscars, Cannes, etc.
The media industry is run by exploiting artists for licensing rights. The middle men and publishers add absolutely nothing to the mix. Google or Spotify or platforms arguably add value by surfacing, searching, categorizing, and so on, but not anywhere near the level of revenue capture they rationalize as their due.
When anyone and everyone can produce a film series or set of stories or song or artistic image that matches their inner artistic vision, and they're given the tools to do so without restriction or being beholden to anyone, then we're going to see high quality art and media that couldn't possibly be made in the grotesquely commercial environment we have now. These tools are as raw and rough and bad performing as they ever will be, and are only going to get better.
Shared universes of prompts and storylines and media styles and things that bring generative art and storytelling together to allow coherent social sharing and interactive media will be a thing. Kids in 10 years will be able to click and create their own cartoons and stories. Parents will be able to engage by establishing cultural parameters and maybe sneak in educational, ethical, and moral content designed around what they think is important. Artists are going to be able to produce every form of digital media and tune and tweak their vision using sophisticated tools and processes, and they're not going to be limited by budgets, politics, studio constraints, State Department limitations, wink/nod geopolitical agreements with nation states, and so on.
Art's going to get weird, and censorship will be nigh on impossible. People will create a lot of garbage, a lot of spam, low effort gifs and video memes, but more artists will be empowered than ever before, and I'm here for it.
Any accolades, be that professional groups, people's awards, rotten tomatoes or IMDB ratings.
> Race to the bottom, least possible effort, plausibly deniable quality, gross exploitation, capitalist bottom line - those are all things I'd use to describe current "art" awards like Grammy, Oscars, Cannes, etc.
I find them ridiculous in many ways, but no, one thing they're definitely not is a race to the bottom.
If you want to see what a race to the bottom looks like, The Room has a reputation for being generally terrible, "bad movie nights" are a thing, and Mystery Science Theater 3000's schtick is to poke fun at bad movies.
> The media industry is run by exploiting artists for licensing rights.
Yes
> The middle men and publishers add absolutely nothing to the mix. Google or Spotify or platforms arguably add value by surfacing, searching, categorizing, and so on, but not anywhere near the level of revenue capture they rationalize as their due.
I disagree. I think that every tech since a medium became subject to mass reproduction (different for video and audio, as early films were famously silent) has pushed things from a position close to egalitarianism towards a winner-takes-all. This includes Google: already-popular things become more popular, because Google knows you're more likely to engage with the more popular thing than the less popular thing. This dynamic also means that while anyone will be free to make their own personal vision (although most of us will have all the artistic talent of an inexperienced Tommy Wiseau), almost everyone will still only watch a handful of them.
> Art's going to get weird, and censorship will be nigh on impossible.
Bad news there, I'm afraid. AI you can run on your personal device, is quite capable of being used by the state to drive censorship at the level of your screen or your headphones.
Nearly all the movies that go to theaters aren't "meaningful art". Not only that but what's meaningful to you isn't necessarily what's meaningful to others.
If someone can get their own personal "Godzilla VS The Iron Giant" crossover made into a feature-length film it will be meaningful to them.
No but what they are is expensive, flashy, impressive productions which is the only reason people are comfortable paying upwards of $25 each to see them. And there's no way in the world that an AI movie is going to come anywhere close to the production quality of Godzilla vs Kong.
And like, yeah, their example videos at the posted link are impressive. How many attempts did those take? Are they going to be able to maintain continuity of a character's appearance from one shot to the next to form a coherent visual structure? How long can these shots be before the AI starts tripping over itself and forgetting how arms work?
We will be the old ones going "back in my day, you had to actually shoot movies on a camera! And background objects had perfect continuity!" And they will roll their eyes at us and retort that nobody pays attention to background objects anyway.
But I have faith that people will notice the difference. The current generation may not care about autotune, but that doesn't mean another generation won't. People rediscover differences and decide what matters to them.
When superhero movies were new, almost everyone loved them. I was entranced. After being saturated with them... the audience dropped off. We started being dissatisfied with witty one-liners and meaningless action. Can you still sell a super-hero movie? Sure. Like all action movies, they internationalize well. But the domestic audiences are declining. It makes me think of Westerns. At one time, they were a hollywood staple. Now, not so much. Yes, they still make them, and a good one will do fine, but a mediocre one... maybe not.
The previous generation's care about autotune was also flatly wrong. Autotune was used by a few prominent artists then and is more widely used now as an aesthetic choice, for the sound it creates which is distinctly not natural singing, as the effect was performed by running the autotune plugin at a much, much higher setting than was expected in regular use.
Tone correction occurs in basically every song production now, and you never hear it. Hell, newer tech can perform tone correction on the fly for live performances, and the actual singing being done on the stage can be swapped out on the fly with pre-recorded singing to let the performer rest, or even just lipsync the entire thing but still allow the performer to jump in when they want to and ad-lib or tweak delivery of certain parts of songs.
The autotune controversey was just wrong from end to end. When audio engineers don't want you to hear them correcting vocals, you don't hear it. I'd be willing to buy another engineer being able to hear tone correction in music, but if a layman says they do, sorry but I assume that person's full of shit.
And maybe they won't have a problem with it, like you say, maybe that'll just be their "normal" but that seems so fucking sad to me.
https://standardebooks.org/
For newer in-copyright works, public libraries commonly offer Libby:
https://company.overdrive.com/2023/01/25/public-libraries-le...
It gives anyone with a participating-system library card free electronic access to books and magazines. And it's unlikely that librarians themselves will be adding AI book-slop to the title selection.
To be clear, I'm not talking great literature. I'm talking Clifford the Big Red Dog type stuff.
That said I still have a number of problems with this assertion:
It will absolutely be down, in part, to economic necessity. Amazon's platform is already dealing with a glut of shitty AI books and the key way they get ahead in rankings is being cheaper than human-created alternatives, and they can be cheaper because having an AI slop something out is way less expensive and time consuming than someone writing/illustrating a kid's book.
Moreover, our economy runs on the notion that the easier something is to do, the more likely people are to do it at scale, and vetting your kids media is hard and annoying as a parent at the best of times: if you come home from working your second job and are ready to collapse, are you going to prepare a nutritious meal for your child and set them up with insightful, interesting media? No you're going to heat up chicken nuggets and put them in front of the iPad. That's not good, but like, what do you expect poor parents to do here? Invent more time in the day so they can better raise their child while they're in the societal fuckbarrel?
And yes, before it goes into that direction, yes this is all down to the choices of these parents, both to have children they don't really have the resources to raise (though recent changes to US law complicates that choice but that's a whole other can of worms) and them not taking the time to do it and all the rest, yes, all of these parents could and arguably should be making better choices. But ALSO, I do not see how it is a positive for our society to let people be fucked over like this constantly. What do we GAIN from this? As far as I can tell, the only people who gain anything from the exhausted-lower-classes-industrial-complex are the same rich assholes who gain from everything else being terrible, and I dunno, maybe they could just take one for the team? Maybe we build a society focused on helping people instead of giving the rich yet another leg up they don't need?
This is what I mean by "complicated factors of culture and habit." An iPad costs more than an assortment of paper books. Frozen chicken nuggets cost more than basic ingredients. But the iPad and nuggets are faster and more convenient. The kids-get-iPad-and-nuggets habit is popular with middle-income American families too, not just poor families where parents work two jobs. The economic explanation is too reductive.
I'm not trying to say that this is the "fault" of parents or of anyone in particular. When the iPad came out I doubt that Apple engineers or executives thought "now parents can spend less time engaging with children" or that parents thought of it as "a way to keep the kids quiet while I browse Pinterest" but here we are.
Is it too much to ask for a hint of caution with regard to our most vulnerable populations brains?
Circa 2020 a huge number of fed-up good-intentioned engineers and designers quit. It had no effect, at all.
To be clear: I am not saying that engineers need to be better at preventing this stuff. I am saying regulators need to demand that companies be careful, and study how this stuff is going to affect people, not just yeet it into the culture and see what happens.
There is a vast difference between a formulaic hollywood movie and some guy with a camera. If I say "Godzilla vs. The Iron Giant" what is the plot? Who is the good guy? Why does the conflict take place?
AI will come up with something. Will it be compelling even to the audience of one?
As a toy, maybe. As an artistic experience... not convinced.
You still aren't getting it. Movie directors aren't making these decisions either.
What they are doing, is listening to market focus groups and checking off boxes based on the data from that.
A market focus group driven decisions for a movie is just as much, if not more so, of an "algorithm" than when a literal computer makes the decision.
Thats not art. Its the same as if a human manually did an algorithm by hand and used that to make a movie.
If it were really all just market decisions, directors would have no influence. This is not remotely the case. Nor are they paid as though that were true.
Yes I agree that it is not remotely the case that AI videos involve zero creative input.
Thats my point. If anything, there could be a lot more creative input into AI videos, than a bog standard hollywood film.
> Some of it is done that way, but by no means all of it
Oh? So just like how that is the case some of the time for AI videos, but by no means for all of it? Yes, thats my point.
I assure you, they don't do surveys on the punchiness and strategy used by foley artists; the slope and toe of the film stock chosen for cut scenes by the DP or that those cut scenes should be shot like cut scenes instead of dream sequences; the kind of cars they use; how energetic the explosions are; clothing selection and how the costumes change situationally or throughout the film; indescribably nuanced changes in the actor's delivery; what fonts go on the signs; which props they use in all of the sets and the strategies they use to weather things; what specific locations they shoot at within an area and which direction they point the camera, how the grading might change the mood and imply thematic connections, subtle symbolism used, the specifics of camera movements, focus, and depth of field, and then there's the deeeep world of lighting... All of those things and a million others are contributions from individual artists contributing their own art in one big collaborative project.
Well no. Instead, I am implying that they are as much of a "technician" so someone who is putting in a huge amount of effort into making AI videos.
If you want to say that it is perfectly possible for someone to put a high amount of vision and make a large amount of creative decisions into AI videos, then I agree.
> All of those things and a million others are contributions
Yes I agree that there can be a million other contributions to making AI videos. Glad you agree too.
Its just that other people need to admit that yes it is possible for AI to include a lot of creative decisions.
You, being someone who claimed to care about that should be willing to admit that it is possible to do this.
I agree completely that creative input, such as the creative input that people can put into AI if they choose to do so, is important.
And just like how in other forms of media, yes sometimes people don't put in that creative input. But its still possible if you put in the effort.
And since you didn't address this at all, I can only assume that you agree with me completely.
I use prompt-based generative AI in creative ways as part of my professional processes all the time. And you know what? They are nowhere close to being useful for generating anything that plays a significant on-screen role in high-end media creation. Anybody who says they are does not know how different the requirements for high-end use cases are.
You're using me as a proxy to toe the line of a ridiculous ideological polemic that I have nothing to do with. If you want to argue with someone that has the uniform, standard-issue set of anti-AI opinions you expect to encounter so you don't have to consider your arguments too much, there's like thousands of them right over on reddit. Easy.
"Its just that other people need to admit that yes it is possible for AI to include a lot of creative decisions."
You response is actually, "yes, I agree with you. You are correct"
It sounds like you simply agree with me.
> You still aren't getting it. Movie directors aren't making these decisions either.
> What they are doing, is listening to market focus groups and checking off boxes based on the data from that.
> A market focus group driven decisions for a movie is just as much, if not more so, of an "algorithm" than when a literal computer makes the decision.
> Thats not art. Its the same as if a human manually did an algorithm by hand and used that to make a movie.
I responded as a professional in the field pointing out how ridiculous that take is, and then everything else that you said is putting words in my mouth based on opinions you assumed I had, but do not. I'm not arguing a "side" here-- I'm pointing out that something you said about practices in my field is entirely baseless. I have too much of a professional stake in this to pick a "side" because I actually have to deliver great work to spec, on time, and can't be bothered to field a whole bunch of people either with dunning-kruger confidence in their understanding technology or dunning-kruger confidence in understanding art having the nerve to be condescending while making entirely baseless, glib comments about what I do for a living, and acting like the righteousness of their cause makes it ok to be full of shit. If you want to be able to argue a "side" where you're just vaguely responsible for your grand idea and it doesn't really matter if you're full of it because nobody else there knows what they're talking about, either, then reddit is a tiny little ascii string down the street.
Hey well you said this. This is agreeing with what my point is, and I am glad we were able to clear up any possible misunderstandings, or I convinced you or similar.
As far as I am concerned you don't have any disagreements with my central point, which is good enough for me.
I am glad we cleared up the main miscommunication.
You're imagining "pls write film" but the case of being able to film something and then adjust and tweak it, easily change backdrops etc could lead to much higher polish on creations from smaller producers.
Would the green mile be any less hard hitting if the lights flickering were caused by an AI alteration to a scene? If the mouse was created purely by a machine?
The printing press led to publishing works being reachable by more people so we got tons of garbage but we also got those few individual geniuses that previously wouldn't have been able to get their works out.
I see similarities in indie video/PC games recently too. Once the tech got to the point that an individual or small group could create a game, we got tons of absolute garbage but also games like Cave Story and Stardew Valley (both single creators IIRC).
Anything that pushes the bar down on the money and effort needed to make something will result in way more of it being made. It also hopefully makes it possible for those rare geniuses to give us their output without the dilution of having to go through bigger groups first.
I'm also excited from the perspective that this decouples skills in the creative process. There have to be people out there with tremendous story telling and movie making skills who don't have the resources/connections to produce what they're capable of.
To do something similar, this has to allow the director (or whomever is prompting the AI) to control all meaningful choices so that they get more or less the movie the intend. That seems far away from what is demonstrated.
So now a director with a limited budget but with a good vision and understanding of the tools available has a better chance to realize their vision. There will be tons of crap put out by this tool as well. But I think/hope that at least one person uses it well.
But because it will make shooting a movie more accessible to people with limited budgets, the movie studios, who literally gatekeep access to their sets and moviemaking equipment, are going to have a smaller moat. The distribution channels will still need to select good films to show in theaters, TV, and streaming, but the industry will probably be changing in a few years if this development keeps pace.
I'm not against tools for directors, but the thing is, directors tell actors things and get results. Directors hire cinematographers and work with them to get the shots they want. Etc. How does that happen here?
Also, as someone else mentioned, there is the general problem that heavily CG movies tend to look... fake and uncompelling. The real world is somehow just realer than CG. So that also has to be factored into this.
It will start out with more believable green screen backgrounds and b roll. Used judiciously, it will improve immersion and cost <$10 instead of thousands. The actors and normal shots will still be the focus, but the elements that make things more believable will be cheaper to add.
Have you ever noticed that explosions look good? Even in hobby films? At some point it became easy to add a surprisingly good looking explosion in post. The same thing will happen here, but for an increasing amount of stuff.
Which doesn't mean it won't keep happening (economics), but it doesn't necessarily mean any improvement in movie quality.
https://www.vulture.com/article/movies-fire-computer-generat...
https://archive.is/u8Ugr
But maybe you do get Deadpool & Wolverine 3
Guess where the money is?
But, in reality, even making that kind of film is miles away from these examples.
Well, given the studios still hold the copyright, they can severely constrain supply to keep profits up.
My suspicion is that this kind of stuff gradually reduces some of the labor involved in making films and allows studios to continue padding their margins.
It's a powerful tool. A painting isn't better because the artist made their own paint. A movie made with IRL camera may not be better than one made with an AI camera.
However, I see an interesting middle ground appear: a talented writer could utilize the AI tooling to produce a movie based upon their own works without having to involve Hollywood, both giving more writers a chance to put their works in front of an audience as well as ensuring what's produced more closely matches their material (for better or worse).
The algorithms and people making content for the algorithm were trends that have dominated for years already.
None of that is "real" art, when you are just making something optimized for an algorithm.
Similarily, a film director "just" gives guidance to a bunch of people: actors, camera operator, etc. Do you consider the movie is his creation, even if he didn't directly perform any action? A photographer just has to push a button and the camera captures an image. Is the output still considered his creation? Yes and Yes, so I think we should consider the same with AI assisted art forms. Maybe the real topic is the level of depth and sophistication in the art (just like the difference between your iPhone pictures and a professional photographer's) but in my opinion this is orthogonal to it being human or AI generated.
To be honest so far we have mostly seen AI video demos which were indeed quite uninteresting and shallow, but now filmmakers are busy learning how to harness these tools, so my prediction is that in no time you will see high quality and captivating AI generated films.
(1) https://artefact-ai-film-festival.com/golden-hours-66f869b36... Please consider liking it!
Excellent work!
If yall needed evidence of these tools giving everyday people the ability to make emotion-tugging creations, I'll send you a picture of the tears!
Now I'm thinking I can finally make the (IMO) dope music videos that come to me sometimes when I'm listening to a song I really love.
[1] The first minutes of UP
I liked it.
What happened to the mom? Why does the kid get older and younger looking? why does the city flicker in the beginning? which kid is his in the ballet performance? why do they randomly have "lazy eye"? i could keep going but i think we all get my point.
I can intuit the tropes used by the AI to convey meaning, and i'd be willing to list them all with relevant links for the paltry sum of $50. Be warned, it will be a very large list. Tropes and "memes" are doing 100% of the heavy lifting of this "art".
Sorry, human. As someone who stopped creating art on a daily basis due to market dilution (read: it's too hard to build a fanbase that i care about), i am very critical of most "art" produced anyhow.
this is dogshit.
Not that cyber-bullying and usurpation schemes escalating a whole new level being less of a concern in the aftermaths, to be clear.
Most people will not notice if the soundtrack to a new TV show is made by a 5 word AI prompt of "exciting build-up suspense scene music" while they're playing pouring money into their mobile gacha game to get the "cute girl, anime, {color} {outfit}" prompt picture that is SSS rank.
You or I might not care for AI slop, but it's a lot cheaper to produce for Netflix or Zinga or Spotify or whatever, and if they go this route, they don't have to pay for writers, actors, illustrators, songwriters, or licensing for someone else's product. They'll just put their own AI content on autoplay after what you're currently watching, and hope most people don't care enough to stop it and choose something else.
Ever notice how they never show anyone moving quickly in these clips?
Counterpoint: home "studio" recording has been feasible for decades, but music execs are not ruffled. Sure, you get a Billie Eilish debut album once a generation, but the other 99.99% of charting music is from the old guard. The media/entertainment machine is so much bigger than just creating raw material.
As they say, porn is always the leading spear of technology. It's something to keep an eye on (no pun intended) to understand how society will accept/integrate generated content.
However, commercially it seems like a niche within the existing structures of porn. Mostly competing with the market for animated stuff. At least that's where it is right now, and its already at photorealistic parity with human content creators.
We have not had the ability to create interesting ai porn vids yet. How would we? Meta just showed movie gen.
But i'm pretty sure the short images of moving woman very subtle might gotten the one or other of. Just wait a little bit until you can really create wat you are looking for.
I'm not sure exactly what models the account owners use, but I think it's a mix of Stable diffusion video touched up with adobe tooling.
What scares me most is that in my opinion, by far the best prompt writers are the ones who are deeply "motivated" and "experienced" with prompting. Often the best prompters have only one hand on the keyboard at a time.
I can trivially fine-tune and create more art from certain artists in an hour than they have produced themselves in their whole careers. This makes a lot of people very upset.
This sounds marginally above fanfiction, so I do think it'll be very easy to tell them apart. "Terminator, except with Adam Sandler and set on Mars" is a cute, gimmicky idea, not a competitor to serious work.
I'm sure this can be used to create entertaining movies that are fun and wacky. I don't think it can create impactful movies.
It's no more ridiculous than saying what makes a painting impactful is the brush strokes. But if I copy Picasso's work stroke for stroke, why am I not Picasso? After all, the dumbass paints like a child, admittedly! How could someone like him ever be considered a great painter?
However, merely describing something is not doing the thing. Otherwise, the business analysists at my company would be software engineers. No, I make the software, and they describe it.
The end-goal here is humanless automation, no? Then I'm not sure your assumption holds up. If there's no human, I question the value.
You may question the value but if it’s anything like rugs you won’t be in the majority. People pay a significant premium for artisanal handmade rugs but that being said, more than 95% of the rugs people use are machine made because they’re essentially indistinguishable from a handmade one and are much, much cheaper and just as functional.
I think you can do this with video too, just more challenging right now.
On social media platforms, typically the most popular content triggers the strongest emotions. It's rage-bait however, or sadness bait, or any other kind of emotional manipulation. It tricks the human mind and drives up engagement, but I don't think that is indicative of its value.
To be clear, I'm certain that's not what you're doing, and the music is good. But I think it's complicated enough that triggering emotions isn't enough data to ascertain value.
I don't know, exactly, what combination of measurements are needed to ascertain value. But I'm confident human-ness is part of the equation. I think if people are even aware of the fact a human didn't make something they lose interest. That makes the future of AI in entertainment dicey, and I think that's what fuels the constant dishonesty around AI we're seeing right now. Art is funny in how it works because, I think, intention does matter. And knowledge about the intention matters, too. It maybe doesn't make much sense, but that's how I see it.
Meanwhile, as someone who has been engaged with the AI art community for years, and spent years volunteering part-time as a content moderator for Midjourney, the process of creating art via AI with intentionality is deeply human.
As an MJ mod, I have seeeeeeen things.... It's like browsing though people's psyche. Even in public portfolios people bare their souls because they assume no one will bother to look. People use AI to process the world, their lives, their desires, their trauma. So much of it is straight-up self-directed art therapy. Pages and pages, thousands of images stretching over weeks, sometimes months, of digging into the depths of their selves.
Now go through that process to make something you intend to speak publicly from the depth of your own soul. You don't see much of that day to day because it is difficult. It's risky at a deeply personal level to expose yourself like that.
But, be honest: How much deeply personal art do you see day to day? You see tons of ads and memes. But, to find "real art" you have to explicitly dig for it. Shitposting AI images is as fun and easy as shitposting images from meme generators. So, no surprise you see floods of shitposts everywhere. But, when was the last time you explicitly searched out meaningful AI art?
You bring up a good point - very little. But, to be fair, those people aren't necessarily trying to convince me it's art.
I think you're mostly right but I am a little caught up on the details. I think it's mostly a thing of where the process is so different, and involves no physical strokes or manipulation, that I doubt it. And maybe that's incorrect.
However, I will also see a lot of people who don't know how to do art pretending like they've figured it all out. I also see the problem with that. It wouldn't be such a problem if people didn't take such an overly-confident stance in their abilities. I mean, it's a little offensive for that guy mucking around for an hour to act like he's DiVinci. And maybe he's a minority, I wouldn't know, I don't have that kind of visibility into the space.
I think a lot of the friction comes from that. Shitposts are shitposts, but I mean... we call them shitposts, you know? They, the people that make them, call them shitposts. There's a level of humility there I haven't necessarily seen with "AI Bros".
I think, if you really love art, AI can be a means to create a product but it can also be a starting point to explore the space. Explore styles, explore technique, explore the history. And I think that might be missing in some cases.
For a personal example, I'm really into fashion and style. I love clothes and always have. But it's really been an inspiration to me to create clothes, to sew. I've done hand sewing, many machine stitches too. And I don't need to - I could explore this in a more "high-level" context, and just curate clothing. But I think there's value in learning the smaller actions, including the obsolete ones.
https://x.com/ClaireSilver12
https://www.clairesilver.com/collections
from the POV of fashion illustration. Her "corpo|real" collection took something like 9 months to create and was published nine months ago.
To use a real-world example: if the Renaissance-era patrons had merely written down their preferences and had work made to match those preferences, it's highly unlikely that you'd have gotten the Mona Lisa or David.
Which is to say that, there will definitely be some interesting and compelling art made with AI tools. But it will be made by a specific person with an artistic vision in mind, and not merely an algorithm checking boxes.
At best, you'll get something like a generic sitcom. The idea that "all visual stimulus will be satisfied by hypertuned AI models" doesn't line up with how people experience the arts, at all.
I don't want to carry mechanical solutions labelled culture - deterministic enough, despite hallucinations - into the next generation that follows my own. It's an impressive advancement for automation, sure, but just not a value worth sharing as human development.
That being said, I think GenAI could be a valuable addition in any blueprint-/prototype-/wireframing phase. But, ironically, it positions itself in stark contrast to what I would consider my standards to contemporary brainstorming, considering the current Zeitgeist:
Out of curiosity, what is it that people do with these things? Do they put them on TikTok?
The demos was made by nerds (said with love) with a limited time window. Wait until the creatives get a hold of the tool.
I've been doing this with ChatGPT, except it's more of a "turn into a screenplay" then "create a graphic of each scene" and telling it how I want each character to look. It's works pretty well but results in more of a graphic novel than a movie. I'm definitely been waiting for the video version to be available!
Many creative works these days require the effort and input of so many people, so much time, and so much money that they can't have a specific creative vision. Mediums like book, comics, indie movies, and very low budget indie games, where the the end product was created by the smallest number of people, have the most potential to be interesting and creative. They can take risks. This doesn't mean they will be good, most aren't, but it means that the range of quality is much broader, with some having a chance to shine in ways which big budget projects just can't. The issue with small teams and small budgets is that they are inherently limited in what they can create. Better tools allows smaller groups of people to make things that previously would have required an entire studio but without diluting the creative vision.
Will this also result in a tidal wave of low effort garbage? Of course it will. But that can be ignored.
RIP Pika and ElevenLabs… tho I guess they always can offer convenience and top tier UX. Still, gotta imagine they’re panicking this morning!
Given how effective the still images of Trump saving people in floodwater and fixing electrical poles have been despite being identifiable as AI if you look closely (or think…), this is going to be nuts. 16 seconds is more than enough to convince people, I’m guessing the average video watch time is much less than that on social media.Also, YouTube shorts (and whatever Meta’s version is) is about to get even worse, yet also probably more addicting! It would be hard to explain to an alien why we got so unreasonably good at optimal content to keep people scrolling. Imagine an automated YouTube channel running 24/7 A/B experiments for some set of audiences…
The end result for them wanting humane conditions was getting murdered by the machine owners & the state.
Perhaps the rabidly pro-AI people shouldn't be on the side of the murderous psychopaths who wanted to extract maximum profit by employing children and displacing thousands of people if they don't want to be viewed as similarly psychotic.
If one watches movies, reads books, etc. just to pass the time, maybe this would be some kind of boon. But for those of us looking for meaningful commentary on life, looking to connect with other human beings, this would be some circle of hell. It's some kind of solipsism.
If you disagree with that, you're basically saying La Jetee isn't art, which would be a hard sell.
There are a LOT of choices in making a movie, and if you just let the AI make them, you are getting "random" (uncontrolled) choices. I don't think that is going to compare favorably to the real thing.
If you can specify all that, then it's just a tool. Cool. But it's still going to take pro-level skills to use it.
But would AI be able to quote Vertigo, like La Jetee does? Doesn't art, at least to some degree, require intent (including all intentional subversions of that intent dogma, of course)?
Of course AI can never truly experience being human, it has no emotions, but it is excellent at mimicry and it can certainly provide a meaningful outside perspective.
Is there anything to say about humanity that is not in the training corpus already?
Nothing AI has yet done has demonstrated anything at the level of art or mastery. I guess I'm unconvinced that throwing a million stories into the blender and synthesizing is going to produce a compelling one.
That prediction became true for like 5% of the population, everyone else is probably stupider than they were before, thanks to social media.
Similarly, I think your prediction will apply to a small subset of humanity.
I get that this is kind of a fundamental line in the sand for most of the "AI art" going around, and it seems like most people fall on one side or the other. "I consume art for entertainment" vs "I interact with art to experience the human condition".
I also don't want to say that AI Art has no value, because I think as a tool to help artists realize their vision it can be very useful! I just don't think that art entirely made by AI is interesting.
The problem: In my limited playing of these tools they don't quite make the mark and I would easily be able to tweak something if I had all the layers used. I imagine in the future products could be used to tweak this to match what I think the output should be....
At least the code generation tools are providing source code. Imagine them only giving compiled bytecode.
It so happens that there are innumerable samples of prose and source code and rendered songs and videos and images to use as this training data.
But that's not so much the case for professional workflows (outside of software development).
If the tools can evolve to generating usefully detailed and coherent media projects instead of just perceptually convincing media assets, it's going to be a while before they get there.
As it stands, the only chance you have of depicting a consistent story across a series of shots is image-to-video, presuming you can use LoRAs or similar techniques to get the seed photos consistent in themselves.