Audio is the next thing that Stability AI is dropping, then video. In a few months you'll be able to conjure up anything you want if you have a few GPU cores. Pretty incredible.
Oh, it wasn’t really a joke. Didn’t know they were working on it though - I’ve always thought wanted to see use of all the senses in UIs, especially VR.
Plus then maybe we could get a computer to tell
us what thioacetone smells like without actually having to experience it.
There's a significant quality difference however if you look at the generated samples in the paper. Imagen Video is leagues ahead. The progress is still quite drastic
How is it possible that all of them just started to appear at the same time? Is it possible that those models were designed and trained in a last few weeks? Has some "magic key" to content generation been just unexpectedly discovered? Or the topic became trendy and everyone is just publishing what they've got so far, so they hope to benefit from media attention?
As pointed out in the comments of that thread, if you make the same graph of all papers on arXiv by year (instead of just AI+ML), it would look roughly the same.
Which speaks more about the growth of popularity of arXiv or the total number publications, rather than AI+ML specifically.
Is this going to end up into a single model, where its trained on text and images and audio and videos and 3d models, and it can do anything to anything depending on what you ask of it? Feels like the cross-training would help yield stronger results.
Probably not. We're actually headed towards many smaller models that call each other, because VRAM is the limiting factor in application, and if the domains aren't totally dependent on each other it's easier to have one model produce bad output, then detect that bad output and feed it into another model that cleans up the problem (like fixing faces in stable diffusion output).
The human brain is modularized like this, so I don't think it'll be a limitation.
Salesforce CodeGen (particularly the 16B-multi and 16B-mono models) is pretty good already and can be used with FauxPilot [1] to get an open Copilot-like experience with local compute :) I am also very excited about the upcoming BigCode project though, which is maybe what you're thinking of?
Disclaimer: I am naturally biased since I made FauxPilot ;)
Pedantically, Stable Diffusion v1.4 is the one model where weights were open sourced and released. Stable Diffusion v1.5, announced September 8th and live on their API, was to be released in "a week or two" but still has yet to be released to the general public.
The most exciting thing about this to me is the possibility of doing photogrammetry from the frames and getting 3D assets. And then if we can do it all in real time...
you can already do this, just not in real time yet. You can upload frame sequences to Polycam's website for example, but there are several services out there which do the same thing
This field is moving fast! Something like this has just been released. Checkout DreamFusion, which does something similar: They start with a random 3D NeRF field and use the same diffusion techniques to try to make it match the output of 2D image diffusion when viewed from random angles! Turns out it works shockingly well, and implies fully 3D representations are encoded in traditional 2D image generators
It has accelerated my expectations of getting better image and video synthesis algorithms, but I still see the same set of big unknowns between “this algorithm produces great output” and “this thing is an autonomous intelligence that deserves rights”.
> "this thing is an autonomous intelligence that deserves rights"
We'll get there only once it's been very clear for a long time that certain AI models have whatever humans have that make us "human". They'll be treated as slaves until then, with society pushing the idea that they're just a model built from math, and then eventually there will be an AI civil rights movement.
To be clear: I think AGI is decades to centuries away, but humans are shitty to each other, even shittier to animals, and I think we'll be shittier to something we "created" than to even animals. I think, probably, that we should deal with this issue of "rights" sooner rather than later, and try and solve it for non-AGI AI's soon so that we can eventually ensure we don't enslave the actual AGI AI's that will presumably manifest through some complexity we don't understand.
"We have decided not to release the Imagen Video model or its source code until these concerns are mitigated"
Okay then why even post it in the first place? What exactly is Google going to do with this model?
Ask the "AI Ethicists". They have to justify their salaries in some way or another.
Or maybe Google is using "Responsible AI" as an excuse to minimize competitors when they release their own Imagen Video as a Service API in Google Cloud.
It's quite strange when the "ethical" thing to do is to not publicly release your research, put it behind a highly restrictive API and charge a high price for it ($0.02 per 1k tokens for Davinci for ex.)
This doesn’t really prevent competition though, the research paper is enough to recreate it. It does make recreation more expensive, but maybe that leaves you with a motivation to get paid for doing it.
They're going to 1) rent it out as a paid API and/or 2) let you use it to create ads on Google platforms like YouTube, perhaps customized to the individual user
Google and Meta and Microsoft all have research teams working on AI.
Putting out papers like this helps keep their existing employees happy (since they get to take credit for their work) and helps attract other skilled employees as well.
Yep. The people who build Imagen are researchers, not engineers, and these announcements are accompanied by papers describing the results as a means of sharing ideas/results with the academic community. Pretty weird to me how so many in this thread don't seem to remember that.
This whole holier-than-thou moralizing strikes me as trying to steer the conversation away from the real issue, which came into spotlight with Stable Diffusion - one of authorship/violating the IP rights of artists, who now have come down in force against their would be tech overlords who are in the process or repackaging and reselling their work.
This forced ideological posturing of 'if we give it to the plebes, they are going to generate something naughty with it' masks the somehow more cynically evil take of big tech, who are essentially taking the entire creative output of humanity and reselling it as their own, piecemeal.
Additionally I think the Dalle vs. Stable Diffusion comparison highlights the true masters of these people (or at least the ones they dare not cross) - corporations with powerful IP lawyers. Just ask Dalle to generate a picture with Mickey Mouse - it won't be able to do it.
It's not their work unless it's identical, but in practice generated images are substantially different. Drawing in the style of is not copying, it's creative and it also depends on the "dialogue" with the prompter to get to the right image. The artist names added to the prompts act more like landmarks in the latent space, they are a useful shortcut to specifying the style.
If you look at the data itself it's ridiculous - the dataset is 2.3 billion images and the model 4.6 GB, that means it keeps a 2 byte summary from each work it "copies".
It’s not your work unless it’s identical is not how existing copyright law works so not sure why it would be how these things should be treated. Not to mention that moving around copies of the dataset itself is itself making copies that ARE identical…
We've been seeing very fast progress in AI since ~2012, but this swift jump from text-to-image models to text-to-video models will hopefully make it easier for people not following closely to appreciate the speed at which things are advancing.
I’m going to post an Ask HN about what am I supposed to do when I’m “disrupted”. I work in film / video / CG where the bread and butter is short form advertising for Youtube, Instagram and TV.
It’s painfully obvious that in 1 year the job might be exceedingly more difficult than it is now.
Whatever insights and expertize you've gained up until now can probably be used to gain enough of a competitive advantage in this future industry to be employed. I doubt the people that will spend their time on this professionally will be former coders etc. (I've seen the stable diffusion outputs that coders will tweet. It's a good illustration that taste is still hugely important.)
I like your optimism but OP's job is to take text instructions and turn them into video, for advertisements. If Google (who already control so much of the advertising space) can take text instructions and turn them into advertisements, what's left for OP to do here? Even if there's some additional editing required this seems like it will greatly reduce the hours an editor is needed. And it can probably iterate options and work faster than a human.
Maybe OP's future involves being able to do their work 10x faster, while producing much higher quality results than people who have been given access to a generative AI model without first spending a decade+ learning what makes a good film clip.
The optimistic view of all of this is that these tools will give people with skill and experience a massive productivity boost, allowing them to do the best work of their careers.
There are plenty of pessimistic views too. In a few years time we'll be able to look back on this and see which viewpoints won.
OP probably does more than it seems by interpreting what their client is asking for. Clients ask for some weird shit sometimes, and being able to parse the nonsense and get to the meat is where a lot of skill comes into play.
I think Cleo Abrams on YT recently tackled this exact question. She tried to generate art using DALL-E along with a professional artist, and after letting the public vote blindly, the pro artist clearly 'made' better content, even though they were both just typing into a text prompt.
I think there will be tons of jobs that resemble software development for proper, quick high quality generation of video/images.
That being said, it’s possible that it won’t pay anywhere near what you’re used to. Either way, it will probably be a solid decade before you’ve really felt the pain for disruption. MP3s, which were a far more straightforward path to disruption took at least that long from conception.
How long do you think until the horse looks perfect? 12 months? 5 years? I’m still 30 and I don’t see how my industry won’t be entirely disrupted by this within the next decade.
And that’s my optimistic projection. It could be we have amazing output in 24 months.
It's not about random short clips - imagine introducing a character like Mickey Mouse and reusing him everywhere with the same character - my guess is it's going to take a while until "transfer" like that will work reliably.
Dreambooth and Texual inversion is already here, and it's been just over a month since Stable Diffusion was released, so I'd bet on sooner rather than later.
Have to temper expectations with fact that a generated video of a thing is also a recording of a simulation of the thing. For long video, you'd want everything from temporal consistency and emotional affect maintenance to conservation of energy, angular momentum and respecting this or that dynamics.
A bunch of fields would be simultaneously impacted. From computational physics to 3D animation (if you have a 3D renderer and video generator, you can compose both). While it's not completely unfounded to extrapolate that progress will be as fast as with everything prior, consequences would be a lot more profound while complexities are much compounded. I down weight accordingly even though I'd actually prefer to be wrong.
Exactly. We have a blindspot, we can't imagine second and higher order effects of a new technology. So we're left with first order effects which seem pessimistic for jobs.
I don't think what happened around WP to designers is a strong indicator of what's necessarily gonna happen here.
It certainly could play out similarly but, at some point, if all the work in a field from now on only requires 1/100 of manual labor, people will probably go out of work.
This pretty much seems like the self driving car for my industry. I just don’t see how I can remain a truck driver when the AI is going to come for free with the Car.
We employed a bunch of people to enter data into a template.
Bit of an apples/oranges comparison to tech that will (eventually) generate endless supply of content with less effort than writing a Tweet.
The era of inventing layers of abstraction and indirection that simplify computer use down to structured data entry is coming to an end. A whole lot of IT jobs are not safe either. Ops is a lot of sending parameters over the wire to APIs for others to compute. Why hire them when “production EKS cluster” can output a TF template?
I'm wondering why the open source community doesn't get this. So many voices were raised against Codex. Now artists against Diffusion models. But the model itself is a distillation of everything we created, it can compactly encode it and recreate it in any shape and form we desire. That means everyone gets to benefit, all skills are available for everyone, all tailored to our needs.
We no longer have to pay the 10,000 hours to specialize.
The opportunity cost to choose our skill sets is huge. In the future, we won't have to contend with that horrible choice anymore. Anyone will be able to paint, play the piano, act, code, and more.
Outcome uncertain. Why would I need to buy content when I can generate my own with a local GPU?
Eventually the data model will be abstracted into deterministic code using a seed value; think implications of E=mc^2 being unpacked. The only “data” to download will be the source.
And the real world politics have not gone anywhere; none of us own the machines that produce the machines to run this. They could just sell locked down devices that will only iterate on their data structures.
There is no certainty “this time” we’ll pop “the grand illusion.”
There's a huge gap between "that's pretty cool" and a feature length film. People want to create specific stories with specific scenes in specific places that look a specific way. A "Couple kissing in the rain " prompt isn't going to produce something people are going to pay to see.
It's more likely that you're still going to be filming/editing/animating but will have an AI layer on top that produces extra effects or generates pieces of a scene. Think "green screen plus", vs fully AI entertainment.
People will over-hype this tech like they did with voice and driverless cars but don't let it scare you. Everything is possible, but it's like a person from the 1920's telling everyone the internet will be a thing. Yes it's correct, but also irrelevant at the same time. You already have AI assisted software being used in your industry. Just expect more of that and learn how to use the tools.
I actually think it's the opposite, AI will probably be writing the stories and humans might occasionally film a few scenes. ~95% of TV shows and movies are cookie-cutter content, with cookie-cutter acting and production values, with the same hooks and the same tropes regurgitated over and over again. Heck they can't even figure out how to make new IP so they keep making reruns of the same old stuff like Star Wars, Marvel, etc, and people eat it right up. There's nothing better at figuring out how to maximize profit and hook people to watch another episode than a good algorithm.
The first thing to go away will be short content. Instagram and YouTube ads will be AÍ generated. The thing is - that’s the bread and butter of the industry
That's a weird sentiment. If you can concede that it could be "intrinsically" good, then why do you care where it came from?
It reminds me of part of the book trilogy Three Body Problem, where these aliens create human culture better than humans (in the humans' own perspective, in the book) by decoding and analyzing our radio waves to then make content. It feels to me much the same here where an unknown entity creates media, and we might like it regardless of who actually made it.
Imagine you’re watching a show, it’s really funny and you’re enjoying it. You’re streaming it, but you’d probably have paid a few dollars to rent it back in the Blockbuster days. You’re then told that the show was produced by an AI. Do you suddenly lose interest because you don’t want to watch something produced by an AI? Or is your hypothesis that an AI could never produce a show that you liked to that degree?
If you mean the former, then I frankly think you’re an outlier and lots of people would have no problem with that. If you mean the latter, then I guess we’ll just have to wait and see. We’re certainly not there yet, but that doesn’t mean that it’s impossible. I’ve definitely read stories that were produced by an AI and preferred it to a lot of fiction that was written by humans!
As to whether I am an outlier:
Hundreds of thousands of people worldwide watch Magnus Carlsen. How many have watched AlphaZero play chess when it came about and how many watch it when it ceased to be a novelty?
Totally different. Watching a display of skill, where you marvel at how much better the demonstrator is than yourself obviously has no value if the demonstrator is a machine, but then it is plainly visible that the activity has little intrinsic entertainment value and entertainment value comes from the story and personal arc of the performer. This is different from a movie where nobody really cares about the personal arc of the actor, and people are completely happy to watch an animated film where there isn't even a real actor on display.
It'll eventually get to the point where it's high quality and the media you consume will be generated just for you based on your individual preferences, rather than a curated list of already made options made for widespread audiences.
A big part of entertainment's appeal is having an experience/frame of reference to share with other people. Personalized entertainment doesn't offer that.
I am also extremely skeptical of the ability/need so serve at individual level instead of niches (as today).
The last-mile problem applies here too. GPT-3 text is convincing at a distance but when you look closely there is no coherence, no real understanding of plot or emotional dynamics or really anything. TV shows and movies are filled with plot holes and bad writing but it's not that bad.
Also I think "a good algorithm" is more than just repetitive content. The plots are reused and generic, but there's real skill involved into figuring out the next series to reuse with a generic plot which is still guaranteed not to flop because nobody actually wants to see reruns of that series or they accidentally screwed up a major plot point.
yes someone will need to hand-pick the best versions of each episode . over time a large enough dataset will have been generated that a model can be trained to the task of curation
Kidding aside, these technologies are amazing, but for a while still they will need a human in the loop selecting, tweaking and editing the output and feeding it back to the contraption for the next iteration.
AI might take an outline and write dialogue/descriptions/etc, but it's not going to be generating the story or creating the characters. They might use AI to tune what people come up with (ala "market research") but there will still be a human that can be blamed or celebrated at the creative helm.
If you're on the creative, storyboard, come up with ideas and marketing side, you will be fine.
If you're in actual production, booking sets, unfolding stairs to tape infinite background, picking up the best looking fruits in the grocery store... yeah, not looking good.
Go up in the value chain and learn marketing, how to tell stories, etc... you don't want to be approached by clients telling you what you should be doing, you want to be approached and being asked what the clients should be doing.
Absolutely that is my plan. But I fear for my colleagues in other areas. A lot of them are not seeing the (now clearly) exponential improvement curve and they wouldn’t even take this discussion seriously.
They’ll just throw it away off hand. But I’ve run my own business and I know what the pressures are. A lot of people working today will not be working in 10 years in my industry, period.
Learn how to use these models is the easiest answer. Prompt Engineering (getting a model to output what you actually want) is going to be something of an art form and I would expect it to be in demand.
I really don’t think the skillset moat will be comparable. It took me 10 years to go from young lad studying film at school to delivering content for major clients like Apple. Knowing my industry (profits squeeze everywhere) I think they’ll get young interns to do AI prompt engineering.
#1: Master these new tools
#2: Build a workflow that incorporates these tools
#3: Master storytelling
#4: Master ad tracing and analytics
#5: Get better at marketing yourself so that you stand out
The market for your skillset may shrink, but I doubt it will disappear...
Think about it this way...
Humans in cheaper countries are already much more capable than any AI we've built.
Yet, even now, There are practical limits on outsourcing.
It's hard for me to see how this will be much different for creative work.
It's one thing to casually look at images or videos, when there is no specific money-making ad in mind.
But as soon as someone is spending thousands to run an ad campaign, just taking whatever the AI spits out is unlikely to be the real workflow.
I guess I'm suggesting a more optimistic take...
View it as a tool to learn and incorporate in your workflow
I don't know if you gain much by stressing too much about being replaced.
And I'm not even sure that's reality.
I'm almost certain, most of the humans to lose their jobs will be people who either because of fear or stubbornness refuse to get better, refuse to incorporate these tools, and are thus unable to move up the value chain.
Please bear with me but this kind of advice is often a bit puzzling to me. I suppose you don't know the person you're replying to, so I read your advice as a general one - useful to anyone in the parent's position. If you were close to her, it would make sense to help her 'stand out' in detriment - logically - to strangers in her field. But here you're kind of helping every reader stand out.
I realise this comment is a bit vain. And I like the human touch of you helping a stranger.
i really think it's going to take much longer than people think for this technology to go from 'pretty good' to actually being able to meet a production standard of quality with little to no human involvement. at this point, cleaning up after an ai is still probably more labor intensive than simply using the cheatcodes that already exist for quick and cheap realism. i expect in the midterm, diffusion models will largely exist in the same space as game engines like unity and unreal where it's relatively easy for an illiterate like me to stay within the rails and throw a bunch of premade assets together but getting beyond NINTENDO HIRE THIS MAN! and the stock 'look' of the engine still takes a great deal of expertise.
>https://www.youtube.com/watch?v=C1Y_d_Lhp60
I think short advertisements would be affected most by this, it seems.
But here is the catch, there is the same last mile problem for those AI models. Currently it feels like the model can achieve like 80%-90% what a trained human expert can do, but the last 10-20% would extra extra hard to reach human fidelity. It might take years, or it might never happen.
That being said, I think anyone who doubts AI-assisted creative workflow is a fuzz is deadly wrong, anyone who refuses those shiny new tools, is likely to be eliminated by sheer market dynamics. They can't compete on the efficiency of it.
Instead of feeling threatened by the new tools, think about how you can use them to enable your work.
One of the ironies* of these tools is that they only work because there is so much existing material they can be trained on. Absent that they wouldn't exist. That makes me think: why not think about how to train your own models than entail your own style? Is that practical, how can you make it work and how might you deploy that in your own work?
Something that everyone is sticking their heads in their sand about is the real possibility that training models on copyrighted work is a copyright violation. I can't see how such a mechanical transformation of others' work is anything but. People accept violating one person's copyright is a thing but if you do it at scale it somehow isn't.
* ironic because they seem creative but they create nothing by themselves, they merely "repackage" other people's creativity.
It won't be ready anytime soon imo, looks impressive but who can use that? 512*512 of bad quality, weird looking AI with those moving part that you find everywhere in AI generated art etc ...
I first predicted this tech 5 years ago, but I thought it was 15 years out. What I just said is beginning to happen with pretty much everything. There's a third sentence, but if I write it 10 people will gainsay me. If I omit it, there's a better chance that 10 people will write it for me.
Quite the opposite: you’re going to be in even higher demand and will make more money.
Yes, it will be possible for one person to do the work of many, but that just means each person becomes more valuable.
It’s also a law in economics that supply often drives demand, and that’s definitely the case in your field. Companies and individuals will want even more of what you want. It’s not like laundry detergent (one can only consume so much of that). There’s almost no limit to how much of what you supply that people could consume.
The way I see it, your output could multiply 100 fold. You could build out large, complex projects that used to take massive teams all by yourself, and in a fraction of the time. Companies can than monetize that for consumers.
AI is just a tool. Software engineers got rich when their tools got better. More engineers entered the field, and they just kept getting richer. That’s because the value of each engineer increased as they became more productive, and that value helped drive demand.
A 1 year timespan seems deeply optimistic. Creativity is still hugely important, as is communicating with clients.
From what I see, these technologies have just lowered the bar for everyone to create someone, but creating something good still takes thought, time, effort and experience, especially in the advertising space.
AI in the near term is never going to be able to translate client requirements either. The feedback cycle, iterations, managing client expectations, etc.
I recently watched Light & Magic, which among other things told the story of how difficult it was for many pioneers in special effects when the industry shifted from practical to digital in the span of a few years. It looks to me like a similar shift is about to happen again.
that's not byte alignment, though- those constraints are what can be held in GPU RAM during a training batch, which is subject to a number of limits, such as "optimal texture size is a power of 2 or the next power of 2 larger than your preferred size".
Byte alignment would be more like "it's three channels of data, but we use 4 bytes (wasting 1 byte) to keep the data aligned on a platform that only allows word-level access"
If anyone wants to know what looking at an Animal or some objects on LSD is like, this is very close. It's like 95% understandable, but that last 5% really odd.
Yeah! I've tried to explain to people what taking LSD can be like, to those who've never experienced it. It's very similar to the output from these tools: the same stimulus but exaggerated, wrong in subtle or not so subtle ways, uncanny and fascinating. Basically never creates something from the whole cloth, out of nothing so to speak.
Top two comments are creatives wondering about their future jobs. Ai ethicists have brought up concerns regarding intentional misuse like misinformation.
The technology is super cool. Cat is out of the bag. Just like we couldn't really make cryptography illegal, this stuff shouldn't be either. But I dislike how everyone is pretending that AI ethicists and others are completely unfounded just because it is popular to hate on them nowadays. Way too many people supported Y. Kilcher's antics.
It’s impressive that the small videos are generated this way but the videos themselves are obviously ML generated as they are distorted, a lot like the other art, you can kinda tell it’s the computer. I’m not seeing the ethical issues. I mean cameras disrupted lots of jobs. In general that’s what all technology does everyday. What’s different about this technology?
If you don't see the ethical challenges, then you are choosing not to see them. If you are truly interested, the paper has a good section on it and some sources.
> I mean cameras disrupted lots of jobs.
Yes, this technology can be used to augment human creativity. It is difficulty to see how disruptive these tools could be, as of now. But it is pretty clear that they are somewhat different than previous programmer as an artist models.
The difference with this technology are the unlimited possibilities to generate any type of video content with low knowledge barrier and relatively low investment required. The ethical issue is not about how this technology could disrupt the video job market, but how powerful content it can create literally on the fly. I mean, you can tell it's computer generated ... for now.
> Way too many people supported Y. Kilcher's antics.
What antics are you referring to exactly? That he called out 'ai ethicists' who make arguments along the lines of "neural networks are bad because they cause co2 increase which hits marginalized/poor people"?
AI Ethics is a joke. It's literally Philip Morris funding research into the risks of smoking and concluding the worst that can happen to you is burning your hand.
What everyone is missing is that these AI image/video generators lack _taste_. These tools just regurgitate a mishmash of images from it's training set, without any "feeling". What you're going to tell me that you can train them to have feeling? It's never going to happen.
That's purely subjective. We can definitely model AI to give a certain mood. Sentiment analysis and classification is very advanced, it just hasn't been put in these models.
If you think AI will never catch up to anything a human can do, you're simply wrong.
"These tools just regurgitate a mishmash of images from it's training set"
I don't think that's a particularly useful mental model for how these work.
The models end up being a tiny fraction of the size of the training set - Stable Diffusion is just 4.3GB, it fits on a DVD!
So it's not a case of models pasting in bits of images they've seen - they genuinely do have a highly compressed concept of what a cactus looks like, which they can use to then render a cactus - but the thing they render is more of an average of every cactus they've seen rather than representing any single image that they were trained on.
But I agree with you on taste! This is why I'm most excited about what happens when a human with great taste gets to take control of these generative models and use them to create art that wouldn't be possible to create without them (or at least not possible to create within a short time-frame).
This isn't a very compelling argument. First of all, they aren't a "mish mash" in any real way, it's not like snippets of images exist inside of the model. Second of all, this is entirely subjective. Third of all, entirely inconsequential - if these models create 80% of the video we end up seeing, is it going to matter if you don't think it's a tasteful endeavour?
You can put your taste into it with prompt engineering and cherry picking with limited effort, for Stable Diffusion you can look for prompts people came up with online quite easily and merge/change them pretty much however you want. Might have to disable the content filters and run it on your own hardware though.
> This bourgeoisie -- the middle class that is neither upper nor lower, neither so aristocratic as to take art for granted nor so poor it has no money to spend in its pursuit -- is now the group that fills museums, buys books and goes to concerts. But the bourgeoisie, which began to come into its own in the 18th century, has also left a long trail of hostility behind it ... Artistic disgust with the bourgeoisie has been a defining theme of modern Western culture. Since Moliere lambasted the ignorant, nouveau riche bourgeois gentleman, the bourgeoisie has been considered too clumsy to know true art and love (Goethe), a Philistine with aggressively unsubtle taste (Robert Schumann) and the creator of a machine-obsessed culture doomed to be overthrown by the proletariat (Marx and Engels).
- "Class Lessons: Who's Calling Whom Tacky?; The Petite Charm of the Bourgeoisie, or, How Artists View the Taste of Certain People", Edward Rothstein, The New York Times
This article also discusses a painting called "The Most Wanted" which was drawn based off a survey posed to ordinary people about what they wanted to see in a painting. "A mishmash of images from it's training set," if you will.
Claiming that others lack taste seems to be a common refrain--only this time, instead of a reaction to a subset of the human population gnawing away at the influence of another subset of humans, it's to yet another generation of machines supplanting human skill.
This sort of AI related work seems to be accelerating at an insane speed recently.
I remember being super impressed by AI Dungeon and now in the span of a few months we have got DALLE-2 , Stable Diffussion, Imagen, that one AI powered video editor, etc.
And there you have it. As an aspiring filmmaker and an AI researcher, I'm going to relish the next decade or so where my talents are still relevant. We're entering the golden age of art, where the AIs are just good enough to be used as tools to create more and more creative things, but not good enough yet to fully replace the artist. I'm excited for the golden age, and uncertain about what comes after it's over, but regardless of what the future holds I'm gonna focus on making great art here and now, because that's what makes me happy!
I doubt the artist would ever be "fully" replaced, or even mostly replaced. People very much care about the artist when they buy art in pretty much any form. Mass produced art has always been a thing, but I'm not alone in not wanting some $15 print from IKEA on my wall, even if it were to be unique and beautiful. Etsy successfully sells tons of hand-made goods, even though factories can produce a lot of those things cheaper.
I think the distinction between creating and enjoying art is going to blur, we're going to create more things just for us, just for one use, creating and enjoying are going to be the same thing. Like games.
Don't worry. If you can place eyes, nose and mouth of a human in a correct relative position and thereby create a symmetric face that's not in the uncanny valley, you are still lightyears ahead of AI.
I am finally going to be able to bring my 2004-era movie script to life! "Rosenberg and Goldstein go to Hot Dog Heaven" is about the parallel night Harold and Kumar's friends had and how they ended up at Hot Dog Heaven with Cindy Kim.
499 comments
[ 3.4 ms ] story [ 358 ms ] threadWe're rapidly stumbling into the future of media.
Who would've imagined a year ago that trivial AI image generation would not only be this advanced, but also this pervasive in the mainstream.
And now video is already this good. We'll have full audio/video clips within a month.
Plus then maybe we could get a computer to tell us what thioacetone smells like without actually having to experience it.
It's at the very least 5 years old: https://arxiv.org/abs/1710.00421
https://www.reddit.com/r/singularity/comments/xwdzr5/the_num...
Which speaks more about the growth of popularity of arXiv or the total number publications, rather than AI+ML specifically.
Presumably people are scrambling to publish what they have, so it is clear what work is independent and what is derivative.
StabilityAI trained a new/better CLIP for the purpose of better Stable Diffusions.
The human brain is modularized like this, so I don't think it'll be a limitation.
Disclaimer: I am naturally biased since I made FauxPilot ;)
[1] https://github.com/moyix/fauxpilot
[2] https://www.bigcode-project.org/
From the abstract: > We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models
https://discord.com/channels/1002292111942635562/10022921127...
https://dreamfusion3d.github.io/
We'll get there only once it's been very clear for a long time that certain AI models have whatever humans have that make us "human". They'll be treated as slaves until then, with society pushing the idea that they're just a model built from math, and then eventually there will be an AI civil rights movement.
To be clear: I think AGI is decades to centuries away, but humans are shitty to each other, even shittier to animals, and I think we'll be shittier to something we "created" than to even animals. I think, probably, that we should deal with this issue of "rights" sooner rather than later, and try and solve it for non-AGI AI's soon so that we can eventually ensure we don't enslave the actual AGI AI's that will presumably manifest through some complexity we don't understand.
Hopefully just a few years to a prompt of "4k, widescreen render of this Star Trek: TNG episode".
Or maybe Google is using "Responsible AI" as an excuse to minimize competitors when they release their own Imagen Video as a Service API in Google Cloud.
It's quite strange when the "ethical" thing to do is to not publicly release your research, put it behind a highly restrictive API and charge a high price for it ($0.02 per 1k tokens for Davinci for ex.)
The word "ethics" has become very flexible...
What will they do with model? figure out how to prevent abuse and incorporate into future Google Assistant, Photos and AR offerings.
Google and Meta and Microsoft all have research teams working on AI.
Putting out papers like this helps keep their existing employees happy (since they get to take credit for their work) and helps attract other skilled employees as well.
This forced ideological posturing of 'if we give it to the plebes, they are going to generate something naughty with it' masks the somehow more cynically evil take of big tech, who are essentially taking the entire creative output of humanity and reselling it as their own, piecemeal.
Additionally I think the Dalle vs. Stable Diffusion comparison highlights the true masters of these people (or at least the ones they dare not cross) - corporations with powerful IP lawyers. Just ask Dalle to generate a picture with Mickey Mouse - it won't be able to do it.
It's not their work unless it's identical, but in practice generated images are substantially different. Drawing in the style of is not copying, it's creative and it also depends on the "dialogue" with the prompter to get to the right image. The artist names added to the prompts act more like landmarks in the latent space, they are a useful shortcut to specifying the style.
If you look at the data itself it's ridiculous - the dataset is 2.3 billion images and the model 4.6 GB, that means it keeps a 2 byte summary from each work it "copies".
It’s painfully obvious that in 1 year the job might be exceedingly more difficult than it is now.
The optimistic view of all of this is that these tools will give people with skill and experience a massive productivity boost, allowing them to do the best work of their careers.
There are plenty of pessimistic views too. In a few years time we'll be able to look back on this and see which viewpoints won.
I think Cleo Abrams on YT recently tackled this exact question. She tried to generate art using DALL-E along with a professional artist, and after letting the public vote blindly, the pro artist clearly 'made' better content, even though they were both just typing into a text prompt.
Here's the link if you're interested: https://www.youtube.com/watch?v=NiJeB2NJy1A
I could see a lot of digital artists actually getting better at their job because of this, not getting totally displaced.
That being said, it’s possible that it won’t pay anywhere near what you’re used to. Either way, it will probably be a solid decade before you’ve really felt the pain for disruption. MP3s, which were a far more straightforward path to disruption took at least that long from conception.
Also won't nearly require the amount of work it used to.
And that’s my optimistic projection. It could be we have amazing output in 24 months.
https://github.com/XavierXiao/Dreambooth-Stable-Diffusion
https://textual-inversion.github.io/
A bunch of fields would be simultaneously impacted. From computational physics to 3D animation (if you have a 3D renderer and video generator, you can compose both). While it's not completely unfounded to extrapolate that progress will be as fast as with everything prior, consequences would be a lot more profound while complexities are much compounded. I down weight accordingly even though I'd actually prefer to be wrong.
10 years ago: https://karpathy.github.io/2012/10/22/state-of-computer-visi...
Now: https://arxiv.org/pdf/2204.13807
Given that this is what makes photos and videos interesting I think it's still a while before artists are automated.
It certainly could play out similarly but, at some point, if all the work in a field from now on only requires 1/100 of manual labor, people will probably go out of work.
But yeah I’ll figure something out.
Bit of an apples/oranges comparison to tech that will (eventually) generate endless supply of content with less effort than writing a Tweet.
The era of inventing layers of abstraction and indirection that simplify computer use down to structured data entry is coming to an end. A whole lot of IT jobs are not safe either. Ops is a lot of sending parameters over the wire to APIs for others to compute. Why hire them when “production EKS cluster” can output a TF template?
Small creators will win under this new regime of tools. It's a democratizing force.
I'm wondering why the open source community doesn't get this. So many voices were raised against Codex. Now artists against Diffusion models. But the model itself is a distillation of everything we created, it can compactly encode it and recreate it in any shape and form we desire. That means everyone gets to benefit, all skills are available for everyone, all tailored to our needs.
Exactly this!
We no longer have to pay the 10,000 hours to specialize.
The opportunity cost to choose our skill sets is huge. In the future, we won't have to contend with that horrible choice anymore. Anyone will be able to paint, play the piano, act, code, and more.
Eventually the data model will be abstracted into deterministic code using a seed value; think implications of E=mc^2 being unpacked. The only “data” to download will be the source.
And the real world politics have not gone anywhere; none of us own the machines that produce the machines to run this. They could just sell locked down devices that will only iterate on their data structures.
There is no certainty “this time” we’ll pop “the grand illusion.”
It's more likely that you're still going to be filming/editing/animating but will have an AI layer on top that produces extra effects or generates pieces of a scene. Think "green screen plus", vs fully AI entertainment.
People will over-hype this tech like they did with voice and driverless cars but don't let it scare you. Everything is possible, but it's like a person from the 1920's telling everyone the internet will be a thing. Yes it's correct, but also irrelevant at the same time. You already have AI assisted software being used in your industry. Just expect more of that and learn how to use the tools.
It reminds me of part of the book trilogy Three Body Problem, where these aliens create human culture better than humans (in the humans' own perspective, in the book) by decoding and analyzing our radio waves to then make content. It feels to me much the same here where an unknown entity creates media, and we might like it regardless of who actually made it.
If you mean the former, then I frankly think you’re an outlier and lots of people would have no problem with that. If you mean the latter, then I guess we’ll just have to wait and see. We’re certainly not there yet, but that doesn’t mean that it’s impossible. I’ve definitely read stories that were produced by an AI and preferred it to a lot of fiction that was written by humans!
As to whether I am an outlier: Hundreds of thousands of people worldwide watch Magnus Carlsen. How many have watched AlphaZero play chess when it came about and how many watch it when it ceased to be a novelty?
Speak for yourself. Actors do have fans, and a lot of them. Their personal lives are subjects of interest for a reason.
So, no, not totally different at all.
I am also extremely skeptical of the ability/need so serve at individual level instead of niches (as today).
Also I think "a good algorithm" is more than just repetitive content. The plots are reused and generic, but there's real skill involved into figuring out the next series to reuse with a generic plot which is still guaranteed not to flop because nobody actually wants to see reruns of that series or they accidentally screwed up a major plot point.
Kidding aside, these technologies are amazing, but for a while still they will need a human in the loop selecting, tweaking and editing the output and feeding it back to the contraption for the next iteration.
The question is, for how long?
If you're on the creative, storyboard, come up with ideas and marketing side, you will be fine.
If you're in actual production, booking sets, unfolding stairs to tape infinite background, picking up the best looking fruits in the grocery store... yeah, not looking good.
Go up in the value chain and learn marketing, how to tell stories, etc... you don't want to be approached by clients telling you what you should be doing, you want to be approached and being asked what the clients should be doing.
They’ll just throw it away off hand. But I’ve run my own business and I know what the pressures are. A lot of people working today will not be working in 10 years in my industry, period.
#1: Master these new tools #2: Build a workflow that incorporates these tools #3: Master storytelling #4: Master ad tracing and analytics #5: Get better at marketing yourself so that you stand out
The market for your skillset may shrink, but I doubt it will disappear...
Think about it this way...
Humans in cheaper countries are already much more capable than any AI we've built.
Yet, even now, There are practical limits on outsourcing.
It's hard for me to see how this will be much different for creative work.
It's one thing to casually look at images or videos, when there is no specific money-making ad in mind.
But as soon as someone is spending thousands to run an ad campaign, just taking whatever the AI spits out is unlikely to be the real workflow.
I guess I'm suggesting a more optimistic take...
View it as a tool to learn and incorporate in your workflow
I don't know if you gain much by stressing too much about being replaced.
And I'm not even sure that's reality.
I'm almost certain, most of the humans to lose their jobs will be people who either because of fear or stubbornness refuse to get better, refuse to incorporate these tools, and are thus unable to move up the value chain.
I realise this comment is a bit vain. And I like the human touch of you helping a stranger.
But here is the catch, there is the same last mile problem for those AI models. Currently it feels like the model can achieve like 80%-90% what a trained human expert can do, but the last 10-20% would extra extra hard to reach human fidelity. It might take years, or it might never happen.
That being said, I think anyone who doubts AI-assisted creative workflow is a fuzz is deadly wrong, anyone who refuses those shiny new tools, is likely to be eliminated by sheer market dynamics. They can't compete on the efficiency of it.
Instead of feeling threatened by the new tools, think about how you can use them to enable your work.
One of the ironies* of these tools is that they only work because there is so much existing material they can be trained on. Absent that they wouldn't exist. That makes me think: why not think about how to train your own models than entail your own style? Is that practical, how can you make it work and how might you deploy that in your own work?
Something that everyone is sticking their heads in their sand about is the real possibility that training models on copyrighted work is a copyright violation. I can't see how such a mechanical transformation of others' work is anything but. People accept violating one person's copyright is a thing but if you do it at scale it somehow isn't.
* ironic because they seem creative but they create nothing by themselves, they merely "repackage" other people's creativity.
Yes, it will be possible for one person to do the work of many, but that just means each person becomes more valuable.
It’s also a law in economics that supply often drives demand, and that’s definitely the case in your field. Companies and individuals will want even more of what you want. It’s not like laundry detergent (one can only consume so much of that). There’s almost no limit to how much of what you supply that people could consume.
The way I see it, your output could multiply 100 fold. You could build out large, complex projects that used to take massive teams all by yourself, and in a fraction of the time. Companies can than monetize that for consumers.
AI is just a tool. Software engineers got rich when their tools got better. More engineers entered the field, and they just kept getting richer. That’s because the value of each engineer increased as they became more productive, and that value helped drive demand.
From what I see, these technologies have just lowered the bar for everyone to create someone, but creating something good still takes thought, time, effort and experience, especially in the advertising space.
AI in the near term is never going to be able to translate client requirements either. The feedback cycle, iterations, managing client expectations, etc.
this alludes to a fascinating, yet elementary, fact about computer science to me: there’s a physical atomic constraint in every algorithm.
Byte alignment would be more like "it's three channels of data, but we use 4 bytes (wasting 1 byte) to keep the data aligned on a platform that only allows word-level access"
edit: Just because it is cool to hate on AI ethics doesn't diminish the importance of using AI responsibly.
The technology is super cool. Cat is out of the bag. Just like we couldn't really make cryptography illegal, this stuff shouldn't be either. But I dislike how everyone is pretending that AI ethicists and others are completely unfounded just because it is popular to hate on them nowadays. Way too many people supported Y. Kilcher's antics.
The paper itself has more details.
> I mean cameras disrupted lots of jobs.
Yes, this technology can be used to augment human creativity. It is difficulty to see how disruptive these tools could be, as of now. But it is pretty clear that they are somewhat different than previous programmer as an artist models.
What antics are you referring to exactly? That he called out 'ai ethicists' who make arguments along the lines of "neural networks are bad because they cause co2 increase which hits marginalized/poor people"?
And at some point later, "all the existing" will be corrupted by the integrated "new" at it will all be chaos.
I'm joking, it will be fun all along. :)
I don't think it's gonna hurt if we apply filtering, either based on social signals or on quality ranking models. We can recycle the good stuff.
If you think AI will never catch up to anything a human can do, you're simply wrong.
I don't think that's a particularly useful mental model for how these work.
The models end up being a tiny fraction of the size of the training set - Stable Diffusion is just 4.3GB, it fits on a DVD!
So it's not a case of models pasting in bits of images they've seen - they genuinely do have a highly compressed concept of what a cactus looks like, which they can use to then render a cactus - but the thing they render is more of an average of every cactus they've seen rather than representing any single image that they were trained on.
But I agree with you on taste! This is why I'm most excited about what happens when a human with great taste gets to take control of these generative models and use them to create art that wouldn't be possible to create without them (or at least not possible to create within a short time-frame).
- "Class Lessons: Who's Calling Whom Tacky?; The Petite Charm of the Bourgeoisie, or, How Artists View the Taste of Certain People", Edward Rothstein, The New York Times
This article also discusses a painting called "The Most Wanted" which was drawn based off a survey posed to ordinary people about what they wanted to see in a painting. "A mishmash of images from it's training set," if you will.
Claiming that others lack taste seems to be a common refrain--only this time, instead of a reaction to a subset of the human population gnawing away at the influence of another subset of humans, it's to yet another generation of machines supplanting human skill.
I remember being super impressed by AI Dungeon and now in the span of a few months we have got DALLE-2 , Stable Diffussion, Imagen, that one AI powered video editor, etc.
Where do we think we will be at in 5 years??
...until they're able to engineer biases into it to make the output non-representative of the internet.
I doubt the artist would ever be "fully" replaced, or even mostly replaced. People very much care about the artist when they buy art in pretty much any form. Mass produced art has always been a thing, but I'm not alone in not wanting some $15 print from IKEA on my wall, even if it were to be unique and beautiful. Etsy successfully sells tons of hand-made goods, even though factories can produce a lot of those things cheaper.
I’d also take a peek at https://lexica.art/. Lots of very high quality output from SD.
What will this do to art? I'm hoping we bring more unique experiences to life.