As I said before, World models will become simulation engines of AAA games where there is no need to release the actual game and even if the real of the game exists it will never be released and will be available on their hosted cloud back-end, then simulated and streamed back to you.
So, Stadia + World Models = New generation of DRM for games.
Game industry is still struggling to find a way to incorporate regular LLMs without suffering catastrophic edge cases. A long way to go before these can do more than fancy tech demo, I think.
Why? Why would a non-deterministic engine be better for a game when it can be instead structured and made deterministic and then more effectively distributed to consumer hardware?
> We'll be riding the huge wave of indie titles that are incoming.
The problem will be separating the wheat (people with good stories and concepts enabled by Ai) from the chaff (people with no taste slopping out trash with Ai)
And nobody will play that trash, and the world won't change.
Also, lmao, they're already trying to nickel and dime you and sell you microtransactions that took 5 minutes to make for 10 bucks. If you think they're going to pay the costs for running a world model for every single person, lol.
None of this makes sense. Think of the economics involved here: is it cheaper to run tens/hundreds of thousands of GPU clusters to simulate a unique game world for millions of players, or would it be cheaper to just write the god damn code once, and then host it for download, or to allow them to be streamed?
I see comments on here that refer to something someone has said before, for example, "as I said before", or "I said this the other month", as if the entire universe is following a single person's statements. It seems very self-referential and self-consumed.
I don't know if I'm missing the point, but Genie 4 almost looks worse than Genie 3 in terms of player control? It doesn't even seem to respond to inputs, it just looks like they took some AI generated videos and put the key widgets on top.
I think you're exactly right. This was already a problem with Genie 3, but they've appear to have gone all in on video generation over controllability.
The original Genie was a great piece of work with a real sense of purpose - to enable agents to learn about the real world in the way Dreamer or Schmidhuber's paper sketched. But now most of the people that drove that mission have left Deepmind (Jack Parker-Holder, Tim Rocktaschel, Demis) and it's become this instead.
Astonishing to me that a company like Google would be willing to publish such a terrible looking product demo. Interactive, but with completely made-up inputs that are ignored?
The volume of content on YouTube is enormous, I’m kind of surprised they offer auto captions at all given that they have little to no competition in the space.
Ok but like, surely the Deepmind team could sneeze and get flawless captions for this video. Why rely on the default YouTube captions instead of just attaching correct ones? It seems silly.
Yeah if they cared they could create their own captions for sure. They probably didn’t even think about it. Maybe some intern will do it after reading this thread
They already spent a lot of processing time per video before AI in transcoding the videos to different resolutions, formats and quality. Captions are pretty cheap in comparison.
So world models capture inherent complexity of physical reality to help teach AI about hidden laws which probably helps train both LLMs and multimodal models used for autonomous vehicles and robots? And AMD acquiring World Labs is a move to outdo Nvidia’s work in this space. And Google probably sees this as a gateway to a new platform of autonomous vehicles and humanoids that they could rent out as a CSP? Competing with Nvidia, Tesla and complementing Waymo. Am I getting that right?
That all sounds right to me, but apparently they can also be used for simply generating videos, or "unlocking the creation of new interactive experiences"... which both seem like competing goals with everything you just mentioned.
Surprised that "do you really think we're going to make it?" scene made the cut of the first video, her expression and tone is totally out of sync with the scene and demonstrates the problem with almost every AI gen video I've come across when it comes to capturing humanity.
The painting->interactive scene looks astonishing though
In the clip with a marble bust dropped on a watermelon, at this point https://imgur.com/a/IHLnkUs the bust should've toppled over, if the physics were properly modeled.
That's the thing about machine learning over time, and the big hole in the whole world model idea. Time and time again we make simulations and train models and we think that will work. It never does.
And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation.
So we get to a point and people keep making complicated simulations of one kind or another ... and then someone who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point.
How often do things in the real world not behave exactly as you expected them to?
I feeling I get from this is that this could be the mental model for a robot in the real world, continuously corrected by actual events.
We all have a world model in our heads of what we think is going to happen, we adjust quickly (or not) based on what does. This is pretty incredible stuff.
You're missing the forest for the trees. The less than amazing physics isn't the point.
When you have a conversation, you have an idea in your mind of how the conversation is going to go. You predict what the other person is going to say before they say it.
When you are driving, you predict what the other drivers are going to do, you predict what signs say, how long you have until the green. If you see two red lights attached to the back of a car flash on, you may predict that the car is about to slow down and you start slowing down before their speed has changed.
Forget video games and movies. I believe these models will be that inner predictive model for robots.
> How often do things in the real world not behave exactly as you expected them to?
They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works.
Your expectation is the point, not the physics. I'm not arguing that their model has an amazing grasp of physics. I'm arguing that this can be used as a predictive internal model for a robot.
I hear you, but my objection comes from the stress on "realism" in Google's post: how Genie4 is modeling the "real world" so well. If they didn't crow so much about realism, I wouldn't have cared as much.
Which is fine for a video game. Not at all fine if you want to train robots with it. Trying to combine both use cases into a single model is doomed to failure IMO.
I would not be so opinionated, modern coding agents derive from chatbot technology. Very different problem domain. They share language but language is universal so that doesn't say much. Then they figured out that the pretraining is a good starting point for RLVR. Who's to say a more cinematic imagination can't be a good starting point for a world model? It doesn't need to be the whole solution.
This is a reflection of a complaint I frequently make about so called "world models". It's more accurate to call them video game models. They don't simulate the world, they simulate video games. For that reason a common issue with them is that the physics are video game physics rather than real world physics.
I'm sure you're probably right about video games being used in training and, as a result, the models reflecting video game physics: No air resistance, no static/dynamic friction curve, etc.
However, I don't think that's the problem in the example of the marble bust not toppling. It would have toppled in a video game engine as well; they model gravity and inertia just fine.
I think what's happening is the model isn't being a world model at all, it's being a language model. It's trying to make the "scripted" thing happen, and it probably "feels" like the script should end in the bust on the pedestal.
Yes this is right. Something about the way it works under the hood means it wants: 1. bust falls 2. it crushes the watermelon 3. it teeters 4. it settles on the plinth. It is sort of interpolating between these states, which it feels are a realistic sequence of events, not actually attempting to model what would happen in a real "world".
I agree but I feel like the exaggerated physics were intentional and fitting with the video theme. If they had wanted it to topple, they wouldn't have posted the video.
People are arguing if the marble bust should've toppled over or not, completely missing the fact that marble is brittle and the bust would have surely cracked
A big chunk of the right side of castle disappeared, including catapult, when camera turned back while flying on dragon. See 1:25 to 1:29. I was not expecting this to happen in a "world model".
That entire clip looks like it's straight out of a complete bargain-bin videogame. The weird smoothness of the ride, the pivot in mid air, the wings apparently don't have collision with the poles they fly past.
Definitely wouldn't have been one of my picks to show off my fancy new model.
There's also a whole extra bonus castle that appears to the left when the camera looks left around 0:22. Which is amusing both because it wasn't there on the initial approach, and also because nobody puts two castles each with their own distinct curtain walls right next to each other like that.
The 80's Japanese train station clip also had the conductor speaking clear english in an accent I can't quite place. Slight British tinge to it but not really? Not sure. The ambient audio also makes no sense.
I've seen weird stuff in images that OpenAI sends me in it's promotional emails too. I would think that if you were trying to show off you'd make sure there's none of these obvious AI issues in what you choose to show off with. Maybe they don't really care? "Gemini choose 10 scenes that would be impressive for our announcement and add them to the post."
The oddest thing about this, to me at least, is that they pick that as their showcase. Did they themselves not notice? Or did they think their potential customers wouldn’t? Not sure really which of the two is worse.
Gotta love the kraken destroying the city in the background of the Hong Kong lovers motorcycle scene despite nothing in the prompt about monsters. Filmmakers are still safe.
But remember, today is the worst the models will ever be. I agree they are still safe though... they will probably use AI to make even more films faster and more creative since it enables them to do more.
It's hard for me to understand Google's redefinition of "world models" (basically: a txt2videogame modality for LLMs) as anything other than an attempt to bury Yann Lecun's definition of "world models", which includes an implicit bear case against LLMs and thus Google's stock price.
Looks cool, but it's very hard to get me excited about yet another proprietary model for which absolutely zero technical details are released (not even the number of parameters), which is safetymaxxed to the limit, and whose usage quota will (most likely) be so limited that it'll be effectively useless for anyone who doesn't work at Google or doesn't have infinite money.
It's just a lame gimmick, this is not 2024 anymore, people have started to get sick of this AI thingie, especially when it comes to video/image generation. That Simon guy with his pelican is the exception that can only be seen on tech forums like this one.
As to why US (and probably also some Chinese) companies have started pouring hundreds of billions into AI-image generation is beyond my comprehension, but on the other hand I'm no economic genius.
If I can’t realistically use it then I don’t care, especially when there’s no independent proof yet it works this well, given a cursory glance. If it trickles down into future developments I can use then cool, but it likely won’t because they want a moat. You don’t have to shield the poor defenseless multi-billion dollar megacorp from the genuine apathy they inspire.
If this is the best “frontier” they can achieve, then it’s no wonder they haven’t done anything groundbreaking in the space yet that produces practical results.
You're damn right if you think quota for this model will be reached after two seconds. It's for the investors (and friends and xooglers) to make real use of this, not for you pleb.
I’ve decide that the best (and maybe only) way for me to interact with google AI models, is on the search results
It’s where I get the most value, and my expectations are very low, no attachment if it goes away
Of course they are mining me for data, but they were doing that before anyway (and I do have a Gmail account, although I’ve almost completely stopped using Chrome - btw, it’s crazy that you can’t add plugins/extensions to Chrome on Guest mode)
What is this for? I cannot understand these models. They are very much polluting the internet with nonsense and use capacity that people and businesses need for actual work.
If these are the cause of why I have to pay 2-4x I used to for a computer, then Google should go fuck themselves.
Google aren't the only ones with this tech. You can try out a 24/7 rolling set of video feeds for yourself at https://fal.live
An LLM generates four alternative prompts for what could happen next and there's a real time vote running for each option. It's trippy, like watching someone's dream.
449 comments
[ 0.16 ms ] story [ 130 ms ] threadSo, Stadia + World Models = New generation of DRM for games.
Ai generated games suck even more, no skill; no taste, there is no TLC put into them
Then there is the economics of it, cost prohibitive to (re)generate a scene and ship that over internet pipes
There are so many options for games now that just sticking with the biggest AAA companies seems like a waste.
We'll be riding the huge wave of indie titles that are incoming.
The problem will be separating the wheat (people with good stories and concepts enabled by Ai) from the chaff (people with no taste slopping out trash with Ai)
I mean - reviewers!
been checking out the archives lately, nothing recent excites me
I also love thrifting and artifact hunting.
Also, lmao, they're already trying to nickel and dime you and sell you microtransactions that took 5 minutes to make for 10 bucks. If you think they're going to pay the costs for running a world model for every single person, lol.
Do you have even the slightest idea of the level of expectation that even the most casual gamer has?
This isn’t achieving much beyond prototyping. They’re burning so much money for no reason.
The original Genie was a great piece of work with a real sense of purpose - to enable agents to learn about the real world in the way Dreamer or Schmidhuber's paper sketched. But now most of the people that drove that mission have left Deepmind (Jack Parker-Holder, Tim Rocktaschel, Demis) and it's become this instead.
> Genie form [sic] can simulate scenarios
It's kind of amazing: the juxtaposition of such advanced generative AI and such terrible speech-to-text coming from the same parent company.
Real nice prompting there, Google. It also still looks like a video game rendering.
The painting->interactive scene looks astonishing though
And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation.
So we get to a point and people keep making complicated simulations of one kind or another ... and then someone who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point.
If it's the latter, I don't see a 'hole'.
I feeling I get from this is that this could be the mental model for a robot in the real world, continuously corrected by actual events.
We all have a world model in our heads of what we think is going to happen, we adjust quickly (or not) based on what does. This is pretty incredible stuff.
As an aside, why fall into the role of being an apologist for google-hype and Big Tech?
When you have a conversation, you have an idea in your mind of how the conversation is going to go. You predict what the other person is going to say before they say it.
When you are driving, you predict what the other drivers are going to do, you predict what signs say, how long you have until the green. If you see two red lights attached to the back of a car flash on, you may predict that the car is about to slow down and you start slowing down before their speed has changed.
Forget video games and movies. I believe these models will be that inner predictive model for robots.
They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works.
However, I don't think that's the problem in the example of the marble bust not toppling. It would have toppled in a video game engine as well; they model gravity and inertia just fine.
I think what's happening is the model isn't being a world model at all, it's being a language model. It's trying to make the "scripted" thing happen, and it probably "feels" like the script should end in the bust on the pedestal.
Question I have is when will they be able to render something more then 30 seconds.
And how much will it cost? There has to be a HUGE amount of computing power behind this. As a video game developer I live in fear and anticipation.
Definitely wouldn't have been one of my picks to show off my fancy new model.
"Which one!?"
I've seen weird stuff in images that OpenAI sends me in it's promotional emails too. I would think that if you were trying to show off you'd make sure there's none of these obvious AI issues in what you choose to show off with. Maybe they don't really care? "Gemini choose 10 scenes that would be impressive for our announcement and add them to the post."
As he passes by the tower, it becomes round, by 0:40
Would be fun to muck around with world models. Are there any that I can try for under £200 p/m without being part of a special project?
I’ve also wondered what they mean by “world model”, whereas I interpret them as being about predicting outcomes - something an LLM cannot reliably do.
Is this new model an LLM or a predictive outcome model (as you say a la Yann Lecun), or some hybrid?
It's just a lame gimmick, this is not 2024 anymore, people have started to get sick of this AI thingie, especially when it comes to video/image generation. That Simon guy with his pelican is the exception that can only be seen on tech forums like this one.
As to why US (and probably also some Chinese) companies have started pouring hundreds of billions into AI-image generation is beyond my comprehension, but on the other hand I'm no economic genius.
If this is the best “frontier” they can achieve, then it’s no wonder they haven’t done anything groundbreaking in the space yet that produces practical results.
I feel refusal'd just *looking* at this page.
It’s where I get the most value, and my expectations are very low, no attachment if it goes away
Of course they are mining me for data, but they were doing that before anyway (and I do have a Gmail account, although I’ve almost completely stopped using Chrome - btw, it’s crazy that you can’t add plugins/extensions to Chrome on Guest mode)
If these are the cause of why I have to pay 2-4x I used to for a computer, then Google should go fuck themselves.
An LLM generates four alternative prompts for what could happen next and there's a real time vote running for each option. It's trippy, like watching someone's dream.