First AI thing that’s made me feel a bit of derealization…
…and this is the worst the capabilities will ever be.
Watching the video created a glimmer of doubt that perhaps my current reality is a future version of myself, or some other consciousness, that’s living its life in an AI hallucinated environment.
It's an unsettling feeling as what's more complicated - all the atoms and galaxies, trillions of life forms, the unimaginable distances of our universe OR a relatively simple world model that is our conscious experience and nothing else.
No it's good, you're ahead of the curve, most people aren't there yet.
The next step is to realize that, if life is a cheap simulation, not everyone might have... uh... fully simulated minds. Player Characters vs NPCs is what gamers would say, though it doesn't have to be binary like that, and the term NPC has already been ruined by social media rants. (Also, NPC is a bad insult because most of the coolest characters in games are NPC rivals or bosses or whatnot.)
> - Physics is still hard and there are obvious failure cases when I tried the classical intuitive physics experiments from psychology (tower of blocks).
> - Social and multi-agent interactions are tricky to handle. 1vs1 combat games do not work
> - Long instruction following and simple combinatorial game logic fails (e.g. collect some points / keys etc, go to the door, unlock and so on)
> - Action space is limited
> - It is far from being a real game engines and has a long way to go but this is a clear glimpse into the future.
Even with these limitations, this is still bonkers. It suggests to me that world models may have a bigger part to play in robotics and real world AI than I realized. Future robots may learn in their dreams...
I similarly am surprised at how fast they are progressing. I wrote this piece a few months ago about how I think steering world model output is the next realm of AAA gaming:
But even when I wrote that I thought things were still a few years out. I facetiously said that Rockstar would be nerd-sniped on GTA6 by a world model, which sounded crazy a few months ago. But seeing the progress already made since GameNGen and knowing GTA6 is still a year away... maybe it will actually happen.
Reality is not composed of words, syntax, and semantics. A human modal is.
Other human modals are sensory only, no language.
So vision learning and energy models that capture the energy to achieve a visual, audio, physical robotics behavior are the only real goal.
Software is for those who read the manual with their new NES game. Where are the words inside us?
Statistical physics of energy to make machine draw the glyphs of language not opionated clustering of language that will close the keyboard and mouse input loop. We're like replicating human work habits. Those are real physical behaviors. Not just descriptions in words.
Interesting! This feels like they're trying to position it as a competitor to Nvidia's Omniverse, which is based on the Universal Scene Descriptor format as the backbone. I wonder what format world objects can be ingested into Genie in - e.g. for the manufacturing use cases mentioned.
It's hard to get an acceptable VR output for today's rendering engines still. In the examples provided, the movement seems to be slow and somewhat linear, which doesn't translate to head movements in VR. VR needs 2 consistent videos with much higher resolutions and low latency is a must. The feedback would still be very dependent on people's tolerance to all imperfections - some would be amazed, others would puke. That's why VR still isn't in the spotlight after all the years (I personally find it great).
I think VR will come at the same time they make multiplayer. There needs to be differentiation between the world-state and the viewport. Right now, I suspect they're the same.
But once you can get N cameras looking at the same world-state, you can make them N players, or a player with 2 eyes.
This is very encouraging progress, and probably what Demis was teasing [1] last month. A few speculations on technical details based on staring at the released clips:
1. You can see fine textures "jump" every 4 frames - which means they're most likely using a 4x-temporal-downscaling VAE with at least 4-frame interaction latency (unless the VAE is also control-conditional). Unfortunately I didn't see any real-time footage to confirm the latency (at one point they intercut screen recordings with "fingers on keyboard" b-roll? hmm).
2. There's some 16x16 spatial blocking during fast motion which could mean 16x16 spatial downscaling in the VAE. Combined with 1, this would mean 24x1280x720/(4x16x16) = 21,600 tokens per second, or around 1.3 million tokens per minute.
3. The first frame of each clip looks a bit sharper and less videogamey than later stationary frames, which suggests this is could be a combination of text-to-image + image-to-world system (where the t2i system is trained on general data but the i2w system is finetuned on game data with labeled controls). Noticeable in e.g. the dirt/textures in [2]. I still noticed some trend towards more contrast/saturation over time, but it's not as bad as in other autoregressive video models I've seen.
What is the purpose of this? It seems designed to muddy the waters of reality vs. falsehood and put creatives in film/tv out of jobs. Real Jurassic Park moment here
Arrogant profit seeking capitalist dick measuring that will break our societies and ours and our childrens worldviews under some pathetic label of "exploring a new scientific frontier"
Why even be sarcastic about it ? There is no human invention that has not exploded thanks (or because of) pornographic possibilities. HD-DVD vs Blueray, Internet...I'd even argue that XR is not as big as it could be because it is really clamped down to deviant usage !
I thought I was not going to see too many negative comments here, yet I was mistaken. I thought if it's not LLM, people would have a more nuanced take and could look at the research with an open mind. The examples on the website are probably cherry-picked, but progress is really nice compared to Genie 2.
It's a nice step towards gains in embodied AI. Good work, DeepMind.
I'm not sure this is interesting beyond the wow effect. Unless we can actually get the world out of the AI. The real reason chatgpt and friends actually have customers is that the text interface is actually durable and easily to build upon after generation. It's also super ez to feed text into a fresh cycle. But this, while looking fancy, doesn't seem to be on the path to actually working out. Unless there is a sane export to unreal or something.
147 comments
[ 2.7 ms ] story [ 104 ms ] threadReminds me of when image AIs weren't able to generate text. It wasn't too long until they fixed it.
I DECLARE BANKRUPTCY vibes here
…and this is the worst the capabilities will ever be.
Watching the video created a glimmer of doubt that perhaps my current reality is a future version of myself, or some other consciousness, that’s living its life in an AI hallucinated environment.
The next step is to realize that, if life is a cheap simulation, not everyone might have... uh... fully simulated minds. Player Characters vs NPCs is what gamers would say, though it doesn't have to be binary like that, and the term NPC has already been ruined by social media rants. (Also, NPC is a bad insult because most of the coolest characters in games are NPC rivals or bosses or whatnot.)
> Genie 3’s consistency is an emergent capability
So this just happened from scaling the model, rather than being a consequence of deliberate architecture changes?
Edit: here is some commentary on limitations from someone who tried it: https://x.com/tejasdkulkarni/status/1952737669894574264
> - Physics is still hard and there are obvious failure cases when I tried the classical intuitive physics experiments from psychology (tower of blocks).
> - Social and multi-agent interactions are tricky to handle. 1vs1 combat games do not work
> - Long instruction following and simple combinatorial game logic fails (e.g. collect some points / keys etc, go to the door, unlock and so on)
> - Action space is limited
> - It is far from being a real game engines and has a long way to go but this is a clear glimpse into the future.
Even with these limitations, this is still bonkers. It suggests to me that world models may have a bigger part to play in robotics and real world AI than I realized. Future robots may learn in their dreams...
https://kylekukshtel.com/diffusion-aaa-gamedev-doom-minecraf...
But even when I wrote that I thought things were still a few years out. I facetiously said that Rockstar would be nerd-sniped on GTA6 by a world model, which sounded crazy a few months ago. But seeing the progress already made since GameNGen and knowing GTA6 is still a year away... maybe it will actually happen.
Not for video games it isn’t.
Reality is not composed of words, syntax, and semantics. A human modal is.
Other human modals are sensory only, no language.
So vision learning and energy models that capture the energy to achieve a visual, audio, physical robotics behavior are the only real goal.
Software is for those who read the manual with their new NES game. Where are the words inside us?
Statistical physics of energy to make machine draw the glyphs of language not opionated clustering of language that will close the keyboard and mouse input loop. We're like replicating human work habits. Those are real physical behaviors. Not just descriptions in words.
https://www.theguardian.com/technology/2025/aug/05/google-st...
Gemini Robot launch 4 mo ago:
https://news.ycombinator.com/item?id=43344082
good writers will remain scarce though.
maybe we will have personalized movies written entirely through A.I
That's an insane product right there just waiting to happen. Too bad Google sleeps so hard on the tech they create.
But once you can get N cameras looking at the same world-state, you can make them N players, or a player with 2 eyes.
Really great work though, impressive to see.
1. You can see fine textures "jump" every 4 frames - which means they're most likely using a 4x-temporal-downscaling VAE with at least 4-frame interaction latency (unless the VAE is also control-conditional). Unfortunately I didn't see any real-time footage to confirm the latency (at one point they intercut screen recordings with "fingers on keyboard" b-roll? hmm).
2. There's some 16x16 spatial blocking during fast motion which could mean 16x16 spatial downscaling in the VAE. Combined with 1, this would mean 24x1280x720/(4x16x16) = 21,600 tokens per second, or around 1.3 million tokens per minute.
3. The first frame of each clip looks a bit sharper and less videogamey than later stationary frames, which suggests this is could be a combination of text-to-image + image-to-world system (where the t2i system is trained on general data but the i2w system is finetuned on game data with labeled controls). Noticeable in e.g. the dirt/textures in [2]. I still noticed some trend towards more contrast/saturation over time, but it's not as bad as in other autoregressive video models I've seen.
[1] https://x.com/demishassabis/status/1940248521111961988
[2] https://deepmind.google/api/blob/website/media/genie_environ...
/s
It's a nice step towards gains in embodied AI. Good work, DeepMind.