Václav here from the team, we're happy to answer questions :)
The most surprising part to me is the auto-recovery behavior we mention at the end of the blog post, since any other model I've seen always stays diverged once it goes off the rails once. But MIRA really doesn't like to be out-of-distribution. To be completely honest we're not entirely sure why this happens.
Great work! My question is about what you mentioned at the end - how well do world models operate when out of distribution? In some sense we hope these models learn something "deeper" about how the world works and can apply that knowledge to different tasks.
I saw lots of awesome ablations in the paper (loved it!), but I'm curious if you analyzed the latents to get an intuition for what the model actually learned. Or, is it just that it learned the training data distribution really, really well?
you should do that too! the goal is not to replace the game but to foster research on these method, and hopefully apply them to data-constrained settings like robotics
Wow! At first, I expected this to be a demonstration of an AI playing rocket league, but I rapidly realized this is actually a model simulating rocket league. Wild! It feels just like the real game.
A lot actually, since the model has all information given to it in the four views, it doesn't have to deal with any "theory of mind" of modeling the other players or being consistent over long times. See [1], there's a video of a single-player model where a car disappears behind a ball and never reappears.
Multiplayer has its own risks such as the four views desynchronizing, but overall it gives a big boost to the model.
12 comments
[ 3.0 ms ] story [ 16.7 ms ] threadI saw lots of awesome ablations in the paper (loved it!), but I'm curious if you analyzed the latents to get an intuition for what the model actually learned. Or, is it just that it learned the training data distribution really, really well?
[1] https://mira-wm.com/blog-post/#hidden-information
Mira was trained on 10k hours of Rocket League data. The model has 5B parameters and runs 4-player games at 20 fps on a single B200 GPU.
We've released a playable online demo, an in-depth technical report as well as a 1k hour dataset of 4-players gameplay:
Technical report: https://mira-wm.com/paper Repo: https://github.com/mira-wm/mira