Not to belittle this or anything (it does look good and show promise), it feels like they somehow generate several consistent (but discrete) views of a given world, then feed all that to the good old pose estimation + gaussian splatting workflow. Whenever you leave the generated area (which isn't exactly huge on the few I tested) you get tell-tale signs of GS.
It's amazing to see how this space is developing. About 7 years ago I was building "spatial media" with https://ayvri.com
Nobody believed us when we said AI would create 3D virtual worlds that were indistinguishable from the real thing, and we'd be able to transport people to different places.
I particularly like the artistic effect of the drawing that brings the person into this world. Like a point-cloud that then gets "filled in".
I have little doubt this was a design decision and I think it is very well executed.
Something about the camera perspective creates a skew that makes things feel artificial to me. It's a minor thing that bothers me, but I'd like the geometry to feel more like what I normally see. Video generation models tend to feel more natural in perspective.
I'm looking forward to the future of games and movies if these world models keep improving. Imagine if anyone with an interesting idea could sketch it, plug it into a world model and share the result with everyone. It'd open up a huge amount of possibilities.
Not to mention being able to explore worlds from already existing works. Care to go for a ride on a broomstick? How about simply walking into Mordor? It's exciting.
OK, so I've talked about this phenomenon with ChatGPT, and I think that the issue here is that to a lot of people, a song needs to be more than just a "song". There's some sort of requirement for it to be the un-faked result of having certain experiences. It has to relate to something happening in reality, and to be derived from it, and cannot exist in a vacuum separated from the rest of reality. Otherwise to them, the music isn't "real".
It's hard to appreciate how magical this is until you get beat up by traditional photogrammetry-based set acquisition or trying to build it from scratch in 3D.
In a couple of years you are going to see high school drama groups everywhere staging film productions with this tech.
12 comments
[ 3.3 ms ] story [ 33.0 ms ] threadNobody believed us when we said AI would create 3D virtual worlds that were indistinguishable from the real thing, and we'd be able to transport people to different places.
I particularly like the artistic effect of the drawing that brings the person into this world. Like a point-cloud that then gets "filled in".
I have little doubt this was a design decision and I think it is very well executed.
Not to mention being able to explore worlds from already existing works. Care to go for a ride on a broomstick? How about simply walking into Mordor? It's exciting.
OK, so I've talked about this phenomenon with ChatGPT, and I think that the issue here is that to a lot of people, a song needs to be more than just a "song". There's some sort of requirement for it to be the un-faked result of having certain experiences. It has to relate to something happening in reality, and to be derived from it, and cannot exist in a vacuum separated from the rest of reality. Otherwise to them, the music isn't "real".
There is a better Blender Gaussian splat add-on now: https://superhivemarket.com/products/splatforge
We worked with filmmaker Joshua Kerr on this project that took him back to his 8 year old zombie epic roots: https://www.worldlabs.ai/case-studies/lightcraft
It's hard to appreciate how magical this is until you get beat up by traditional photogrammetry-based set acquisition or trying to build it from scratch in 3D.
In a couple of years you are going to see high school drama groups everywhere staging film productions with this tech.