The differences are pretty big, but the simplest way to illustrate is to try to use gazebo, isaac...etc, and then try to build a whole physically interactive kitchen.
First off, it's gonna take you 3 months to author that thing, if you don't ragequit along the way.
Then, when you go to run, your "50 million steps per second" sim becomes 500 steps per second.
The reason we have robots doing backflips and acrobatics instead of actually useful stuff like picking up your house is making the scenes and getting the data is tough. It requires sensors like cameras and rendering, vs purely proprioceptive-only envs with a flat ground plane and no other physics interactions.
Right now, the industry is doing manual teleop to collect data because it's straight up easier than trying to build these sorts of things in simulators.
This (and a global pandemic) are central to the plot of The Talos Principle, a 2014 puzzle game. Can't say much more without spoilers, but we'd better hope its other predictions for the future don't come true.
I was the Principal architect for ML-Agents at Unity for a while and this looks like it could be a more elegant version of what we were doing (cause it’s not a sidecar to an engine).
I’m going to try it to see if I can make my Go-1 edu do some work around the house finally
This is the company that Yan Chernikov - aka TheCherno has cofounded. This engine is based on the engine (Hazel) he built as part of his game engine series on YouTube. If you're interested there is probably at least 100 hours of devlogs about this engine which is kind of cool.
Is there a danger of overfitting if you train something on a physics sim? How do you prevent the model to exploit the differences to real life? Surely there are some numerical errors and other idealizations that result in some stuff being a good solution but not working in real life, or is the sim that accurate?
11 comments
[ 3.9 ms ] story [ 37.4 ms ] threadThe differences are pretty big, but the simplest way to illustrate is to try to use gazebo, isaac...etc, and then try to build a whole physically interactive kitchen.
First off, it's gonna take you 3 months to author that thing, if you don't ragequit along the way.
Then, when you go to run, your "50 million steps per second" sim becomes 500 steps per second.
The reason we have robots doing backflips and acrobatics instead of actually useful stuff like picking up your house is making the scenes and getting the data is tough. It requires sensors like cameras and rendering, vs purely proprioceptive-only envs with a flat ground plane and no other physics interactions.
Right now, the industry is doing manual teleop to collect data because it's straight up easier than trying to build these sorts of things in simulators.
This is why we're building Lucky.
I’m going to try it to see if I can make my Go-1 edu do some work around the house finally
I guess the better question is how much does photo-realism quality matter for this kind of sim2real work? A lot I would wager.
Hasn’t the Bullet physics engine been used for robotics for over a decade?
I don’t understand this “first game engine for robotics” messaging.
As an aside, this website crashes for me on safari on iOS.
https://www.youtube.com/@TheCherno
[0] https://developer.nvidia.com/isaac