Things like this are often not "a choice", as in twenty engineers did not sit in a room and discuss the merits and drawbacks of each format. Rather, they are "a doing", as in the intern tasked with scanning the object used the first tool that appeared in his DDG search.
These are just standard OBJ files with diffuse textures only and trivial material settings (all white).
Additional there is some metadata that specifies that the same data is to be used as displayable model and collision geometry.
So, essentially, this data is highly portable, photoreal aesthetics is not really a goal.
(Also see some notes in the paper, e.g. "Limitations of the Dataset".)
Removing lighting before object recognition may be a separate step, yes.
But in this case it's just that their goal is to scan objects, shapes as such, and not intrinsic properties of their constituent materials (such as specularity).
I think it's great to see Gazebo acknowledged here. I've often wondered why it's not more often considered for the basis of RL systems, as it is designed for robotic simulation, and supports multiple physics engines.
How is this novel? There are far higher quality object scans available on sites like Sketchfab and Turbosquid, not to mention free assets on the Unity and Unreal asset stores.
And possibly even more perplexing: why is it presented as an academic paper?
It doesn’t claim to be a breakthrough - it’s just a set of assets where Google has set a minimum quality bar, you’d have to evaluate stuff on Sketchfab for quality if you wanted to use it.
I guess they want people to be able to cite it, which is why they wrote a paper.
Also, many people still use ShapeNet (for example for pre-training optical flow algorithms before the self-supervised stage) and this is an obvious quality improvement over that, albeit with much fewer models.
The thing to know about 3D scanning -- and more generally, automated photorealistic 3D modeling of the entire physical world -- is that it is an unsolved problem. A lot of the 3D scans you'll find were created with techniques (e.g. photogrammetry) that do not scale to a majority of the physical world.
The concern with assembling datasets for ML from e.g. Sketchfab and Turbosquid, besides typical copyright restrictions, is very high variability in 3D model quality. I love that Epic Games indeed seems to be very excited about 3D scanning (acquiring several companies), and their Unreal Engine does offer a large number of free 3D scans. But the scope of their publication so far is just a drop in the ocean of possibility for 3D computer vision.
Even if I may disagree with and see further opportunities in the techniques employed for 3D scanning, I commend the Google authors on their hard work over multiple years collecting massive datasets, their detailed academic communication, and their open source licensing. In my view, anyone working on and communicating new techniques for 3D scanning/modeling/vision like this, and especially anyone who is publishing large numbers of 3D models for the world, is contributing to a much bigger future, where we can have the entire world fully 3D modeled.
Sure, I'll bite. I like your thrust and conclusion, even if I don't agree with how you got there.
I've worked in the 3D scanning world for almost a decade, so my bias is really just surprise that Google produced outcomes that were far below the quality I hoped for and anticipated, given how great so many other conceptually similar (and also free) assets are.
Here's the thing: regardless of the purity and nobility of the goal or the quality of their academic paper, at the end of the day, the assets are either great or they are not. I believe that it's limiting to assign extra quality marks to something just because the folks who made it are "doing it right" in an academic context.
I respect folks who volunteer to do beach cleanups, too. It's absolutely positive, from the immediate impact, the educational perspective, getting folks involved. It's great exercise! But it's also tragically short-term, and if they wrote an academic white paper about how they picked up garbage after the fact, it wouldn't change the fundamentals.
If you saw an Arxiv paper on a particular 30 volunteer beach cleanup on one Saturday, you might be forgiven for wondering why they wrote it up as an academic paper, too.
I wish they did that with the Computer History Museum's collections. Or the Living Computer Museum in Seattle. Maybe even paying them to help them continuing to exist.
Extra bonus for doing this with the disassembled parts during restoration so everyone can 3D print their own Cray 2 aquariums.
27 comments
[ 859 ms ] story [ 767 ms ] threadhttps://app.ignitionrobotics.org/GoogleResearch/fuel/collect...
Interesting choice.
The most unreasonable thing in your comment and that's saying a lot
The pictures of the scanned objects are put on the 3D model.
So, essentially, this data is highly portable, photoreal aesthetics is not really a goal. (Also see some notes in the paper, e.g. "Limitations of the Dataset".)
How is this novel? There are far higher quality object scans available on sites like Sketchfab and Turbosquid, not to mention free assets on the Unity and Unreal asset stores.
And possibly even more perplexing: why is it presented as an academic paper?
Also, many people still use ShapeNet (for example for pre-training optical flow algorithms before the self-supervised stage) and this is an obvious quality improvement over that, albeit with much fewer models.
The concern with assembling datasets for ML from e.g. Sketchfab and Turbosquid, besides typical copyright restrictions, is very high variability in 3D model quality. I love that Epic Games indeed seems to be very excited about 3D scanning (acquiring several companies), and their Unreal Engine does offer a large number of free 3D scans. But the scope of their publication so far is just a drop in the ocean of possibility for 3D computer vision.
Even if I may disagree with and see further opportunities in the techniques employed for 3D scanning, I commend the Google authors on their hard work over multiple years collecting massive datasets, their detailed academic communication, and their open source licensing. In my view, anyone working on and communicating new techniques for 3D scanning/modeling/vision like this, and especially anyone who is publishing large numbers of 3D models for the world, is contributing to a much bigger future, where we can have the entire world fully 3D modeled.
I've worked in the 3D scanning world for almost a decade, so my bias is really just surprise that Google produced outcomes that were far below the quality I hoped for and anticipated, given how great so many other conceptually similar (and also free) assets are.
Here's the thing: regardless of the purity and nobility of the goal or the quality of their academic paper, at the end of the day, the assets are either great or they are not. I believe that it's limiting to assign extra quality marks to something just because the folks who made it are "doing it right" in an academic context.
I respect folks who volunteer to do beach cleanups, too. It's absolutely positive, from the immediate impact, the educational perspective, getting folks involved. It's great exercise! But it's also tragically short-term, and if they wrote an academic white paper about how they picked up garbage after the fact, it wouldn't change the fundamentals.
If you saw an Arxiv paper on a particular 30 volunteer beach cleanup on one Saturday, you might be forgiven for wondering why they wrote it up as an academic paper, too.
Extra bonus for doing this with the disassembled parts during restoration so everyone can 3D print their own Cray 2 aquariums.