Originally, it took more than 1 GPU-week to train a single static scene. After multiple rounds of algorithmic optimizations, it's now practical to train & render whole city blocks, as seen in the Waymo demo.
The best part of it, is the answer to the classic question: "how many polygons?". Because the answer is zero.
So this is a speculative side project that isn't integrated into the main flow of work going on at Waymo. That suggests to me that there isn't anything as technical and forward thinking as this going on in the core teams. If they did they would show it off because clearly anything flashy they are quite eager to show off.
They appear to be building trains with wheels, pretty uninspiring.
The intern in question is Matthew Tancik, who is one of the primary driving forces behind Neural Radiance Fields. It is one of those cases, where the employer (Waymo) is lucky to have such an intern.
I can't comment on the second part of the message, because I don't know what I am allowed to say (I work for Google, but not Waymo).
As a PhD student alongside Matt, it’s not any intern. Ph.D. internships are much more scoped to specific lines of work and are a way to potentially fund promising researchers for the long term.
In fact, Berkeley AI Research has a built-in way for companies to fund BAIR research and to bring on BAIR Ph.D. students as interns: https://bcommons.berkeley.edu/home
NeRF is one of the big "holy sh*t" moments for me in recent years. I've always thought using polygons as our only way to represent non-trivial 3d shapes felt awfully inelegant, like using raster images instead of vector graphics.
Definitely recommend checking out the HN discussion of the original NeRF paper[0]
> If you give it a bunch of photos of a scene from different angles, this machine learning method lets you see angles that did not exist in the original set.
Imagine matching the output of the block nerf + GPS data combined with a live camera view to get ultra-high positioning precision...
Or to use changes over time in 2 views to detect anomalies or maintenance heuristics...
So many potentially interesting use cases
If NERF interest you, don't miss NVidia last month paper, Instant Neural Graphics Primitive : https://news.ycombinator.com/item?id=29959700 which helps bridge the gap between traditional voxel maps, and neural network representation.
12 comments
[ 3.4 ms ] story [ 41.6 ms ] threadOriginally, it took more than 1 GPU-week to train a single static scene. After multiple rounds of algorithmic optimizations, it's now practical to train & render whole city blocks, as seen in the Waymo demo.
The best part of it, is the answer to the classic question: "how many polygons?". Because the answer is zero.
https://nvlabs.github.io/instant-ngp/
So this is a speculative side project that isn't integrated into the main flow of work going on at Waymo. That suggests to me that there isn't anything as technical and forward thinking as this going on in the core teams. If they did they would show it off because clearly anything flashy they are quite eager to show off.
They appear to be building trains with wheels, pretty uninspiring.
I can't comment on the second part of the message, because I don't know what I am allowed to say (I work for Google, but not Waymo).
In fact, Berkeley AI Research has a built-in way for companies to fund BAIR research and to bring on BAIR Ph.D. students as interns: https://bcommons.berkeley.edu/home
NeRF is one of the big "holy sh*t" moments for me in recent years. I've always thought using polygons as our only way to represent non-trivial 3d shapes felt awfully inelegant, like using raster images instead of vector graphics.
Definitely recommend checking out the HN discussion of the original NeRF paper[0]
[0] https://news.ycombinator.com/item?id=22637721
Yeah wasn't sure what I was seeing.
Is this too much to ask for? :/
(I spent about a year writing an optimiser for camera localisation from multiple views)
UPD: found in arxiv: 0.25M parameters (e.g. ~1MB) per block; demo video used 16 blocks, so 16MB total for Mission Bay.