ImageBind: Holistic AI learning across six modalities (ai.facebook.com)

The 6 modalities that the model are images, text, audio, depth, thermal, and motion data. Thermal and motion data completely caught me by surprise.

Huge step towards laying down the framework for AIs that understand the world around them like humans do.

Incredible work by Meta AI, and glad they're open sourcing their work (code and weights https://github.com/facebookresearch/ImageBind)

Code: https://github.com/facebookresearch/ImageBind

Blog Post: https://ai.facebook.com/blog/imagebind-six-modalities-bindin...

Demo: https://imagebind.metademolab.com/

Ah great the last missing link in making killbots at scale.

Do these things use/incorporate a physics model? It seems like the missing link to me.