Magma: A GPT-style multimodal model for any combination of image and text (github.com)

MAGMA semantically understands images beyond traditional object classification and can, for exmaple, be used to automatically generate image captions for arbitrary images. Some of these results are really astonishing and surprisingly precise.