Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective (arxiv.org)

> We propose adopting diffusion language models for text embeddings, motivated by their inherent bidirectional architecture and recent success in matching or surpassing LLMs especially on reasoning task.

I didn't realize diffusion language models were at this point yet. But what's the catch? why aren't diffusion models (or some kind of hybrid) taking over?