Ask HN: What are the recent most influential Deep Learning advancements?

What are some of the recent transformative ideas in deep learning and where to find them? My only source so far is Arxiv Sanity Preserver [0].

The latest advancements I know are attention mechanisms and transformers in CNNs. What are some of the newer/better ones?

[0] https://arxiv-sanity.com/