Transformers seem great. IMO the big challenge is still to better understand how they perform in operation.

The fundamental architectures of transformers haven't changed in years. I wonder what kind of design changes will be made in the future and how that impacts these results.

Interesting, thanks for sharing! Somehow I'm not surprised. My experience building systems for real world applications is that choosing a simple CNN is usually the way to go, and it's all in the data. Choosing more complex model classes also comes at a cost, higher variance and more likelihood of issues related to robustness and generalization.

Interesting, but what about the more recent YOLO versions? Do you have any research on if they encounter the same issues?