I wanted to explore how useful transformers would be in a typical text classification task. I found that while they are great out of the box and easy to work with, they are too slow for a production situation you would commonly find at a company.

I can't imagine deploying a transformer to label millions of items unless that labeling had massive revenue attached to it. They are simply too slow and resource intensive.