We implemented an AI-powered customer support triage system that initially looked promising in testing. In production, it actually increased our support costs by ~30% because:
The AI would confidently misroute 15-20% of tickets, requiring human review of ALL AI decisions
and the Customers lost trust after a few bad experiences and started explicitly requesting human agents
also Support agents spent more time correcting AI mistakes than they saved
The breaking point was data quality - our training data was too clean compared to real customer queries. We ended up rolling back to rule-based routing with AI as an optional suggestion tool instead.
1 comment
[ 3.2 ms ] story [ 11.8 ms ] threadThe AI would confidently misroute 15-20% of tickets, requiring human review of ALL AI decisions and the Customers lost trust after a few bad experiences and started explicitly requesting human agents also Support agents spent more time correcting AI mistakes than they saved
The breaking point was data quality - our training data was too clean compared to real customer queries. We ended up rolling back to rule-based routing with AI as an optional suggestion tool instead.