Ask HN: How do you know when your AI agent's output quality degrades?

Running an AI agent in production and curious how others handle this. Cost and latency monitoring is solved (Helicone, Langfuse, etc.) but I can't find a good answer for: how do you get alerted when the quality of your agent's responses drops before a customer tells you?

What are you doing today?