LLM Evals Are Just Tests. Why Are We Making This So Complicated? (cameronwestland.com)

So, did the tests allow you to build a system that never confused existing features with new features? That seems like the problem statement, but I think I'm only seeing probabilistic testing.