Track and visualize LLM model performance over time (github.com)

this is a really neat project: "an automated, daily evaluation suite to track model performance over time, monitor for regression during peak load periods, and detect quality changes across flagship LLM APIs."