APIEval-20: A Benchmark for Black-Box API Test Suite Generation (huggingface.co)

Nice to see a benchmark in this space especially with black-box constraints.

curious.. let me see if this works for our internal setup

like that the scoring bias is toward bug detection & not test generation only. generating lots of tests with AI is easy but that doesn't necessarily mean they're good