JevBench: Benchmarking decision models (Clef, pplx-decider and 15 more) (leanmcp.github.io) 1 points by paidev 18d ago ↗ HN
[–] verdverm 18d ago ↗ Thoughts on adding benchmarks like coin flipping, dice rolls, and other well known outcome questions?Some of the decision models do really poorly on this with the outcome probability and being very confident about the decision.
[–] fignews 16d ago ↗ Very cool resource! I was wondering why clef-omni isn't part of the picture tests?
3 comments
[ 3.7 ms ] story [ 17.8 ms ] threadSome of the decision models do really poorly on this with the outcome probability and being very confident about the decision.