Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings (lmsys.org) 50 points by MMMercy2 3y ago ↗ HN
[–] HoshinoAI 3y ago ↗ It's glad to see the old technique is used for new models.You may also learn from 7.33 dota update which uses a new ranking algorithm called Glicko. [–] zhisbug 3y ago ↗ could you provide any reference? Is it a variant of ELO? [–] HoshinoAI 3y ago ↗ check matchmaking section: https://www.dota2.com/newfrontiersValve listed some reason for making the change.https://en.wikipedia.org/wiki/Glicko_rating_system [–] bradknowles 3y ago ↗ Elo != ELO.One is a rating system named after the creator, Arpad Elo. See https://en.wikipedia.org/wiki/Elo_rating_systemThe other is a rock band that was formed in 1970. See https://en.wikipedia.org/wiki/Electric_Light_Orchestra
[–] zhisbug 3y ago ↗ could you provide any reference? Is it a variant of ELO? [–] HoshinoAI 3y ago ↗ check matchmaking section: https://www.dota2.com/newfrontiersValve listed some reason for making the change.https://en.wikipedia.org/wiki/Glicko_rating_system [–] bradknowles 3y ago ↗ Elo != ELO.One is a rating system named after the creator, Arpad Elo. See https://en.wikipedia.org/wiki/Elo_rating_systemThe other is a rock band that was formed in 1970. See https://en.wikipedia.org/wiki/Electric_Light_Orchestra
[–] HoshinoAI 3y ago ↗ check matchmaking section: https://www.dota2.com/newfrontiersValve listed some reason for making the change.https://en.wikipedia.org/wiki/Glicko_rating_system
[–] bradknowles 3y ago ↗ Elo != ELO.One is a rating system named after the creator, Arpad Elo. See https://en.wikipedia.org/wiki/Elo_rating_systemThe other is a rock band that was formed in 1970. See https://en.wikipedia.org/wiki/Electric_Light_Orchestra
[–] weichiang 3y ago ↗ Surprised to learn StableLM is worse than plain LLaMA. link to their leaderboard: leaderboard.lmsys.org [–] circuit10 3y ago ↗ I’ve heard that it’s really bad for it’s size
9 comments
[ 5.3 ms ] story [ 33.9 ms ] threadYou may also learn from 7.33 dota update which uses a new ranking algorithm called Glicko.
Valve listed some reason for making the change.
https://en.wikipedia.org/wiki/Glicko_rating_system
One is a rating system named after the creator, Arpad Elo. See https://en.wikipedia.org/wiki/Elo_rating_system
The other is a rock band that was formed in 1970. See https://en.wikipedia.org/wiki/Electric_Light_Orchestra