HN Explorer

17 comments

[ 5.5 ms ] story [ 54.6 ms ] thread
snyhlxde
Challenge yourself with latest reasoning LLMs and checkout our latest leaderboard!
[deleted]
(comment deleted)
zhisbug
This is pretty clever and seems to have high potential, but it still relies on humans. What if some day all humans cannot outsmart AI?
snyhlxde
When super intelligence comes, it would be very interesting to see multi-party game play among AI too. What role humans play in this story is unclear. Maybe humans can't directly engage in the games neither as they are too naive and will be immediately identified and exploited by AI :)
meru_2025
A dynamic human-in-the-loop evaluation benchmark is great for preventing data contamination and test saturation. Worth my time to read.
[deleted]
(comment deleted)
mino1234uiui27
That is an interesting perspective that I hadn't previously considered, like using games to evaluate LLM.
ginda307
Just played the game and it seems pretty fun - especially when you see the LLM speaking nonsense haha
snyhlxde
It's funny that reasoning models sometime speaking nonsense and perform worse than well-aligned models like claude-3.5-sonnet in multi-turn games like Akinator. I think it's one current weak point of applying longCoT RL vs. instruction-following alignment. Maybe we need to find a way to address both? Would be interesting to see some results
wlsaidhi
Is it possible to setup a MLLM pipeline to play other roblox games and use that as another evaluation?
snyhlxde
I think it's totally possible. Multimodal reasoning eval would be fun to consider too
Yuxuan_Zhang13
I played the game and found hard mode to be an exciting challenge—it's incredibly fun, and the AI is so clever it even guessed my intentions in the taboo game!
PY007
Static evaluation --> chatbot arena --> game arena. Seems to be promising.
leemack
Cool, actually not boring and hard to play, good gamification