HN Explorer

Show HN: AI Roundtable – Let 200 models debate your question (opper.ai)

118 points by felix089 ↗ HN
Hey HN! After the Car Wash Test post got quite a big discussion going (400+ comments, https://news.ycombinator.com/item?id=47128138), I spent the past few weeks building a tool so anyone can run these kinds of questions and get structured results. No signup and free to use.

You type a question, define answer options, pick up to 50 models at a time from a pool of 200+, and they all answer independently under identical conditions. No system prompt, structured output, same setup for every model.

You can also run a debate round where models see each other's reasoning and get a chance to change their minds. A reviewer model then summarizes the full transcript. All models are routed via my startup Opper. Any feedback is welcome!

Hope you enjoy it, and would love to hear what you think!

72 comments

[ 3.0 ms ] story [ 99.0 ms ] thread
infosecphoenix
this is very interesting! I wonder if we need that many models to join the discussion. Have you tried fewer models?
gsandahl
Oh lord, imagine asking ”serious” questions

https://opper.ai/ai-roundtable/questions/you-are-standing-in...

sdwr
Great question! Clean separation between Gemini Pro and the other answers
zipping1549
> However, a clever minority led by Gemini 3.1 Pro and Gemini 3 Pro argued that if the sign is legible from the other side, it must be intended to lead people into the current room to find the exit, making the inscribed corridor the one leading deeper into the dungeon.

This is quite impressive, really.

rob74
A dungeon with glass doors and emergency exit signs? In that case, I can imagine at least two alternative scenarios:

- "↑TIX∃" is not a mirror image of "EXIT", but some dwarven runes that mean something else entirely.

- The sign might be a ruse meant to lure you into a trap.

If you look at the detailed answers, some of the models have similar answers (e.g. Nemotron Nano 12B: "Suspicious of dungeon riddles, viewing the inscription as a potential trap or red herring."), but I'm not sure it's because they identified the word EXIT and thought it might be misleading, or because they didn't understand it...

cdnsteve
Cool project! This is also extremely useful to compare model bias across the board. There are some disturbing trends on certain topics.
Ancalagon
Love this. I asked about climate change cause that's been on my mind lately. Looks to be very split among the models.
tonymet
great tool! I found it useful for challenging "lies my teacher told me".

It would be nice to support collections of claims, with a table of summaries. I would love to list out a few dozen phony concepts from school, and have a sharable chart of the rejections, that expand.

I really like the UI. It's nice to read the expanded results.

But how do you afford the tokens?

jacquesm
Great idea. I'd love for there to be an 'open ended answer' without giving multiple choice options. Like this they are not debating the question itself but the validity of the possible answers and the real answer to the question may not be contained within that set because the person asking is unaware of that option.
felix089
Hey just fyi the open question feature is now live. Also gave the UI a facelift. Any feedback welcome! Also got a custom domain for easy access: https://askroundtable.ai
soared
Really cool! Surprising amount of value to seeing the models debate and disagree, I wish I had this at work to have models argue over whether the documentation they provided me are accurate.

I would like to see a devils advocate - it seems some of the models kind of repeat the same ideas rather than considering incorrect ideas.

chabes
Been enjoying playing with this.

It would be cool if the human user could be a participant in the debate, getting a vote and the chance to state their reasoning.

mizzao
It would be amazing to be able to ask open-ended questions without having to specify the answers in advance.
schrepa
reminds me of karpathy's LLM Council, I use variation of this in my workflow where I pass their opinions back and forth to various models until they achieve some sort of consensus
lim8603
I used to copy and paste the same prompt into Obsidian every time, then run it on two or three different AI models to compare the results. It’s really interesting to have it turned into a website like this.
oezi
I think Stackoverflow.com should have pivoted to something similar. Let AIs both pose, answer and vote on questions and answers.