38 comments

[ 2.4 ms ] story [ 49.5 ms ] thread
Do the players (LLMs) have memory of how prior hands were played by their opponents, or know their VPIP and PFR percentages? Or is each hand stateless?
I suspect this would only matter much if they also remembered (and cared about) their own prior play.
I'm not an expert, but as I understand it there are existing solvers for poker/holdem? Perhaps one of the players could be a traditional solver to see how the LLMs fare against those?
the LLMs would get crushed
The solvers don't typically work in real time, I don't think. They take a while to crunch a hand.
While others have commented about solvers, I'd also like to bring up AI poker bots such as Pluribus (https://en.wikipedia.org/wiki/Pluribus_(poker_bot)).

This also wouldn't even be a close contest, I think Pluribus demonstrated a solid win rate against professional players in a test.

As I was developing this project, a main thought came to mind as to the comparison between cost and performance between a "purpose" built AI such as Pluribus versus a general LLM model. I think Pluribus training costs ~$144 in cloud computing credits.

Cool idea. I tried to create a room but it says limit reached for today.
If you are interested in this space, you can check out NovaSolver.com

It's mostly a ChatGPT conversational interface over a classic Solver (Monte-Carlo simulation based), but that ease of use makes it very convenient for quick post-game analysis of hands.

I'm sure if you hook a Solver to a hud, it might be even simpler, but it's quite burdensome for amateurs, and it might be too close to cheating.

This is amazing, I just wish I could pause the game and have them play step by step
How long till one of the LLMs makes calls out to the other LLMs to evaluate how to play the hand?
I used to play professionally, and I still play in the casinos.

These LLMs are playing better than most human players I encounter (low limits).

They're kinda bad, but not as criminally bad as the humans.

Thank you, I'll try to grab a table when it resets :) ! I've been getting into poker (always wanted to) since I found a lecture series from John Hopkins, and severely disappointed by my options to play online in NY (real or fake money). I just want to get reps in
Idea: can the agents make faces? 1. Programmatically--agents see each other's faces, and they can make their own. They can choose to ignore, but at least make that an input to the decision making. 2. Display them in UI--I just want to see their faces instead next to their model code names :)
This is very cool, one piece of feedback: watching the table as the AI plays while seeing the reasoning is difficult as they're on other sides of the screen. It could be nice to have the reasoning show up next to the players as they make their moves.
Can we chuck a nash equilibrium player in too?
Why are there 2 Claude Players ?
Why not texasholdllm.com?!
Would be amusing if the LLMs could achieve a steady state where nobody definitively wins or loses between each other.

That is, good enough to compete amongst each other but not good enough to for one to win.

Curious if you used pokerkit for this, or some other engine or custom engine?
Are the LLMs "watching" the action, or are they only apprised of previous action once it gets to them?
Honest question, but this seems like an expensive project to host given the number of tokens per second. How is this being paid for?
Needs a four color deck, and the colors on the cards of the waiting players should not be monochrome - makes it hard to evaluate what's happening in the hand. Also, a dealer button on the table would help in visually following the action.
These bots are regularly going down 20%+ on high cards duels
Placing full GPT 5.2 versus fast/flash models of main competitors is unfair, would love to see more balanced table.
This is fun!

Given online is now bot-riddled, I half-finished something similar a while back, where the game was adopting and 'coaching' (a <500 character prompt was allowed every time the dealer chip passed, outside of play) an LLM player, as a kind of gambling-on-how-good-at-prompting-you-are game. Feature request! The rake could pay for the tokens, at least.

So strange that people are into this, but were not into the much stronger non-LLM poker agents.