Should be fun when you're able to look at all the teams that built the algorithms playing each other, the way they think about building their ML etc and then place your bets around that or something. Sounds like actually kinda fun gambling.
You could invest your money in a hedge-fund which uses ML, and there are a bunch to choose for. But you will probably not get many details about they go about things.
Bridges (the supercomputer) was used to train the model, and it's also being used to power the live competition.
For any that may be interested, Bridges is available at no charge for open, peer-reviewed research. It's specifically designed for nontraditional, data-intensive uses of high-performance computing. AI is a good example, as are domains that are increasingly applying AI.
But Michael Feldman did not say that he could do that, he merely said the universe. Through context, it is obvious that he is restricting to the observable universe, just like when people say "American" to specifically to refer to people in the United States.
Sorry, I hadn't realized we were arguing about this level of pedantry.
I just think you missed my very dry wit highlighting how even such an overused phrase can be technically incorrect, the worst kind of incorrect. Even when the rest of the more-technical-than-an-overused-phrase article could be correct.
My apologies, I didn't realize people would take my original response so seriously.
The best no-limit hold-em players in the world are also presumably quite good at limit hold-em, limit omaha, 7 card stud, etc., not to mention other card games like bridge. Serious question: When these programs are built to beat humans in one very particular task, are they any better at beating humans at similar tasks?
Different games have different difficulties for computers. For average human player just counting the odds may be hard.
Just different Texas-hold'em variations have different challenges for computers.
Texas-hold em is very simple if you just play the cards and calculate odds. Limit hold-em is much easier for computer than no-limit hold em. Head's up no-limit hold-em is much simpler than multi-way.
In that situation you would have to somehow control for collusion amongst the poker pros against the machine. Entirely blind, constrained bet amounts, etc.
Yes! Alternatively: just make everything computer mediated, don't tell the humans who else is participating (blind), and also not how many other players are bots or humans.
That could potentially work but I think that poker pros might want the publicity and how would you stop them from talking? It is a small, well connected community.
Precision is easy for computer. You don't need AI or supercomputers to calculate good ranges or pot odds.
In multi-way poker the way you estimate range of another player has effect on how you estimate the range of other players and how you present yourself.
AIs and humans have different skill sets. Could be that HU is harder for a human but easier for AIs that we can write today -- math is not a problem for a computer but modeling many opponents might be.
If all players are equally skilled (you aren't deviating from optimal strategy to exploit another player's weaknesses) it actually has no effect. If the first player (under the gun) folds in poker it players exactly like a game with 1 less player. There is tiny "card removal" effect but this wouldn't fundamentally change any strategies besides tweaking hand ranges.
I disagree. One reason, is that in a 6 or 9-handed game, most players will be folding 70-80% of the time. The AI will be able to understand players tendencies faster than heads-up. And in those games, the strategies and ranges are more predictable than heads-up.
It's much easier to determine the best strategy in a multi-way game than it is in a heads-up game.
30k hands (as opposed to 20k in the first experiment) is usually still not large enough of a sample size from a player's perspective to overcome up and downswings. I suppose a compromise must have been made out of respect for the player's time. I would be very interested in seeing everyone's equity vs actual winnings at the end.
The pay per hand is very good for any normal mid to higher stakes reg player, but I wonder if these guys are taking a pay cut to participate... considering these guys probably charge $500 - $1000 an hour for coaching, and that's aside from their table winnings. I haven't been in the poker scene for some time now, and am a bit out of touch.
I wish we could get some real time player commentary. I would love to see how they adapt with their thought process.
Can somebody explain why poker is a big deal? I don't play poker and I don't see why is it complex for an AI. I can see why chess/go could be a big deal: there's definitely a solution to the game, but the space is too large for it to be computed, so you have to optimize somehow, which real players somewhat can. When computers were slow, humans did it better than computers.
Poker, I thought, doesn't really have any solutions, it is largely about luck, no one really knows what's going on, so aside from some probabilistic computations it is about guessing what your opponent thinks while not giving up what you think. Choosing and seeing the behavior. I would think that playing against AI is generally pointless, since computers don't worry. I mean, surely there are some general guidelines on strategy, since somebody plays better that the others, but following it shouldn't be harder for a computer than for a human, quite the opposite.
Poker in a sense, is the most realistic game in that it has multiplayer, hidden information and stochastic aspects. While chess and go have huge state spaces (as does no limit poker), they are also fairly straight forward due to their 2 player perfect information nature. If say you took an algorithm key to alphago, something like UCT, and tried to apply it to poker, it would diverge from nash equilibrium. Nash Equilibrium by the way, is the sense in which the game has solutions.
In poker, you're looking at your expected value per decision (good players also do this at a meta-level), you're trying to place a bound on the other players' holdings and trying to bluff at an optimal rate. A good poker bot has to be capable of all this, which has proved extremely difficult. This is why it has proven difficult until just about this year, for researchers to be able to clearly say that at least in the heads up case and given enough compute to train, bots are at expert human level.
What might turn out interesting is the fact that though headsup is more difficult for humans and multiplayer is arguably the simpler, the reverse seems to be true for bots. The state space multiplies such that naive application of equilibrium play and tree search simply do not cut it.
I don't understand. Maybe there are some good resources to read, so I could see it better?
Essentially, you are explaining why poker it hard. This is no surprise, and I already included it into my question to an extent. The real question is, why is it harder for computers than for humans? Both seem to be completely oblivious to what is truly going on due to imperfect information nature of poker, and both seem to be able to make simple estimations. Computers are better at multiplying fractions (needed to estimate probability) and computers are better at not-worrying (needed to bluff well). I'd assume it is humans who are at disadvantage, and your post doesn't explain (it seems) why is it the otherwise.
As said before because of hidden information the optimal strategy appears to diverge unless you have more (all) the information in the decision tree. Basically you can never make a decision with the specific hand you have in isolation without knowing what you'd do with every other hand (and same for your opponent). This is very different than chess/go!
So while poker may even have a simpler game tree than chess/go you need to have more of it upfront to solve the game, and being "off" means much much greater error. Depth matters less than breadth.
Said another way in chess/go you can brute force get near optimal for the current decision, then reset and do again for the next decision. But with poker you have to solve for ALL POSSIBLE decisions (hands) up front (and inclusive of all possible board situations), and THEN do AGAIN 3 more times (pre-flop flop turn river) plus the numerous times you are raised means this is much more than 4 times total.
Example:
Bot is out of position with AdAh on AsKsQc facing a big raise. In isolation its probably best to re-raise and simulations show this makes the most money. But you also need to account for the times you have AT (no flush draw) and need to save AT money.
But this logic doesn't stop here. Typically on the river out of position you will check the majority of your range if your opponent has bet every street (this is one of the few "proven" strategies because you always want the pot smaller out of position on average). So, you literally need to have the entire game solved in order to know HOW TO GET TO THE RIVER in a balanced way with good and bad hands. And, getting to the river to play profitably likely means giving up profitability on every other street - this is the diverging from optimal that default algorithms will be wrong on unless they have the entire game tree in front of them.
This again has nothing to do with question in hand. Yes, poker is imperfect information game, everybody knows that. Why does it make it easier for humans than for computers? Humans cannot do these computations in the head any better than computers, quite the contrary.
Thinking through this more right now, it seems like the default way of computers to solve this penalizes them. Every time they get more depth it hurts them unless you account for more breadth.
So lets say the programers know this. This STILL probably doesn't help much because...well idk. What's interesting is Tensor Flow is supposed to be massively parallel (breadth) and abstract the software/algorithms to do this. But if you follow this logic the answer must be gradient decent algorithms also get stuck somewhere... This must have to do with hyper-sensitivity to really having the full decision tree before optimizing.
The answer seems to be if computers are modeling this exactly as humans do, (using neural networks which then simplify this problem with heuristics), it just takes this much computer power to simulate 1 human brain optimizing for this.
Alternatively if they model it in a more brute force way it must take even more computer power because this is proving to be less efficient.
This is very good question. We covered why poker is hard. Now, lets cover why it is hard for computers.
Simply: building model of the opponent(s) is part of poker.
In games like Chess and Go without hidden information you can just "play the game" best you can. Other players are not important. Optimal strategy against good player is optimal against bad player are the same. It's just playing the game.
In poker this is not true. Bets reveal information about the player's strategy and his hand. Figuring the other player out is part of the game. Against bad player, optimal strategy is usually different than against good player. Playing against tight-agressive is different than playing against tight-passive. In multi-way poker it's important to know how strong is the player sitting left to you and change your game accordingly.
That makes sense. But does it mean that if I don't have a real idea of what I'm doing (i.e. you cannot assume my moves are optimal from my point of view) your expertise is essentially useless against me?
If someone plays crazy, the solution is to go back playing simple near optimal poker aka playing the cards, calculate expected values and bet accordingly.
Because a computer has to adapt to the opponents strategy or else the human can simply exploit the rule based algorithm that the computer is following. Most dedicated online players are already using an arsenal of software for probability, statistics, and player profiling while playing, so a computer would really not have an edge with the math since the human is running the same computations. A human can start poking around and explore methods to fool the computer.
"four of the most accomplished professional poker players in the world: Jason Les, Dong Kim, Daniel McAulay and Jimmy Chou"
I am ex-online poker professional and I still follow poker news. These guys do not play online high-stakes NL Holdem, and online is the place for best players wrt strategy.
Last time line-up was better. There was Doug Polk who is one of the very top online players, and he had great results offline too. Btw last time CME held computer-human match he was pissed that 'tie' result was announced because human players only beat computer within 90% confidence interval and 'victory' would be 95% confidence interval as it was explained.
There are dozens of players who play higher stakes, more tournament winnings, more google hits, higher rankings than these 4. It's a shame CME could not bring human players that are actually good.
49 comments
[ 0.94 ms ] story [ 104 ms ] threadShould be fun when you're able to look at all the teams that built the algorithms playing each other, the way they think about building their ML etc and then place your bets around that or something. Sounds like actually kinda fun gambling.
Someone will come up with an AI that's really good at betting on other AI's..
You could invest your money in a hedge-fund which uses ML, and there are a bunch to choose for. But you will probably not get many details about they go about things.
It... did not go in the direction most of the betting market thought it would.
(The challenge is of course to find table that only have drunken humans.)
For any that may be interested, Bridges is available at no charge for open, peer-reviewed research. It's specifically designed for nontraditional, data-intensive uses of high-performance computing. AI is a good example, as are domains that are increasingly applying AI.
Ah, I was scanning the article looking for this phrase. They delivered.
https://en.wikipedia.org/wiki/Observable_universe#Matter_con...
Sorry, I hadn't realized we were arguing about this level of pedantry.
My apologies, I didn't realize people would take my original response so seriously.
Just different Texas-hold'em variations have different challenges for computers.
Texas-hold em is very simple if you just play the cards and calculate odds. Limit hold-em is much easier for computer than no-limit hold em. Head's up no-limit hold-em is much simpler than multi-way.
Heads-Up is much easier game than poker against multiple opponents.
The ultimate test for AI is multi-way against 6 to 9 professional poker players. It's completely different game with more layers of complexity.
Proof: Top HU players can crush 6-9 handed but 6max players get killed HU.
In multi-way poker the way you estimate range of another player has effect on how you estimate the range of other players and how you present yourself.
It's much easier to determine the best strategy in a multi-way game than it is in a heads-up game.
The pay per hand is very good for any normal mid to higher stakes reg player, but I wonder if these guys are taking a pay cut to participate... considering these guys probably charge $500 - $1000 an hour for coaching, and that's aside from their table winnings. I haven't been in the poker scene for some time now, and am a bit out of touch.
I wish we could get some real time player commentary. I would love to see how they adapt with their thought process.
Poker, I thought, doesn't really have any solutions, it is largely about luck, no one really knows what's going on, so aside from some probabilistic computations it is about guessing what your opponent thinks while not giving up what you think. Choosing and seeing the behavior. I would think that playing against AI is generally pointless, since computers don't worry. I mean, surely there are some general guidelines on strategy, since somebody plays better that the others, but following it shouldn't be harder for a computer than for a human, quite the opposite.
So, what is true?
In poker, you're looking at your expected value per decision (good players also do this at a meta-level), you're trying to place a bound on the other players' holdings and trying to bluff at an optimal rate. A good poker bot has to be capable of all this, which has proved extremely difficult. This is why it has proven difficult until just about this year, for researchers to be able to clearly say that at least in the heads up case and given enough compute to train, bots are at expert human level.
What might turn out interesting is the fact that though headsup is more difficult for humans and multiplayer is arguably the simpler, the reverse seems to be true for bots. The state space multiplies such that naive application of equilibrium play and tree search simply do not cut it.
Essentially, you are explaining why poker it hard. This is no surprise, and I already included it into my question to an extent. The real question is, why is it harder for computers than for humans? Both seem to be completely oblivious to what is truly going on due to imperfect information nature of poker, and both seem to be able to make simple estimations. Computers are better at multiplying fractions (needed to estimate probability) and computers are better at not-worrying (needed to bluff well). I'd assume it is humans who are at disadvantage, and your post doesn't explain (it seems) why is it the otherwise.
So while poker may even have a simpler game tree than chess/go you need to have more of it upfront to solve the game, and being "off" means much much greater error. Depth matters less than breadth.
Said another way in chess/go you can brute force get near optimal for the current decision, then reset and do again for the next decision. But with poker you have to solve for ALL POSSIBLE decisions (hands) up front (and inclusive of all possible board situations), and THEN do AGAIN 3 more times (pre-flop flop turn river) plus the numerous times you are raised means this is much more than 4 times total.
Example: Bot is out of position with AdAh on AsKsQc facing a big raise. In isolation its probably best to re-raise and simulations show this makes the most money. But you also need to account for the times you have AT (no flush draw) and need to save AT money.
But this logic doesn't stop here. Typically on the river out of position you will check the majority of your range if your opponent has bet every street (this is one of the few "proven" strategies because you always want the pot smaller out of position on average). So, you literally need to have the entire game solved in order to know HOW TO GET TO THE RIVER in a balanced way with good and bad hands. And, getting to the river to play profitably likely means giving up profitability on every other street - this is the diverging from optimal that default algorithms will be wrong on unless they have the entire game tree in front of them.
Thinking through this more right now, it seems like the default way of computers to solve this penalizes them. Every time they get more depth it hurts them unless you account for more breadth.
So lets say the programers know this. This STILL probably doesn't help much because...well idk. What's interesting is Tensor Flow is supposed to be massively parallel (breadth) and abstract the software/algorithms to do this. But if you follow this logic the answer must be gradient decent algorithms also get stuck somewhere... This must have to do with hyper-sensitivity to really having the full decision tree before optimizing.
The answer seems to be if computers are modeling this exactly as humans do, (using neural networks which then simplify this problem with heuristics), it just takes this much computer power to simulate 1 human brain optimizing for this.
Alternatively if they model it in a more brute force way it must take even more computer power because this is proving to be less efficient.
Simply: building model of the opponent(s) is part of poker.
In games like Chess and Go without hidden information you can just "play the game" best you can. Other players are not important. Optimal strategy against good player is optimal against bad player are the same. It's just playing the game.
In poker this is not true. Bets reveal information about the player's strategy and his hand. Figuring the other player out is part of the game. Against bad player, optimal strategy is usually different than against good player. Playing against tight-agressive is different than playing against tight-passive. In multi-way poker it's important to know how strong is the player sitting left to you and change your game accordingly.
I am ex-online poker professional and I still follow poker news. These guys do not play online high-stakes NL Holdem, and online is the place for best players wrt strategy.
Last time line-up was better. There was Doug Polk who is one of the very top online players, and he had great results offline too. Btw last time CME held computer-human match he was pissed that 'tie' result was announced because human players only beat computer within 90% confidence interval and 'victory' would be 95% confidence interval as it was explained.
There are dozens of players who play higher stakes, more tournament winnings, more google hits, higher rankings than these 4. It's a shame CME could not bring human players that are actually good.