terrencechan

Bots may be close to solving two handed poker. They seem to be further off to solving multi-handed poker. But the real question is, even if they do eventually solve multi-handed poker, does it really matter?

I've been playing multi-handed limit for a few years and I don't feel that I have to be the best player at the table to make money. As long as I am in the top three, it mostly depends on what my position is on the fish and how well I can play against the field. Position probably even takes precedence. So if I get lucky enough to get a a good table with good position, or more accurately if I am patient enough to wait around long enough to get put at a table with good position then I can make money no matter how many bots are at the table.

Poker is about exploiting the weak players while avoiding the strong players. If the strong players are bots then that only makes them more predictable and easier to play off of.

Anyway, I hope that's true. Otherwise, online poker is doomed. And someday when neural implants are cheap and plentiful all poker is doomed.

Well, in the simple, "I want to make money as a poker player online" calculus, the quality of poker bots matters very very much. To put it succinctly, if poker bots got good enough to consistently play high-stakes poker, they would probably dominate (and undetectably, if at a high enough stake. The trivial workaround for being detected as a bot is to have your poorly-paid assistants enter the bot's plays manually, and beginning and ending sessions in a somewhat reasonable fashion; I'm sure the former poker-security specialist who hangs around here will suggest reasons why my trivial workaround won't work, but it would be worth a lot of money to figure the answer out).

I am not as confident as TC that poker will ever be solved, or even computer-dominated. Well...I think computer-dominated is possible. The issue with comparing it with checkers (fully solved) and chess (verging on computer-dominated) is that, from my uneducated guess, the problem is much less well-defined.

One example: if I show you a chess position, the moves that led to it (aside from the very minor questions of whether castling has occurred, or whether the position may run afoul of the 50-move or three-repeated-position rules) are irrelevant. Play proceeds without reference to what has happened so far.

In contrast, poker seems to require that you profile your opponent's play, at least if you hope to play as well as possible.

I think there is a difference in emphasis more than anything technical. (The technical differences all have known techniques to overcome.) Most poker players are not satisfied with playing break-even poker--- they want to win. Even if given a complete game-theoretic strategy they would probably not use it.

The solution of checkers guarantees at least a draw. It doesn't maximize the lopsidedness of your win if your opponent plays badly.

I think a solution of the former sort (game-theoretic) for some poker games will be feasible over the next 10-20 years, about as long as it took to solve Chckers. But the latter sort of solution (exploitive) is not so well-defined. Poker may or may not become computer-dominated in the exploitive sense without being solved in the game-theoretic sense, or it might be game-theoretically solved but underperform human players at exploiting mistakes.

Poker is not a deterministic game, unlike chess and checkers. Not only is there a random element to every play, the determination of the correct play happens in a circumstance in which not all information about the play is knowable.

Simply put, you don't know what the flop, turn, river, or opponent's hole cards are. "Bad beat" is essentially poker slang for a hand which, while giving a strong +EV expectation, loses that time.

In comparison, the now-solved checkers is done. Given any legal opening, Chinook can guarantee the best possible outcome*.

I'm not sure you can solve poker in the same way, for complicated but boring reasons revolving around the random elements.

What I would bet on is that bot-domination is possible, maybe even in the near future. But what will that mean? Unlike checkers, there won't suddenly be a day on which the last human wins the last hand of poker. There may be a day on which the last pro hangs it up because they are busted and can no longer make money at poker, or there may just be a whole new set of protocols created to thwart bots at online tables (I can think of a few systems that would at least force a human to pretend to play even if the bot did all the work).

*technically, the checkerbot has only weakly solved checkers, which is to say it is known what the optimal response to all opening lines is. A "strong" solution would mean that the computer knew the best play given any legal position, even a midgame that had left the "known best" lines of play at some point.

But, a random game just means you have to use probability, and unknown information about other player's holdings just means you have to use game theory. You still end up with a strategy that says "if your opponent does X in situation Y, then you do Z". The differences are that your action will often be a random mix, and the result is an expected value rather than a guarantee.

I think the game is just as "done" if its expected value is known, and a game-theoretic optimal strategy described, even though a winner is not guaranteed. (For example, Checkers + coin flip to break draws is also solved and the human can win, but the 50% chance of winning doesn't make it any more interesting.)

rcousine says:
"Well, in the simple, "I want to make money as a poker player online" calculus, the quality of poker bots matters very very much."

Here is where you and I part company. For most of us (Terrence may be the possible exception) there are already players out there who would dominate us. If we are to make any money, we must avoid them heads up or coexist with them at multi-player tables. If we can't do that, then we might not even make money in the absence of the bots. The presence of bots probably adds more good players and therefore forces us to choose more wisely, more often. But it does not fundamentally change the equation. And unlike playing against human opponents who are better than we are, bots are likely to be more predictable and less exploitive of us. It is not necessarily a bad thing to have predictable (albeit good) players at your table when you are trying to exploit bad players.

If your point is simply that good players take money off the table that I could have won. Then maybe you are right. But I'm not sure how to judge the effects of having a 24 hour bot fueled game running to draw the fish in compared to playing against a smaller, non-bot field.

I know that at one point, when poker was getting a lot of publicity, it seemed like the supply of fish and the money they brought to the table was limitless. Nowadays it seems less so. But that may also be a passing fad.

Bots fundamentally change the equation because they are effectively infinite multitablers.

There is only one Terrence. The existence of Terrence only drains money out of one or two tables, a few hours a day. Terrence's time is valuable, and so games below a certain threshold of stakes are unaffected by Terrence.

Suppose bots were legal, and I had a bot that played as well as Terrence. In theory that bot could play 24/7 on every single non-microlimit table on every single online poker site. The incremental cost of having the bot play 500 tables instead of 1 or 2 is minimal.

You say you'd just find a game with only one Terrencebot and enough bad humans to make it worthwhile? Guess what, someone else has a Hossbot, and that game is even more profitable for them. The metagame iteratively converges to put enough bots in every game to make it unprofitable for humans who can beat other humans but not the bots.

Hence, widespread availability of bots who can beat the best humans == death of online poker.

If you make bots illegal and try to detect and bar them, well, you're just adding some small amount of per-table expense for countermeasures. You might protect the lower-stakes games by making them more trouble/risk than they're worth for bot operators. There's too much potential profit to expect mid/high limit games to stay safe.

And unlike playing against human opponents who are better than we are, bots are likely to be more predictable and less exploitive of us.

This is debatable. I think it may be easier to randomize bots than a human, making the bots less predictable. And the exploitive part, I think computers may have an advantage in that dept. as well.

I think the future of online poker, and poker in general, is mixed games and other poker games. Surely it is much easier to program a computer to play limit holdem than a mixed game or other games. Or am I being naiive?

Of course it's harder to program a bot that plays mixed games, because you have to teach it to play all those other games. But really it's a trivial task for the computer to change games. If I program the perfect LHE bot and you program the perfect O8 bot and some other guy programs the perfect Stud bot and someone programs the perfect triple draw bot, then it wouldn't really be hard to combine our work and create a mixed game bot, would it?

Polaris does act instantly, which is perhaps a little unnerving to the average player

Presumably it would be a simple fix to build in some kind of randomised delay into such a bot to make it less easy to detect that you were playing against a bot rather than another online player?

Oh, sure. I wrote an interface to connect a bridge AI to a popular online bridge site in the late 90s (with full permission of the site owner and full knowledge of opponents). Not only did it adapt its pace to be less jarring to humans, it also engaged in rudimentary table talk, the bridge equivalent of nh/ty.

(People occasionally tried to engage my bot in conversation. Others would verbally abuse it, delighting in the ability to do so without retaliation. Eventually I coded in some triggers to have the bots occasionally reply to such people, or kick them off the table, as appropriate. Alternately, sometimes they'd do this while I was watching, and I'd play ventriloquist and send commands to make the bot talk back at them.)

Online, there's very little to differentiate a bot from a not-especially-chatty human.

it also engaged in rudimentary table talk, the bridge equivalent of nh/ty.

What about "Why the fuck didn't you pull my double, partner?!"

Bots won't doom online poker, the DOJ will.

Apparently (based on multiple 2+2 threads), there are already HU bots that win at 30-60 and maybe even higher. There were a group of bots that beat 10-20 6max on party. I'm certain there are a lot more going undiscovered.

I just hope the online sites can do something to stop bots. Maybe fingerprint enabled computer mice or something..

Terence,

Nice post, I was wondering how Polaris would look to a top limit HE pro. I also wonder how it would do against a better limit HE player than Phil. Thanks for posting you insight on this matchup.

In response to dougie_nutz
"Does it really matter?
I've been playing multi-handed limit for a few years and I don't feel that I have to be the best player at the table to make money."

Lets assume that there is a bot that can beat the average 15-30 online limit holdem game for 1.5 BB per hr, I believe there probably is. Lets say the bot doesn't beat you but doesn't lose anything to you either, so the bot is close to even with you.
Now with only one computer lab not much bigger than your average online cafe, a sophisticated computer person could probably find away to have 50 of these bot's running at all times on 15-30 tables across all the online sites. Obviously they would have to set it up so that they were 50 different bots running in reasonable playing intervals from different Looking IP's and hardware etc, all doable. If you don't believe that's doable then assume 5 different guys can separately figure out how to run 10 bots each or 10 guys run 5 bots each, you get the picture.

So 50 bots averaging 45$ an hour each, 24 hrs a day, equals 54, 000$ a day or 1.62 Million dollars a month. I think this could probably be done from only one computer lab anywhere in the world, ie Russia, Yugoslavia, Korea etc.
So if you don't see how the ability for one computer lab alone to drain the 15-30 limit community of 1.6 million a month would affect your overall ability to make money in middle limit online games, then I don't know what else to say.

If bots arn't already a big problem for online middle limit holdem, I believe they will become one of the biggest problems if not the biggest facing online poker moving foward.

So if you don't see how the ability for one computer lab alone to drain the 15-30 limit community of 1.6 million a month would affect your overall ability to make money in middle limit online games, then I don't know what else to say.

Well, I certainly acknowledged this criticism in my response to another poster:

If your point is simply that good players take money off the table that I could have won. Then maybe you are right. But I'm not sure how to judge the effects of having a 24 hour bot fueled game running to draw the fish in compared to playing against a smaller, non-bot field.

Now let me emphasize that I am not saying that you are definitely wrong, I am just trying to look at it from a different angle and point out what I think are interesting ideas for discussion.

The general consensus seems to be that once someone produces a bot that has solved poker, online poker is dead. Or at least the ability of human experts to make money at online poker will be dead. That is one idea that I am challenging.

For one thing, if bots sucked all of the money out of poker and killed the game, then with the game dead, why would the people who run the bots keep playing the game? The people who run the bots wouldn't want to destroy online poker. They just want to make money off of it like every poker pro out there. Has the presence of poker pros killed the game?

But I don't even think that it is a given that bots would hurt the game that much. I know that the most intuitive conclusion is that they would hurt the game by taking money off the table. But these things do not always work out in the most intuitive way.

Several years ago a few of the top poker players really resisted the idea of showing hole cards on TV. They argued that if their hole cards were shown then other expert players would be able to analyze their play and spot weaknesses and exploit them. They were convinced that showing hole cards was bad for their game. They were not dumb people but they were horribly wrong nonetheless. Hole card cams are now widely recognized as one of the main reasons for the success of televised poker and televised poker may be the largest (or second largest after online poker, I don't really know) contributing factor to the growth of the poker industry. Showing hole cards was not bad for anybodies' game.

The presence of bots could always have similarly unintended consequences.

Unfortunately, I am not really knowledgeable enough to make any conclusions. My main hope was to raise some possibilities for more knowledgeable people to ponder and respond. I'm really curious what Terrence thinks since he is both a very good player and has a lot of knowledge of the industry.

I'm really curious what Terrence thinks since he is both a very good player and has a lot of knowledge of the industry.

I'm inclined to think that bots -- or more accurately, the computation of optimal or near-optimal play of various forms of poker -- will in fact be the downfall of online poker. But as outlined in the post itself, it probably won't be until that day.

First, the near-equilibrium approach doesn't work for multi-player games -- we're probably years away from having the same degree of success in ring games, even for three players. So it *might* hurt some heads-up specialists, but even that is speculation.

The poker economy is something of a pyramid, built on a base of losing players who just keep pumping money into the system. I expect online poker will shrink when the losers no longer get enough entertainment value for their dollars. However, lots of people continue to play online casino games like craps and roulette, where they simply can't win. Predicting the end of online poker simply because bots can play well is just plain silly.

[Incidentally, there is a lot of mysticism and nonsensical beliefs about No Limit. We could probably have a heads-up No Limit player on par with Polaris in a week or two, if we wanted to put our time into that. And humans play No Limit much worse than they play Limit... But again, multi-player games are a completely different animal.]

Terrence: the bots you were watching were not the strongest in the line-up. In the Tuesday afternoon session, Phil ended up playing one of the weaker bots the majority of the time, due to a bug in the Coach. In the evening, the players were told who they would be facing (Mr. Pink), and that the bot wouldn't be doing any learning of any kind, so they could take a lot of liberties, playing in a highly predictable fashion. They faced the same bot in the first session, and were given the complete information logs to analyze its play and search for weaknesses. We did this for scientific reasons -- we were trying to maximize the amount we learned, rather than trying to maximize our chances of winning the match.

I think Agent Orange is stronger in practice, because its constant aggression makes it harder to find and exploit the gaps. According to the unbiased DIVAT assessment, it outplayed both Phil and Ali in the second session by 0.1 sb/h, which is a pretty hefty chunk. In fact, the analysis indicates that the bots held the edge in play in all four sessions, but there is still quite a lot of noise in the outcome of any one session. The losses may have been largely due to a few big swing hands going the wrong way (e.g. hands where one side folded preflop, while the other saw the flop and won or lost a big pot). The only real result was that neither side won by a significant margin over 4000 hands (2000 duplicate). In other words, the match was inconclusive either way.

I still believe i can grind down any one of the bots in isolation, but that may not be true for very long. Once they can beat me consistently, i believe they will beat anyone, because (a) i'm a very good player (after many years studying and expanding poker theory, my knowledge of the game goes much deeper than anything that has been published), (b) i have a better understanding of the full breadth of possibilities, not just the very narrow view that most human players have, so i'm better equipped to handle anything the bots might derive on their own, and (c) i have inside knowledge of how the bots were built and where the gaps and weaknesses might be. If the bot's skill can overcome all that, then they're welcome to play my chips against any human player.

Mr. Pink and Agent Orange are now available on Poker Academy Online if you want to spar against them. If you don't have a copy of PA Pro, send me an email and i'll give you a key.

- Darse.

Lots of people continue to play online casino games like craps and roulette, where they simply can't win. Predicting the end of online poker simply because bots can play well is just plain silly.

If real-money online poker becomes akin to continually shuffled blackjack - play well and you almost break even, play badly and you lose big, and everyone knows it's a sucker game - I'd call that dead. A few people may play, and sites may still rake a little, but nothing compared to what they do now.

What Polaris's success tells me is that I might not want to focus too narrowly on very-shorthanded limit holdem right now, since it might not have much of a future online.