Artificial Intelligence and Poker

Introduction

In recent times there has been a great deal of discussion about artificial intelligence and poker. More specifically online poker enthusiasts are concerned that automated playing robots or “bots” as they are known have been developed by either online poker sites or players themselves and have been implemented in games for real money. The concern is that “bots” could prove to be stronger than average or indeed world class poker players, thus effecting the profitability of all online poker players. Assuming that “bots” could or have been developed and implemented, this article examines current artificial intelligence research in poker to determine how strong at poker “bots” are at the moment and how strong poker “bots” could potentially be. We conclude with a discussion on the motivations of developing such a "bot".

Artificial Intelligence (AI)

AI can be seen as an attempt to model aspects of human thought on computers. It is also sometimes defined as trying to solve by computer any problem that a human can solve faster.

Historically computers have performed well beyond human capabilities in terms of speed and accuracy when performing large computations. However, an essential aspect of an expert poker player is the ability to adapt to individual opponent playing characteristics to maximise profit. Historically when faced with simulating such adaptive behaviour computers have performed relatively poorly.

AI and Poker

Surprisingly AI technology has rarely been applied to poker given the games popularity. The reason for this as pointed out by [Billings et al. 1998] is most likely due to the complexities of poker. These complexities include “imperfect information since the other players’ cards are not known [sic], multiple competing agents (more than 2 players), risk managements (betting strategies and their consequences), agent modeling (identifying patterns in opponent’s strategy and exploiting them), deception (bluffing and varying your style of play) and dealing with unreliable information (taking into account your opponents deceptive plays). All of these are challenging dimensions to a difficult problem[sic]”.

How Strong are Poker Bots at the Moment?

A major assumption in determining ‘bot’ strength is that all hands are played “straight up”. That is all starting hands, flops, turns and rivers are distributed randomly for “bots” and players alike.

Koller and Pfeffer (1997) have recently developed the first practical algorithm for finding optimal randomised strategies in two player poker using game theory and a tree system to find this strategy, an algorithm they named GALA. However, the size of the tree prompted the authors to state that “we are nowhere close to being able to solve huge games such as full scale poker, and it is unlikely we ever will”. If Kroller and Pfeffer did manage to build such a system it would be unlikely to be of a world class standard, since such a system would be static in nature and lack the requirements to adjust to individual playing characteristics, which would severely limit profit potential.

Perhaps the leaders in AI poker technology are the University of Alberta (http://www.cs.ualberta.ca/~games/poker/) poker research team. They have developed the first known truly adaptive poker “bot” named “Poki”. Poki uses enumeration techniques, whereby the algorithm sorts through possible combinations to determine such things as hands strength and hand potential, which are used to determine betting strategies. The enumeration technique is dynamic in nature adjusting possible combinations of opponents hands from past history of opponents betting strategies. The researchers at the University of Alberta claim “Poki plays a reasonably good game of poker, but there remains considerable research to be done to play at a world-class level”.

How Strong Could Poker “bots” Potentially be in the Future?

Recent advancements in AI technology have increased the adaptive power of computer capabilities. Artificial Neural Networks (ANN’s) are an example of this, where a feed back algorithm (usually back propagation) is used to enable computers to learn and recognise patterns over a sample. Such a technology could be well suited to poker.

ANNs have been used with great success recently in financial prediction, biology, engineering and recently in games such as backgammon and chess. The software program “JellyFish” for backgammon uses such technology and it is widely accepted that the software plays at a very high level. On the highest playing level it matches the best humans in the world.

Surprisingly there has been little poker research conducted using ANN technology. As what work that has been done has proved to be very successful.

Davidson [1999] used an ANN to determine what an opponent would do in a given situation (fold/call/raise). Davidson used 19 inputs for is ANN, variables included such pot odds, number of players dealt in, opponents last action etc. Whilst a long way off simulating full scale poker his worked showed out of sample predictive power of over 80% when predicting an opponents next action.

ANN technology appears to be the most likely method to incorporate playing aspects such as bluffing, opponent modeling and deception, which are requirements to be a truly world class player poker player.

However, due to the complexities of poker it seems that the development of a truly world class ‘bot’ is some time away.

Motivations of Online Poker Rooms and AI

It seems unlike that the vast majority of online poker rooms would be interested in developing an automated world class poker playing ‘bot’. The biggest poker room ‘Paradise Poker’ makes an estimated $70,000 USD in rake takings per day while a smaller site such as Poker.com has estimated rake earnings of up to $10,000 USD according to pokerplus.com estimates. Whilst the development of a world class poker ‘bot’ would surely allow profits from unsuspecting players in the short run. The potential harm to site profits in the long run would be alarming as players substitute into more profitable sites as they

1) Obtain information about the implementation of the ‘bots’.

2) Notice that potential profit opportunities from the site have been reduced by an increase in the ability of players.

To coin an old expression ‘they would be killing the goose that laid the golden egg’.

The most likely source of such a technology would come from university research or an online poker playing enthusiast, since potential rewards to such research would be high.

It is commonly excepted in poker circles that some poker rooms do use automated playing bots, if this is true what might be their motivations?

Most of the allegations have centered around new or smaller poker sites. A plausible motivation for implementing bots might not be profit related at all. Quite the contrary ‘bots’ could be implemented to make the site look busier to new customers who first log onto the site to encourage playing on their first and subsequent visits during the poker rooms infant stage. Given this and the state of current technology any ‘bots’ presented could be easy target to a sophisticated player as their ability is likely to be poor.

Conclusion

Imperfect information in poker makes it a difficult game to model using AI technology. It seems that a truly world class playing “bot” has yet to be developed and may be some time off. The use of ANNs is perhaps the most likely technology to achieve this world class level. With the growth of online poker considerable riches would be on offer to the person or persons that manage to achieve this feet.

References

1.) Billings, D., Papp, D., Schaeffer, J., Szafron, D. 1998, Opponent Modeling in Poker. (http://www.cs.ualberta.ca/~games/poker/) and AAAI, pp.493-499

2.) Billings, D., Pena, L., D., Schaeffer, J., Szafron, D. 1999. Using Probabilistic Knowledge and Simulations to Play Poker. AAAI’99

3.) www.pokerpulse.com

4.) Koller, D. and Pfeffer, A. 1997. Representations and Solutions for Game-Theoretic Problems. Artificial Intelligence 94 (1-2), 167-215.

5.) Davidson A, 1999. Using Artifical Neural Networks to Model Oppenents in Texas Hold’em Poker. CMPUT 499/605 and http://spaz.ca/aaron/school/