Collection of game-theoretic algorithms for Poker. Implemented algorithms include:

• CFR+ (Counterfactual Regret Minimization)
• DBR (Data Biased Response)
• RNR (Restricted Nash Response)
• AIVAT and Imaginary Observations for utility estimation

Current implementation only supports limit betting games and smaller games with up to 5 cards (hole cards + community cards). Many algorithms are also suited only for small Poker games like Leduc Hold'em.

This library was implemented to test and improve the implicit modeling agent which was proposed in [Bard et al., 2013].

Various top level directories contain implementations of the above mentioned algorithms. All algorithms are used either in scripts located with the implementation or in some of the experiments and tests in verification directory.

• cfr - The CFR+ algorithm
• evaluation - Evaluation of trained strategies
• implicit_modeling - Implicit modeling framework introduced in [Bard et al., 2013]
• response - Calculation of response strategy to another (DBR, RNR)
• utility_estimation - Estimation of strategy utility (AIVAT, Imaginary Observations)

Some algorithms produce files containing the trained strategy. Strategy files have following specification:

• Lines starting with # are comments and have no meaning.
• Other lines contain action probabilities at game's information sets and have following structure:

<informationSet> <foldProbability> <callProbability> <raiseProbability>

• The <informationSet> contains numbers of cards dealt to the player or letters representing actions taken by player and opponents. Card numbers are separated by :. Refer to this file for cards numbers translation.

Implementation makes heavy use of tools from ACPC poker infrastructure through Python wrapper which can be found here. Complete poker infrastructure is contained within the wrapper library, therefore to run code in this repo only the ACPC python wrapper library is needed.

If you want to test the strategies generated by CFR implementation in this repo please take the time to read about the architecture of ACPC here first.

This repo contains two types of tests in test and verification directories. The test directory contains unit tests. The verification directory contains tests that verify correctness of implemented algorithms. These tests often produce results that need to be checked and verified by developer. This directory also contains various experiments.

[Bard et al., 2013] Bard, N., Johanson, M., Burch, N., and Bowling, M. (2013). Online implicit agent modelling. In Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS ’13, pages 255–262, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems.