Theoretical Poker
In that section we will explore how poker is solved, what are the math, the solution , and the general theory behind poker.
Teach a bot Rock-Paper-Scissors with Regret Matching
Regret matching (Hart & Mas-Colell, 2000) is the foundation stone under Counterfactual Regret Minimization — the algorithm that cracked computer poker. Here you implement it from scratch on the simplest possible game, then play against your own bot and watch it learn to punish whatever pattern you fall into.
The bot is wired up. getStrategy turns its cumulative regrets into a mixed strategy (probabilities proportional to positive regret), regretsFor computes per-action regrets after each round, and getAction samples its next throw from the strategy. Play below — the bot already learns and punishes your patterns. Expand See morebeneath the history for the payoff matrix, the function contract, and the regret-matching theory.
Your move
Make a throw
waiting
waiting
Bot's strategy
Top bar = how often you have played each move. Bottom bar = the probability the bot will play it next (from getStrategy). Once your TODOs are in, the bot should grow to counter your most frequent throw.
Recent rounds
No throws yet. Play a move — then go implement the TODOs and watch the bot stop being a pushover.
See more — payoff matrix, bot contract & regret-matching theory
The game
A two-player zero-sum game. The table shows the row player's payoff for each match-up: +1 win, −1 loss, 0 draw. Same gesture draws; rock breaks scissors, scissors cut paper, paper covers rock.
Optimal play is the mixed strategy (⅓, ⅓, ⅓) — the Nash equilibrium. Any bias you show is something regret matching can exploit.
Your contract
Three small functions, exactly as in the primer's worked example. They're stubbed to safe defaults so the game runs from the first second — but a stubbed bot is just a coin-flip.
getStrategy(regretSum)regretsFor(mine, theirs)getAction(strategy)Step by step
Read the regret idea
Regret = (utility of an action we could have played) − (utility of the action we actually played), for a fixed opponent move. Positive regret means "I wish I'd done that instead".
u(paper, paper) − u(rock, paper) = 0 − (−1) = 1Implement getStrategy
Clip regrets at zero, normalise so they sum to 1. No positive regret yet? Play uniformly. This is the strategy the bars below visualise each round.
regretSum = [0, 1, 2] → strategy = [0, ⅓, ⅔]Implement regretsFor
After each round, score every action against what you just threw and subtract the bot's own payoff. The caller accumulates this onto regretSum.
bot played rock, you played paper → regrets = [0, 1, 2]Implement getAction
Sample from the strategy with a single uniform random number and a running cumulative sum. Now the bot actually exploits your tendencies.
strategy [0, ⅓, ⅔], r = 0.5 → returns ScissorsWhere this is heading
Your bot does regret matching against you— it converges to a best response to your habits. The primer's next exercise is to let both players run regret matching in self-play; their average strategies then converge to the Nash equilibrium (⅓, ⅓, ⅓). That same machinery, applied to game trees with hidden information, becomes Counterfactual Regret Minimization — the next mission, on Kuhn Poker.
Build a CFR for Kuhn Poker
A learning space for implementing Counterfactual Regret Minimization from scratch on Kuhn Poker, then testing the strategy with a small game simulation.
First steps for creating a Kuhn Poker solver: start with a bit of theory on extensive games and game theory, then read the CFR paper.
Conditional Probabilities
CFR updates a decision using only the situations where that decision is reached. Conditional probability is the language for reasoning about those filtered worlds: how likely an event is once some information is already known.
P(A | B) = P(A and B) / P(B)If the public history is check-bet, estimate the chance a player holds K only among deals that could have produced check-bet.
Reach probabilities weight regrets by the probability that the current information set is reached under the players' strategies.
// Build your Kuhn Poker CFR here.
// Suggested pieces:
// - cards: J, Q, K
// - legal actions by history
// - terminal utility
// - regret matching
// - recursive CFR traversal
// - average strategy display- Deal cards.
- Choose actions from the trained average strategy.
- Resolve the terminal history.
Simulation output will appear here.