Probability estimation in poker: A qualified success for unaided judgment

In Journal of Behavioral Decision Making.

by
James Liley, Tim Rakow *
University of Essex, Colchester, UK
email: Tim Rakow (timrakow@essex.ac.uk)

Abstract
Poker players make strategic decisions on the basis of imperfect information, which are informed by their assessment of the probability they will hold the best set of cards among all players at the conclusion of the hand. Exact mental calculations of this probability are impossible - therefore, players must use judgment to estimate their chances. In three studies, 69 moderately experienced poker players estimated the probability of obtaining the best cards among all players, based on the limited information that is known in the early stages of a hand. Although several of the conditions typically associated with well-calibrated judgment did not apply, players judgments were generally accurate. The correlation between judged and true probabilities was r > .8 for over five-sixths of the participants, and when judgments were averaged across players and within hands this correlation was .96. Players slightly overestimated their chance of obtaining the best cards, mainly where this probability was low to moderate (<.7). Probability estimates were slightly too strongly related to the strength of the two cards that a player holds (known only to themselves), and insufficiently influenced by the number of opponents. Seemingly, players show somewhat insufficient regard for the cards that other players could be holding and the potential for opponents to acquire a strong hand. The results show that even when judgment heuristics are used to good effect in a complex probability estimation task, predictable errors can still be observed at the margins of performance.

Shifting Gears

The current issue of Management Science has a cool article entitled "Poker Player Behavior after Big Wins and Big Losses".

The actual journal article is firewalled for subscribers only, the link takes you to an earlier version on one of the author's website.

Looking at playing histories collected from the 25/50 blind NLHE tables at FullTilt they found that after a big loss players tend to loosen up and after a big win players tend to play less aggressively.

It's a long article that goes into some of the behavioral theories about how wins and losses might effect gamblers.

Data Mining

When I was a student, a long, long time ago, the term data mining described a frowned upon practice. Only bad people did data mining. It was a term used to describe the practice of doing a series of tests of significance on a set of data until a statistically significant effect was found. It was a form of sequential testing but it was a very unscientific, atheoretical form. Bad practice.

All that's changed. Data Mining today is a generally accepted practice, a tool that any good data analyst should have in their tool box. But today's version of data analysis isn't really the same as yesterday's version. There's some subtle but important distinctions to be made.

Wikipedia gives a simple definition for the term data mining

Data mining is the process of extracting patterns from data.

That's as good as any place else as a place to start defining the concept, but it's just a start. When we use the term data mining we're talking about more than just data, we're talking about a lot of data, a whole lot.

That's one of the ways that data mining differs from statistics. The field of statistics was developed as a tool to extract information from small amounts of data. Small sample statistics is the backbone of statistical theory and practice. We don't do that with data mining -- we're looking at extraction of information from huge collections of data.

Statistics is based on using probability models to fit data or to test hypotheses. Data mining is a process of uncovering models, not fitting models. A process of forming hypotheses, not testing them.

Here's a video of the first lecture a course on data mining given for first year grad students at Stanford and given simultaneously at the Google headquarters.



In this lecture he's mostly just defining the concept. I'll follow along in subsequent posts.