Libratus Poker Paper

A research paper describing a key component of Libratus, an artificial intelligence that displayed its poker prowess earlier this year, won one of three best paper awards at the Neural Information Processing Systems (NIPS 2017) conference this week in Long Beach, Calif.

Libratus Poker Paper Plate

After a 20 day marathon challenge at the Rivers Casino in Pittsburgh, Pennsylvania, the results of the Brains Vs.Artificial Intelligence: Upping the Ante shocked the whole poker community when the Artificial Intelligence (AI) “Libratus” beat 4 of the best poker players in the world heads-up in No Limit Texas Holdem. A research paper describing a key component of Libratus, an artificial intelligence that displayed its poker prowess earlier this year, won one of three best paper awards at the Neural Information Processing Systems (NIPS 2017) conference this week in Long Beach, Calif.

Paper.io

Tuomas Sandholm, professor of computer science, and Noam Brown, a Ph.D. student in the Computer Science Department, will present their paper, 'Safe and Nested Subgame Solving for Imperfect-Information Games,' at the conference Tuesday afternoon.

Libratus bested four top professional players in Heads-Up, No-Limit Texas Hold'em during a 20-day marathon competition in January, Brains Vs. Artificial Intelligence: Upping the Ante. The AI led by a combined $1.8 million in chips after playing 120,000 hands. Libratus used the Pittsburgh Supercomputing Center's Bridges computer to compute its strategy before and during the Pittsburgh event.

The NIPS paper focuses on subgame solving, a process by which a game is decomposed into subgames that can be solved independently. Though it's not possible to solve subgames for an imperfect information game such as poker, it is possible to approximate solutions. Sandholm and Brown made subgame solving a key component of Libratus and developed new solving techniques that outperformed prior methods in theory and in practice.

In addition to the NIPS recognition, Libratus and the PSC also recently won the HPCwire Reader's Choice Award for Best Use of AI.

For More Information, Contact:

Byron Spice 412-268-9068 bspice@cs.cmu.edu

Time to pack it in. The only hurdle left for computers to overcome in order to take over the Earth is sentience. This week, the Libratus Poker AI, developed by Tuomas Sandholm and Noam Brown at Carnegie Mellon University, completed a total destruction of four professional poker players in heads-up No-Limit Hold’em. This comes just two years a previous iteration of the AI – Claudico – lost a similar contest.

The four players taking on the electronic card sharp were Jimmy Chou, Dong Kim, Jason Les, and Daniel McAulay. Each played 30,000 hands against the computer from January 11th through January 30th.

In order to weed out some of the luck involved with heads-up No-Limit Hold’em, a few special rules were implemented. First, the players and the AI were given 20,000 chips at the start of each hand. Blinds were 50/100. By resetting the chip stacks every hand, players – living or otherwise – had plenty of room to make plays and good runs by a player couldn’t snowball into a big stack versus small stack scenario.

Additionally, hands were mirrored, meaning that pairs of players received reversed hands. For example, if Chou was dealt 2-7 offsuit and Libratus got pocket Queens in one hand, McAulay would be dealt Queens and the AI would get 2-7 in a mirrored hand. This way, hands were distributed evenly (though deal was not predetermined), so neither the humans nor Libratus could benefit from getting a sick run of cards.

Finally, once players were all-in with a call before the river, no more cards were dealt. In these situations, the winning probabilities were calculated and players received a percentage of the pot corresponding to their equity in the hand. One would think that this would mean that there would be fewer chances taken on all-in calls, but at the same time it also meant that nobody could get lucky and suck-out on an all-in.

Libratus Poker Paper Towel Holder

Essentially, the purpose of all the special rules was to remove as much chance as possible so that skill could be more accurately measured.

Poker

When the contest was over, it was…no contest. Libratus crushed, winning a total of $1,766,250 from the four players. That’s $14.72 per hand. Dong Kim did the best – or least poorly – losing $85,649. Following him were McAuley with a $277,657 loss, Chou with a $522,857 loss, and Les with an $880,087 loss.

According to PokerListings.com, this wasn’t dumb luck for Libratus. The probability of the four men outplaying the computer yet still losing that much money is between 0.0001 and 0.54 percent.

Libratus Poker Paperwork

Libratus is powered by a $9.65 million supercomputer called “Bridges.” It has trained itself, playing billions of hands to tweak its strategy.

Libratus Poker Paper.io

That strategy? Well, of course we don’t know what it is, exactly. At the same time, there is no specific, set strategy that the computer uses. Instead, it examines the specific scenario it is facing in a hand and makes one of several moves. For example, there might be a situation where Libratus has 4-5 pre-flop and its opponent made a min-raise. In that case, it might be programmed to re-raise by a certain amount half the time, re-raise by another amount 25 percent of the time, call 15 percent of the time, and fold 10 percent of the time. After each match, it analyzed the results and adjusted its strategy to try to find the optimal line.