January 20, 2015


Computers Are Learning How To Treat Cancer And Diabetes By Playing Poker And Atari (OLIVER ROEDER, 1/20/15, 538)

Poker has been solved. Michael Bowling, a computer science professor at the University of Alberta -- along with co-authors Neil Burch, Michael Johanson and Oskari Tammelin -- published findings to that effect earlier this month in the journal Science. For a specific poker game -- heads-up limit hold 'em -- a computer algorithm is now indistinguishable from perfect.

What's more, the program -- dubbed Cepheus1 -- is self-taught. Over two months, it played trillions of hands against itself. It learned what worked, what didn't, and it improved. The game is now "solved" in the sense that you could play poker against Cepheus all day, every day for a lifetime and not be able to distinguish the program from the Platonic ideal of a poker player. [...]

[B]oth Bowling and Schaeffer emphasized that, in the end, their work is not really about games at all.

"We're not doing research into games. I'm not here doing work that will allow humans to play better checkers," said Schaeffer. "We're interested in finding ways to make computers perform tasks that you normally think humans should be doing." Games are just the test bed.

"We do have computer programs that are capable of doing very specific tasks very, very well. But that doesn't say a lot for what humans are good at, which is doing a very general space," said Bowling.

Perhaps, rather than interacting with age-old board games or dusty video games, these algorithms can interact with other things -- biological or medical information, say.

Databases used by Schaeffer's checkers program were hundreds of gigabytes large -- very large indeed in 1994 -- and had to be compressed to be useful. So he figured out how. Two years later, Schaeffer founded a company called BioTools. Why? The company was doing analysis of the human genome, to better understand DNA and protein. The DNA sequencing generated massive amounts of data. Data that had to be compressed to be useful.

Bowling's poker work has spilled over into diabetes treatment. Rather than finding the optimal strategy in a poker hand, a similar algorithm can provide optimal treatment recommendations for adjusting insulin injection formulas. The "game," in this case, is dealing with the worst-case responses of a patient to a given treatment. "We have a proof-of-concept paper that shows that robustness according to a certain class of risk measures can be optimized" by solving this carefully constructed game, Bowling explained.

He also has hope for security application, like at an airport. "We're starting to look at the area of game theory for security, where you are trying to find a defender strategy for protecting strategic infrastructure from an attacker trying to exploit your policy. Game theoretic solutions have already been deployed by Milind [Tambe]'s group [at USC], but this result shows our ability to scale to problems of unprecedented size, which can enable new problems to be tackled, or more complex models of existing situations," Bowling said in an email.

And, of course, there's the famous case of Watson. The IBM supercomputer famously defeated Ken Jennings and Brad Rutter at the game show "Jeopardy!" It's now being put to use to improve the treatment of cancer and PTSD.

Posted by at January 20, 2015 1:56 PM

blog comments powered by Disqus