We Folded: AI Bests the Top Human Poker Pros

Roughly a year ago, to the day, Google researchers announced their artificial intelligence, AlphaGo, had mastered the ancient game of Go. At the time, Discover wrote that there was still one game that gave computers fits: poker.

Not anymore.

Late Monday night, a computer program designed by two Carnegie Mellon University researchers beat four of the world’s top no-limit Texas Hold’em poker players. The algorithm, named Libratus by its creators, collected more than $1.5 million in chips after a marathon 20-day tournament in Pittsburgh. The victory comes only two years after the same researchers’ algorithm failed to beat human players.

Tough Nut to Crack

In the past few decades, computer scientists’ algorithms have surpassed human prowess in checkers, chess, Scrabble, Jeopardy! and Go—our biological dominance in recreational pastimes is dwindling. But board games are played with a finite set of moves, the rules are clear-cut and your opponent’s strategy unfolds on the board. Computers are well-adapted to sort through and make the most optimal choices in these logical, rules-based games.

Poker was seen as a stronghold for human minds because it relies heavily on “imperfect intelligence”—we don’t know which cards our opponents hold, and the number of possible moves is so large it defies calculation. In addition, divining an opponent’s next move relies heavily on psychology. The best players have refined bluffing into an art form, but computers don’t fare very well when asked to intuit how humans will react.

These hurdles were obviously no match for the improved algorithm designed by Tuomas Sandholm and Noam Brown. While they haven’t yet released the specific details of their program, it seems that they relied on the well-worn tactic of “training.” Libratus ran trillions of simulated poker games, building its skills through trial and error, until it discovered an optimal, winning strategy. This allowed the AI to learn the nuances of bluffing and calling all by itself, and meant that it could learn from its mistakes.

“The best AI’s ability to do strategic reasoning with imperfect information has now surpassed that of the best humans,” Sandholm said in a statement.

Better Every Day

Sandholm says that the Libratus would review each day’s play every night and address the three most problematic holes in its strategy. When play began the next day, the human players were forced to try new strategies in their attempt to trick the machine. The poker pros would meet every night as well to discuss strategies, but their efforts couldn’t match the processing power of the Pittsburgh Supercomputing Center’s Bridges computer, which drew upon on the equivalent of 3,300 laptops worth of computing power.

Libratus seemed to favor large, risky bets, which initially made the human players balk. They soon learned that it was best to try and defeat the AI early on in a hand, as that’s when the most cards are unseen and uncertainty is greatest. As more cards are flipped and decisions made, the computer was able to further refine its decision making.

The algorithm isn’t limited to poker either. While this version of the program was trained specifically on the rules of Texas Hold ‘Em, it was written broadly enough that it could conceivably learn to master any situation that contains imperfect information, such as negotiations, military strategy and medical planning.

Libratus isn’t quite ready for the World Poker Tour yet. The version of the game it played only included two opponents at a time, unlike most tournaments. Games with more players compound the number of variables at play, making it significantly more difficult for a computer to choose the best course of action.

Why Distracted Drivers Matter for Automated Cars

When a 2015 Tesla Model S collided with a tractor trailer at highway intersection west of Williston, Florida, the resulting crash killed the Tesla driver. An investigation of the May 7, 2016 incident by federal investigators found that the Tesla car’s Autopilot driver-assist system was not at fault and showed that the driver had at least seven seconds to spot the tractor trailer prior to the crash. But the tragedy emphasized the fact that the latest automated cars still require drivers to pay attention and be ready to take back control of the wheel.

The need for human drivers to take control at least some of the time will last until automakers roll out the first fully driverless commercial vehicles. Most studies have understandably focused on how quickly drivers can take back control from future self-driving cars in emergency situations. But researchers at the University of Southampton in the UK found very few studies that looked at takeover reaction times in normal driving situations such as getting on and off of highways. Their new paper found such different takeover reaction times among individual drivers—anywhere from two seconds to almost half a minute—that they suggested automated cars should give drivers the flexibility to choose how much time they need.

“It is evident that there is a large spread in the takeover reaction times, which when designing driving automation should be considered, as the range of performance is more important than the median or mean, as these exclude a large portion of drivers,” researchers wrote.

This matters because the handover between automated and manual steering could prove dangerous if the human drivers remain distracted or unprepared. Makers of automated cars might feel tempted to simply find the average control transition times among drivers and develop technology standards based on the average. But such systems would not work so well for drivers who react much more quickly or slowly than the average, according to the paper published in the online Jan. 26 issue of the Journal of the Human Factors and Ergonomics Society.

The Road to Safer Automated Cars

Most past studies only reported the average takeover reaction times instead of the full spread of takeover reaction times. The University of Southampton study found a fairly large spread despite involving a small sample size of just 26 drivers—10 women and 16 men—operating a Jaguar XJ 350 driving simulator.

The UK researchers tested how well the drivers handled transitions between automated driving and manual driving both with and without the distraction of reading an issue of National Geographic magazine. Having the added distraction added a delay of 1.5 seconds on average to the takeover reaction times. That means automakers may want to consider making self-driving cars that can adjust takeover times if they sense the driver is distracted.

“In light of these results, there is a case for “adaptive automation” that modulates [takeover reaction times] by, for example, detecting whether the driver gaze is off road for a certain period and
providing the driver with a few additional seconds before resuming control,” researchers said.

TheUniversity of Southampton study also took the unprecedented step of looking at how long the human drivers needed to switch from manual driving to automated car control. That spread of times ranged from just under three seconds to almost 24 seconds.

How to Train Your Robot with Brain Oops Signals

Baxter the robot can tell the difference between right and wrong actions without its human handlers ever consciously giving a command or even speaking a word. The robot’s learning success relies upon a system that interprets the human brain’s “oops” signals to let Baxter know if a mistake has been made.

The new twist on training robots comes from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Boston University. Researchers have long known that the human brain generates certain error-related signals when it notices a mistake. They created machine-learning software that can recognize and classify those brain oops signals from individual human volunteers within 10 to 30 milliseconds—a way of creating instant feedback for Baxter the robot when it sorted paint cans and wire spools into two different bins in front of the humans.

“Imagine being able to instantaneously tell a robot to do a certain action, without needing to type a command, push a button or even say a word,” said Daniela Rus, director of CSAIL at MIT, in a press release. “A streamlined approach like that would improve our abilities to supervise factory robots, driverless cars and other technologies we haven’t even invented yet.”

Cultivating Common Sense

Nestled among Seattle’s gleaming lights on a gloomy September day, a single nonprofit wants to change the world, one computer at a time. Its researchers hope to transform the way machines perceive the world: to have them not only see it, but understand what they’re seeing.

At the Allen Institute for Artificial Intelligence (AI2), researchers are working on just that. AI2, founded in 2014 by Microsoft visionary Paul Allen, is the nation’s largest nonprofit AI research institute. Its campus juts into the northern arm of Lake Union, sharing the waterfront with warehouses and crowded marinas. Across the lake, dozens of cranes rise above the Seattle skyline — visual reminders of the city’s ongoing tech boom. At AI2, unshackled by profit-obsessed boardrooms, the mandate from its CEO Oren Etzioni is simple: Confront the grandest challenges in artificial intelligence research and serve the common good, profits be damned.

 AI2’s office atmosphere matches its counterculture ethos. Etzioni’s hand-curated wall of quotes is just outside the table tennis room. Equations litter ceiling-to-floor whiteboards and random glass surfaces, like graffiti. Employees are encouraged to launch the company kayak for paddle breaks. Computer scientist Ali Farhadi can savor the Seattle skyline from the windows of his democratically chosen office; researchers vote on the locations of their workspaces. It’s where he and I meet to explore the limits of computer vision.

At one point, he sets a dry-erase marker on the edge of his desk and asks, “What will happen if I roll this marker over the edge?”

“It will fall on the floor,” I reply, wondering if Farhadi could use one of those kayak breaks.

Narrow AI systems are like savants. They’re fantastic at single, well-defined tasks: a Roomba vacuuming the floor, for example, or a digital chess master. But a computer that can recognize images of cats can’t play chess. Humans can do both; we possess general intelligence. The AI2 team wants to pull these computer savants away from their lonely tasks and plant seeds of common sense. “We still have a long way to go,” Etzioni tells me.

Etzioni’s 20-year vision is to build an AI system that would serve as a scientist’s apprentice. It would read and understand scientific literature, connecting the dots between studies and suggesting hypotheses that could lead to significant breakthroughs. When I ask Etzioni if IBM’s Watson is already doing this, I feel I’ve struck a nerve. “They’ve made some very strong claims, but I’m waiting to see the data,” he says.

But there’s also a darker side to this noble endeavor. If we grow to depend on these emerging technologies, certain skills could become obsolete. I can’t help but wonder: If smarter AIs gobble up more human-driven tasks, how can we keep up with them?

A Glimpse of a Microchip’s Delicate Architecture

Computer chips continue to shrink ever smaller, but we still wring more processing power out of them.

One of the problems that comes with taking our technology to the nanoscale, however, is that we can no longer see what’s going on with them. Computer chips, with their arrays of transistors laid out like cities, have components that measure as little as 14 nanometers across, or about 5,000 times smaller than a red blood cell. Checking out these wonders of engineering without using expensive and destructive imaging techniques is a challenge, to say the least. 

Viewing Technology With Technology

Researchers from the Paul Scherrer Institut in Switzerland say that they may have found a way to look into microchips without ever touching them. Using an imaging technique similar to Computed Tomography (CT) scans, they bombarded a chip with X-rays and used a computer to assemble a 3-D reconstruction of its delicate architecture. The process works by taking a series of 2-D images based on how the X-rays bounce off of the structures, which is then combined into a realistic model.

In a paper published Wednesday in Nature, say that they can resolve details as small as 14.6 nanometers, or about the size of the smallest components in today’s commercial chips. They tested their technique first on a chip with a familiar layout, and then one that wasn’t — both times they successfully reconstructed a model of the chip’s inner workings with enough detail to see how it functioned, including the transistors and interconnects. The images show the intricate patterns of interconnected transistors on the silicon surface — some chips today can contain upwards of 5 billion transistors.

Google Street View Cars Are Mapping Methane Leaks

Natural gas pipeline leaks that pose a safety hazard are quickly addressed. But what about leaks too small to pose a threat? These mall leaks are often overlooked and they collectively release tons of methane, a greenhouse gas 84 times more potent than carbon dioxide.

However, thanks to researchers from Colorado State University, the University of Northern Colorado, and Conservation Science Partners—who’ve teamed up with the Environmental Defense Fund—a small unit of Google Street View cars are turning into mobile methane sensors to monitor leaks that have flown under the radar.

A Mobile Amalgamation For Methane Measurement

Lead researcher Joe von Fischer, a biologist by training, originally bought a laser spectrograph, which scans invisible gases that are opaque under infrared light, a decade ago to use on the Arctic tundra. That is, until he one day decided to put it in his car and drive around Fort Collins. He ended up finding a local methane leak with his mobile amalgam of a methane sensor.

“At the same time, Google was interested in putting some of these new methane analyzer analogs in their vehicles, and the Environmental Defense Fund was interested in methane because it’s so poorly quantified,” says von Fischer.  Naturally he was put in charge of the project.

Since then, their collaboration has seen sensor-equipped Street View cars map methane leaks in pipelines beneath the roads of 14 cities throughout America, releasing the data publicly as citywide maps online. The initiative has even helped spur one New Jersey utility provider to commission a $905 million upgrade to gas lines.

Designing a Moral Machine

Back around the turn of the millennium, Susan Anderson was puzzling over a problem in ethics. Is there a way to rank competing moral obligations? The University of Connecticut philosophy professor posed the problem to her computer scientist spouse, Michael Anderson, figuring his algorithmic expertise might help.

At the time, he was reading about the making of the film 2001: A Space Odyssey, in which spaceship computer HAL 9000 tries to murder its human crewmates. “I realized that it was 2001,” he recalls, “and that capabilities like HAL’s were close.” If artificial intelligence was to be pursued responsibly, he reckoned that it would also need to solve moral dilemmas.

In the 16 years since, that conviction has become mainstream. Artificial intelligence now permeates everything from health care to warfare, and could soon make life-and-death decisions for self-driving cars. “Intelligent machines are absorbing the responsibilities we used to have, which is a terrible burden,” explains ethicist Patrick Lin of California Polytechnic State University. “For us to trust them to act on their own, it’s important that these machines are designed with ethical decision-making in mind.”

The Andersons have devoted their careers to that challenge, deploying the first ethically programmed robot in 2010. Admittedly, their robot is considerably less autonomous than HAL 9000. The toddler-size humanoid machine was conceived with just one task in mind: to ensure that homebound elders take their medications. According to Susan, this responsibility is ethically fraught, as the robot must balance conflicting duties, weighing the patient’s health against respect for personal autonomy. To teach it, Michael created machine-learning algorithms so ethicists can plug in examples of ethically appropriate behavior. The robot’s computer can then derive a general principle that guides its activity in real life. Now they’ve taken another step forward.

“The study of ethics goes back to Plato and Aristotle, and there’s a lot of wisdom there,” Susan observes. To tap into that reserve, the Andersons built an interface for ethicists to train AIs through a sequence of prompts, like a philosophy professor having a dialogue with her students.

The Andersons are no longer alone, nor is their philosophical approach. Recently, Georgia Institute of Technology computer scientist Mark Riedl has taken a radically different philosophical tack, teaching AIs to learn human morals by reading stories. From his perspective, the global corpus of literature has far more to say about ethics than just the philosophical canon alone, and advanced AIs can tap into that wisdom. For the past couple of years, he’s been developing such a system, which he calls Quixote — named after the novel by Cervantes.

Riedl sees a deep precedent for his approach. Children learn from stories, which serve as “proxy experiences,” helping to teach them how to behave appropriately. Given that AIs don’t have the luxury of childhood, he believes stories could be used to “quickly bootstrap a robot to a point where we feel comfortable about it understanding our social conventions.”

Any Ban on Killer Robots Faces a Tough Sell

Fears of a Terminator-style arms race have already prompted leading AI researchers and Silicon Valley leaders to call for a ban on killer robots. The United Nations plans to convene its first formal meeting of experts on lethal autonomous weapons later this summer. But a simulation based on the hypothetical first battlefield use of autonomous weapons showed the challenges of convincing major governments and their defense industries to sign any ban on killer robots.

In October 2016, the Chatham House think tank in London convened 25 experts to consider how the United States and Europe might react to a scenario in which China uses autonomous drone aircraft to strike a naval base in Vietnam during a territorial dispute. The point of the roleplaying exercise was not to predict which country would first deploy killer robots, but instead focused on exploring the differences in opinion that might arise from the U.S. and European side. Members of the expert group took on roles representing European countries, the United States and Israel, and certain institutions such as the defense industry, non-governmental organizations (NGOs), the European Union, the United Nations and NATO.

The results were not encouraging for anyone hoping to achieve a ban on killer robots.

Emerging Editing Technologies Obscure the Line Between Real and Fake

The image is modest, belying the historic import of the moment. A woman on a white sand beach gazes at a distant island as waves lap at her feet — the scene is titled simply “Jennifer in Paradise.”

This picture, snapped by an Industrial Light and Magic employee named John Knoll while on vacation in 1987, would become the first image to be scanned and digitally altered. When Photoshop was introduced by Adobe Systems three years later, the visual world would never be the same. Today, prepackaged tools allow nearly anyone to make a sunset pop, trim five pounds or just put celebrity faces on animals.

Though audiences have become more attuned to the little things that give away a digitally manipulated image — suspiciously curved lines, missing shadows and odd halos — we’re approaching a day when editing technology may become too sophisticated for human eyes to detect. What’s more, it’s not just images either — audio and video editing software, some backed by artificial intelligence, are getting good enough to surreptitiously rewrite the mediums we rely on for accurate information.

The most crucial aspect of all of this is that it’s getting easier. Sure, Photoshop pros have been able to create convincing fakes for years, and special effects studios can bring lightsabers and transformers to life, but computer algorithms are beginning to shoulder more and more of the load, drastically reducing the skills necessary to pull such deceptions off.

In a world where smartphone videos act as a bulwark against police violence and relay stark footage of chemical weapons strikes, the implications of simple, believable image and video manipulation technologies have become more serious. It’s not just pictures anymore — technology is beginning to allow us to edit the world.

Microsoft AI Notches the Highest ‘Ms. Pac-Man’ Score Possible

A Microsoft artificial intelligence has achieved the ultimate high score in Ms. Pac-Man, maxing out the counter at just under a million points.

With its randomly-generated ghost movements, Ms. Pac-Man has proven a tough nut for AI to crack, as it cannot simply learn the patterns that govern the ghosts’ movements. Maluuba, an artificial intelligence company recently acquired by the tech giant, succeeded in outwitting the hungry ghosts by breaking their gaming algorithm into around 160 different parts. They say it took less than 3,000 rounds of practice to achieve the feat, something never done by a human. The researchers didn’t even know what would happen when they hit seven figures, Wired reports. They were somewhat disappointed to find the score simply resets to zero.

Arcade games like Pac-Man, along with many other types of games have a nearly infinite amount of possible scenarios. Ms. Pac-Man has somewhere around 1o^77 feasible game states, far too many to map out best practices for each one. Absent a perfect strategy, the next best thing is to narrow down movements that are the least worst for any situation. Maluuba’s AI does this by assigning each edible bar, fruit and ghost its own function, and each and every one gets a voice in determining how Ms. Pac-Man will behave. Their recommendations are weighted, and the final decision comes from a central aggregator which weighs each component and decides what to do. They go into further detail in a paper posted to their website.

This gave it an edge in a game where the movements of life-ending ghosts and reward-giving fruits are partially random, meaning that the algorithm must change its mind on the fly. This is somewhat similar to the challenges faced by poker-playing AI’s, who must deal with both randomness and human irrationality in making their decisions.

While notable, Maluuba’s achievement is for the moment merely a flashy headline. They say that their decentralized approach to AI could see applications in the worlds of sales and finance in the future though, where algorithms must similarly deal with a host of random variables at once. In addition, having decisions made by a collection of actors, instead of just one, makes it easier to track the algorithm’s thinking. This could alleviate pressing ethical concerns regarding a future where life-altering decisions could fall into the hands of AI. Knowing how and why a computer reached a particular decision allows us to better judge whether its thinking lines up with our own morals and values.