Monday, December 9, 2013

MACBETH: Serious Games To Train Future Intelligence Analysts

Serious Games helping intelligence teams learn to take cognitive biases out of their decision making


Related post:

MACBETH, developed by the University of Oklahoma (OU), is the winner of both the “Best Business Game” and “Best Game Special Emphasis Adaptive Force Award” (judged by the Office of the Secretary of Defense among others), two of the seven award categories of the Serious Games Showcase & Challenge, at I/ITSEC 2013 hold last week in Orlando, Fl.

MACBETH, which stands for Mitigating Analyst Cognitive Bias by Eliminating Task Heuristics, is a Serious Game created to train intelligence analysts and measure their proficiency in recognizing and mitigating the cognitive biases that affect intelligence analysis.

The objective of the game is to improve accuracy of credibility assessments, working as a learning system where players must counteract threats to American interests by using the intelligence data at hand without allowing their cognitive biases to cloud their judgment.

The game simulates a crime investigation, giving players a group of suspects and information to help decide who committed the crime. The player guides agents as they collect information, and then decides whether that information was affected by certain types of cognitive bias.


Through gameplay, the system will highlight the use of a bias, then provide the player with information on and opportunities to practice bias mitigation techniques based on the theoretical model, the Heuristic-Systematic Model of information processing.

The game is designed to test players for three kinds of bias: anchoring bias, projection bias and representativeness bias. Anchoring bias causes the player to rely too heavily on the first piece of information he or she receives. Projection bias is a case where the player projects his or her own thought patterns onto another person, assuming they will be the same. A player operating under representativeness bias might judge an individual person based on stereotypes about a larger group.

The uniqueness of MACBETH approach can be experienced at the different challenge levels as players collect information, review Intel and formulate hypothesis: game goal can only be achieved if players take cognitive biases out of the equation and make better decisions.

Norah Dunbar, associate professor in the OU Department of Communication and the Center for Applied Social Research, and the primary investigator responsible for project oversight, says the team modeled MACBETH after the game “Clue” because it is familiar to most people.  With MACBETH, players spend less time learning the game and more time solving the problem.  Like Clue, MACBETH gives player(s) a group of suspects to choose from and clues that help the player decide who has committed the crime.  Mentors guide a player by providing insight as to what a player did wrong.

The game is already getting interest from several federal agencies.

Although MACBETH was designed with the intelligence community in mind, Dunbar said the same model could be used in other areas. For example, it could be useful for making medical students aware of their biases when diagnosing patients. “I think it's pretty broadly applicable,” she added.




Game Background

The OU team was one of six teams selected to create a video game for the Air Force Research Laboratory in support of the Intelligence Advanced Research Projects Activity (IARPA), the federal agency that funds research and development for the intelligence field.

The agency was interested in developing a program that helped mitigate the kinds of cognitive bias that were most problematic in the intelligence community. After developing the first version of the game, Dunbar's team tested it on a group of undergraduates. One group played the game, while another group watched a video about cognitive bias. A total of 2,000 subjects were tested both at OU and the University of Arizona. 

When researchers gave the participants a follow-up exam on cognitive bias eight months later, the students who had played the game had retained the information better than those who had watched the video.