Sunday, August 2, 2015

Games Have The Potential To Be Assessment Without The Test

In a recent article by Lee Banville, editor of, Dr. Kristen DiCerbo, director of eLearning at the Pearson Research & Innovation Network, states that “Games have the potential to be assessment without the test.”

DiCerbo's research program centers on digital technologies in learning and assessment, particularly on the use of data generated from interactions to inform instructional decisions. A senior research scientist, she has conducted qualitative and quantitative investigations of games and simulations, particularly focusing on the identification and accumulation of evidence.

DiCerbo admits that stealth assessments woven 'invisibly' into digital gaming environments is “a radical idea”, but it is one she believes in and with her background in the data and assessment tools available she thinks she has that potential to make it happen. She also believes we are at a key inflection point and we will arrive at a place technologically where we can gather data to build assessments without administering tests as we know them.

In games, educators can observe a student's sequence of actions, time spent on tasks, multiple attempts at activities, requests for help, communication process, etc. In other words, games allow us to examine a student's process of problem solving, not just the final product at the end. These observations can help educators make valuable inferences regarding students' mastery over skills, while offering new ways to assess factors not easily measured on multiple-choice tests, such as problem solving, critical thinking, collaboration, persistence, and creativity.

“It is important to note that the term invisible does not imply that learners or teachers do not know that assessment is happening. Rather, it implies that the actual activity of assessment is not visible, or interrupting the classroom,” DiCerbo says.

Dr. DiCerbo states that a number of quality games exist today that successfully balance these factors and have tremendous potential as tools for both learning and assessment. However, much work is still needed to maximize their value, particularly in the area of integration.

Assessment can be largely facilitated depending on how well the game elements (engagement) and instructional elements (potential to teach/inform) are integrated within the game. When seamlessly integrated, this connection becomes transparent to the player.

She believes that the assessment potential of games will continue to grow as game developers work ever more closely with education experts.

The “Invisible” Stealth Assessment Gaining Ground

Dr Val Shute, the Mack and Effie Campbell Tyner endowed professor of education at Florida State University, and former principal research scientist at ETS, coined the term "stealth assessment" to describe this approach.

She will give a keynote presentation at ACER's Research Conference 2015 in August on how games designed with stealth assessment capabilities can measure and assess 21st Century competencies. The theme of this year's Research Conference, to be held August 16-18, is Learning Assessments: Designing the Future.

Dr Shute notes that the traditional approach in schools usually divorces assessment from learning. “The typical educational cycle is: teach; stop; administer test; teach, with new content,” Dr Shute said, speaking ahead of the conference.

Dr. Shute promotes the idea of "stealth assessments" that are unobtrusively woven into the fabric of video games in order to measure in real time how players are progressing toward a variety of targets—not just for content learning, but also for checking progress on so-called 21st century skills.

"There's no better system than games to provide challenging problems without completely overwhelming people," she says. "Stealth assessment considers the process of failing and trying again as a source of evidence for persistence."

Here's how the process is intended to work: First, a game will track in real time how long a player spends on a difficult problem or how often a player is willing to retry a problem at which he or she had previously failed. Based on that information, the game can then offer encouragement or adjust the types of problems presented to that player, seeking to find the sweet spot that will be challenging enough to promote learning but not so difficult as to discourage continued effort.

The first round of such games have already been developed, including Newton's Playground, further renamed to Physics Playground, a digital game created by Dr Shute with support from the Seattle-based Bill & Melinda Gates Foundation.

Newton’s Playground is a computer-based, two-dimensional game that requires the player to guide a green ball to a red balloon. The player can nudge the ball to the left and right (if the surface is flat) but the primary way to move the ball is by drawing/creating simple machines (which are called “agents of force and motion” in the game) on the screen that “come to life” once the object is drawn. Everything obeys the basic rules of physics relating to gravity and Newton’s three laws of motion. The stealth assessments that are directly built into the game capture and analyze the log data produced by the player’s interactions with the game.

"Stealth assessment is intended to be invisible and ongoing, to support learning, and remove or seriously reduce test anxiety while not sacrificing validity and consistency," Dr Shute said.

"With stealth assessment, schools no longer have to interrupt the teaching and learning process to administer tests. Instead, assessment is continual and invisible to students, supporting real-time, just-in-time instruction."