Saturday, May 3, 2014

Serious Games Analytics Delivering Value To Play-Learners

Identifying appropriate value metrics within playful learning environments

Related Posts:

Late January, Dr. Christian Sebastian Loh and Dr. Yanyan Sheng from the Southern Illinois University at Carbondale, IL, and Dr. Dirk Ifenthaler from Deakin University, Melbourne, Australia issued the Call for Chapters: Serious Games Analytics as the first edited volume in the Advances in Game-Based Learning series.  The edited volume is scheduled to be published in 2015.

Their declared motivation for this call is their belief that although, on the surface, it appears that either game analytics, or game + learning analytics, would be directly applicable to Serious Games, this is yet to be proven because the analytics from one industry may not easily transfer to another.

They have coined the word play-learners to mean players who are also learners (or vice versa) and state that by observing the actions of play-learners within a Serious Game environment and understanding their decision-making processes that one can identify appropriate metrics for assessment and improvement.

The purpose of this edited volume is to collect in one place how gameplay data in Serious Games may be captured and converted into valuable analytics or actionable intelligence. The editors would like this book to become the ‘go-to’ reference for empirical research methodologies for Serious Games analytics from various fields of expertise.

Analogy to Business Intelligence

Analytics are an important source of Business Intelligence (BI), across all levels of a company or organization irrespective of its size. BI is a broad concept, but the basic goal is to acquire data and turn it into useful information. The goal is to support decision-making – to make it data-driven rather than reliant on gut instinct. 

What these definitions imply is that there are resources that are there, if only they can be tapped and refined, and then transported to the level of awareness where the organization can use it.   

Similarly to business analytics as a subset of BI are Serious Games Analytics as a source of Learning Intelligence (LI).

Developing Students’ Own Learning Intelligence - A Passion For Personalized Learning

Serious Games deployment philosophy is that children learn best by playing, discovering and problem-solving.  Rather than simply talking at children and telling them what to learn, the children are empowered to customize their own path and develop meaningful relationships with subject material.

PlayMaker is a school that has been designed and developed by the GameDesk Institute funded by the Bill and Melinda Gates Foundation and launched in partnership with New Roads Schools.

 Image courtesy of PlayMaker school

The school aims to give children hands-on, play-based experiences that engage learners and give them a deeper understanding of the subject matter context.  They have re-worked learning spaces, grade levels, and traditional subject areas to create a highly engaging, exciting and 21st century school.  

How Does This Work At A Student Level?

Students are in control of their own learning.  They have an ‘Adventure Map’ which is essentially their curriculum but one which they can interact with, personalize and reflect upon.  The Map is made up of curriculum units, which explore a certain subject area in a cross-curricula and project based way.  For example, the roller coaster unit teaches students about friction, speed and distance, kinetic and potential energy, measurement and digital film-making through various exercises including building a physical roller coaster, creating a digital roller coaster and making a short film.  

Image courtesy of PlayMaker school

The modules are made up of elements that work for different types of learners.  Once students have completed a module, they ‘unlock’ new areas of the curriculum

Image courtesy of PlayMaker school

On completing their Adventure Maps, students can see connections between modules and reflect upon their learning (their choices, their progress, standards they have covered etc.). 

The curriculum maps to U.S. standards but is also focused on competencies (which I’m taking the liberty to address as “New Learning Values”) that the PlayMaker school deems important such as systems thinking, problem solving, social-emotional learning, critical thinking, collaboration and other life and career skills.  The school is open source so that the community of teachers and schools can learn from the school (its successes and mistakes).  

Analytics May Be Everything (But What Are We Measuring?)

Designing Digitally, in the recent must-read article Serious Games - Analytics Are Everything!, emphasizes the use of analytics as an important aspect in Serious Game development. “Analytics are basically actionable insights, which means they allow you to take a deeper look at the overall performance of a Serious Game and determine whether or not it is meeting your game objective.”

“Analytics collect data and information about the user's experience throughout the game. This information indicates which tasks, rewards, and challenges are helping the learners accomplish the overall objective of the game and which areas they might be struggling with”, the article proceeds.

The article also provides examples of the numerous ways developers can collect data, among them, determining whether or not a user understands the game mechanics, notating phase changes and logging user performance when a user hits a certain milestone.”

The question here is whether analytics are capturing the value metrics that underlie experiential learning, which are context driven and not necessarily quantifiable.

Aligning The Analytics To New Learning Values

In his most recent article How Games Lead Kids to the Good Stuff: Understanding Context, Jordan Shapiro states that “those who still think of content as the driving force of education may not be ready for game-based learning. In the classroom, what we usually call content is what students have retained if teachers have met their learning objectives.”

“Game-based learning is a great classroom tool because it allows for interdisciplinary learning through contextualized critical thinking and problem solving”, he adds. “Games in the classroom can encourage students to understand subject matter in context — as part of a system. The content becomes useful insofar as it plays a part in a larger multi-modal system.”

“And that’s the crux of the value: Games are all about systems thinking. In order to beat the game, the player needs to master the system”, he says.

The value creation concept, as opposed to content driven assessment, is often difficult to convey but once we agree upon what constitutes value and the metrics to be used, it becomes one of the best anchors to institutionalize discontinuity.

Here are three examples

The Value of Experimentation

One of the most profound transformations we can learn from games is how to turn the sense that someone has ‘failed’ into the sense that they ‘haven’t succeeded yet’.

At PlayMaker school, for example, failure exists as a meaningful opportunity for success. Students are able to experiment with new ideas again and again, allowing them to embrace innovation through developing their own creative solutions. Through experimental play, students are able to understand how the process of the scientific hypothesis works.

Play-learners resilience in pursuit of epic wins could be logged, notated and rewarded, if we agree at the institutional level that failure combined with experimentation constitutes a value in the Serious Games learning environment.

The Value of Collaborative Play to Solve Problems

Via: Agile Coaching Blog

So we have problems to solve and domains to explore. In many modern, complex business domains, no one person could hold a view of the entire problem. Chances are that if they see the whole thing, they are not in the trenches and only have an abstract view of what’s going on.

A further challenge is that even if you have an expert who perfectly grasps the domain, a growing body of research shows they will not be able to easily articulate that understanding to others (implicit x explicit knowledge).  

Fundamental value is derived from the interplay and increased perspective we get by having multiple people interact.

A huge part of the value discovery emerges from the interplay of multiple people simultaneously playing with the same concept

One of the most powerful demonstrations of this is the “wilderness survival” exercises. The game comes in different varieties, but the general structure is this: you are stranded remotely in a forest / arctic tundra /tropical island and you must prioritize from a list of possible things to do which ones you would do and in what order. First set your priority as an individual, then do it as a group. After doing this, each group gets the correct answers based on wildlife survival experts. What’s most interesting is that in almost every version of this game, the group rankings are better than every single individual decisions.

The act of “playing” together produces substantively better ideas that could also be logged, notated and rewarded if, again, we agree at the institutional level that collaborative play to solving problems constitutes a value in the Serious Games learning environment.

The Value of A Multidisciplinary Experience

While learning has been historically four-walled into separate and distinct subjects, gameplay cuts across multiple disciplines.

Serious Games environment could become a perfect vehicle for connecting and relating components of the curriculum in a meaningful, project-based way for both students and teachers.

Curriculum integration levels could also be logged, notated and rewarded if, again, we agree at the institutional level that multidisciplinary experience constitutes a value in the Serious Games learning environment.