Can a game track implicit learning of complex information?

How do brain injury survivors respond to a game-i-fied assessment of implicit learning?

SCOPING THE RESEARCH

Goals: 

Let's watch that learning curve!

Clinicians have to pick evidence-based treatment options that they are confident will work for their clients and efficiently use the somewhat limited treatment time that they have. A common treatment approach is to teach someone to use a new assistive tool, like a calendar planning app. It works really well for some folks but not others, and for those that don't respond, there can be a lot of time wasted that could have been avoided.

By providing a metric of strong implicit learning after brain injury, we can identify the subset of people that would respond to treatment that relies on the same type of learning strategy. And, we can use a different learning strategy for those that don't show strong implicit learning.

Existing Evidence:

Implicit learning is known to be more robust after brain injury so it is used in a variety of treatment approaches: errorless learning, spaced-retrieval training, constraint therapy

Competitor analysis revealed that existing learning assessment tools are either 1. not designed to test implicit learning or 2. not designed for clinicians to implement. They are primarily research oriented tools.

Existing research tools also do not automate complex learning, just simple repeated patterns.

Implicit learning methods in research are not adapted for clinical use. They require offline scoring or time-consuming analytics that therapists would not have time to implement.

Goals of the research:

1. Develop a useable prototype that measures learning
2. Refine the usability of the game prototype

---

Research Questions

1. How do players describe their attitudes about the game?
2. How do players attitudes vary based on their success within the game?
3. Are their any features of the game that interfere with player interactions?
4. Are their any features of the game that interfere with learning?
5. Do metrics reflect simple repeated patterns or more complex learning of rules?

Survey: Game Images

• Validation that images matched possible answer choices in game
• Attitudes towards images planned for game use
• Identification of images that would interfere with learning
• Preferences towards particular images to be included in game

Survey respondents: 

• Ages 20-35 and 50-80
• English speaking
• Gender balanced

Game Usability Testing + Interviews + Game Analytics

45-60 minute structured interviews following game play

• Medical and learning history
• History with cognitive rehabilitation or language therapy
• Overall game experience
• Determination if participants guessed the 'secret' purpose of the game too early
• Motivation for participation in the study
• Participants with aphasia interviewed using supported communication techniques

Game Task

• Sort the cards as quickly as possible to match the images

Participants:

• Primary subgroup of participants with aphasia (language impairment) due to stroke
• All participants matched for age range: 50-80 years
• English speaking
• Gender balanced to match participants with aphasia

Game analytics

• Overall successful learning of hidden rules by end of game
• Trajectory of learning over time: faster or slower or no evidence of learning
• Overall speed of responses

‍

Strategic Impact

Understanding the impact of design choices on game success and participant attitudes supports actionable insights:

1. Recommendations for design and developers when building game interface and interaction.
2. Identification of learning curve metrics that will result in higher accuracy in predicting clinical outcomes.

Research Plan

The planned project period spanned 2 years

1. Year 1
   Communication with stakeholders
   Image Design Survey
   Survey Analysis
   Initial Game Development
   ‍
2. Year 2
   Communication with stakeholders
   Game usability studies + interviews
   Analysis of usability data + iterative design
   Game analytic analysis
   
   Presentation of findings
   ‍

Key Findings

1. Images that players could verbally describe distracted them from the learning goal
2. Game players preferred a simpler keypress interface vs. the flashier drag-and-drop design
3. Players learned simple patterns first and complex patterns more slowly
4. Better pattern learning related to later rule knowledge

Most important take-away: In-game learning metrics can track more than just simple single button press learning! Focus should be on specific metrics validated against therapy outcomes.

‍

Recommendations and Impact

1. Simplify the interaction with the game.
   Initial drag-and-drop design was seen as fancy but tiring by players, and several players stopped playing before they could learn the rules. Other players slowed too much for the game to track their learning.
   Most players preferred button presses over touch-screen interaction and button presses didn't interfere with the learning metrics.
   ‍
2. Identify and develop different specific metrics that are validated against specific therapy outcomes and clinician goals.
   Different clients may have different therapy goals and game play supports multiple types of metrics. Could draw clear relationships between game play and therapy planning for clinicians.
   The learning trajectory in game matches later rules knowledge and some players take longer to show that learning. Game play could be stopped once a certain learning level was reached.
   ‍

‍