Does Changing the Game Metaphor Change the Assessment?
Abstract
This study examined whether altering the metaphor—or visual skin—of a cognitive assessment game affects its psychometric properties. BrainTagger is a suite of serious games used to assess executive functions such as processing speed, response inhibition, cognitive flexibility, and working memory. While the original version used a Whackamole metaphor, user feedback prompted the development of an alternative Gardening version with identical mechanics but a more mature visual theme. Using a within-subject design, 109 undergraduate participants completed both versions of 4 BrainTagger games. Correlation analyses revealed significant relationships between versions across all domains, particularly for working memory ( r = .38), response inhibition ( r = .36 for d-prime, −.75 for median correct RT), cognitive flexibility ( r = .37), and processing speed ( r = .61). However, the speed accuracy tradeoff in working memory performance was sensitive to metaphor changes, likely due to task demands requiring either the phonological loop or the visuo-spatial sketchpad. These findings suggest that while game metaphors can be adapted to suit user preferences without broadly compromising assessment validity, specific cognitive domains—such as working memory—may be more vulnerable to metaphor-driven variance. Implications for the design and validation of cognitive assessment tools are discussed.
Authors: Yifei Zhou, You Zhi Hu, Zichun Xu, Jinfeng Wu, Mark Chignell
Published in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting (2025)