Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment

Mads Kock Pedersen; Carlos Mauricio Castaño Díaz; Mario Alejandro Alba-Marrugo; Ali Amidi; Rajiv Vaid Basaiawmoit; Carsten Bergenholtz; Morten H. Christiansen; Miroslav Gajdacz; Ralph Hertwig; Byurakn Ishkhanyan; Kim Klyver; Nicolai Ladegaard; Kim Mathiasen; Christine Parsons; Michael Bang Petersen; Janet Rafner; Anders Ryom Villadsen; Mikkel Wallentin; Jacob Friis Sherson; Skill Lab players

Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment

Mads Kock Pedersen, Carlos Mauricio Castaño Díaz, Mario Alejandro Alba-Marrugo, Ali Amidi, Rajiv Vaid Basaiawmoit, Carsten Bergenholtz, Morten H. Christiansen, Miroslav Gajdacz, Ralph Hertwig, Byurakn Ishkhanyan, Kim Klyver, Nicolai Ladegaard, Kim Mathiasen, Christine Parsons, Michael Bang Petersen, Janet Rafner, Anders Ryom Villadsen, Mikkel Wallentin, Jacob Friis Sherson^*, Skill Lab players

^*Corresponding author for this work

Research output: Working paper/Preprint › Working paper › Research

177 Downloads (Pure)

Abstract

Psychology and the social sciences are undergoing a revolution: It has become increasingly clear that traditional lab-based experiments fail to capture the full range of differences in cognitive abilities and behaviours across the general population. Some progress has been made toward devising measures that can be applied at scale across individuals and populations. What has been missing is a broad battery of validated tasks that can be easily deployed, used across different age ranges and social backgrounds, and employed in practical, clinical, and research contexts. Here, we present Skill Lab, a game-based approach allowing the efficient assessment of a suite of cognitive abilities. Skill Lab has been validated outside the lab in a crowdsourced population-size sample recruited in collaboration with the Danish Broadcast Company (Danmarks Radio, DR). Our game-based measures are five times faster to complete than the equivalent traditional measures and replicate previous findings on the decline of cognitive abilities with age in a large population sample. Furthermore, by combining the game data with an in-game survey, we demonstrate that this unique dataset has implication for key questions in social science, challenging the Jack-of-all-Trades theory of entrepreneurship and provide evidence for risk preference being independent of executive functioning.

Original language	English
Publisher	arxiv.org
Number of pages	11
Publication status	Published - Sept 2020

Keywords

physics.soc-ph
econ.GN
q-fin.EC

Access to Document

2009.05274v1Submitted manuscript, 858 KB

Cite this

Pedersen, M. K., Díaz, C. M. C., Alba-Marrugo, M. A., Amidi, A., Basaiawmoit, R. V., Bergenholtz, C., Christiansen, M. H., Gajdacz, M., Hertwig, R., Ishkhanyan, B., Klyver, K., Ladegaard, N., Mathiasen, K., Parsons, C., Petersen, M. B., Rafner, J., Villadsen, A. R., Wallentin, M., Sherson, J. F., & players, S. L. (2020). Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment. arxiv.org.

@techreport{452916a9308b439196606476d81aff3c,

title = "Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment",

abstract = " Psychology and the social sciences are undergoing a revolution: It has become increasingly clear that traditional lab-based experiments fail to capture the full range of differences in cognitive abilities and behaviours across the general population. Some progress has been made toward devising measures that can be applied at scale across individuals and populations. What has been missing is a broad battery of validated tasks that can be easily deployed, used across different age ranges and social backgrounds, and employed in practical, clinical, and research contexts. Here, we present Skill Lab, a game-based approach allowing the efficient assessment of a suite of cognitive abilities. Skill Lab has been validated outside the lab in a crowdsourced population-size sample recruited in collaboration with the Danish Broadcast Company (Danmarks Radio, DR). Our game-based measures are five times faster to complete than the equivalent traditional measures and replicate previous findings on the decline of cognitive abilities with age in a large population sample. Furthermore, by combining the game data with an in-game survey, we demonstrate that this unique dataset has implication for key questions in social science, challenging the Jack-of-all-Trades theory of entrepreneurship and provide evidence for risk preference being independent of executive functioning. ",

keywords = "physics.soc-ph, econ.GN, q-fin.EC, Cognition, Game-based, citizen science",

author = "Pedersen, {Mads Kock} and D{\'i}az, {Carlos Mauricio Casta{\~n}o} and Alba-Marrugo, {Mario Alejandro} and Ali Amidi and Basaiawmoit, {Rajiv Vaid} and Carsten Bergenholtz and Christiansen, {Morten H.} and Miroslav Gajdacz and Ralph Hertwig and Byurakn Ishkhanyan and Kim Klyver and Nicolai Ladegaard and Kim Mathiasen and Christine Parsons and Petersen, {Michael Bang} and Janet Rafner and Villadsen, {Anders Ryom} and Mikkel Wallentin and Sherson, {Jacob Friis} and players, {Skill Lab}",

note = "11 pages, 4 figures, and 2 tables",

year = "2020",

month = sep,

language = "English",

publisher = "arxiv.org",

type = "WorkingPaper",

institution = "arxiv.org",

}

Pedersen, MK, Díaz, CMC, Alba-Marrugo, MA, Amidi, A , Basaiawmoit, RV , Bergenholtz, C , Christiansen, MH, Gajdacz, M, Hertwig, R, Ishkhanyan, B, Klyver, K, Ladegaard, N , Mathiasen, K , Parsons, C , Petersen, MB , Rafner, J , Villadsen, AR , Wallentin, M , Sherson, JF & players, SL 2020 'Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment' arxiv.org.

TY - UNPB

T1 - Measuring Cognitive Abilities in the Wild

T2 - Validating a Population-Scale Game-Based Cognitive Assessment

AU - Pedersen, Mads Kock

AU - Díaz, Carlos Mauricio Castaño

AU - Alba-Marrugo, Mario Alejandro

AU - Amidi, Ali

AU - Basaiawmoit, Rajiv Vaid

AU - Bergenholtz, Carsten

AU - Christiansen, Morten H.

AU - Gajdacz, Miroslav

AU - Hertwig, Ralph

AU - Ishkhanyan, Byurakn

AU - Klyver, Kim

AU - Ladegaard, Nicolai

AU - Mathiasen, Kim

AU - Parsons, Christine

AU - Petersen, Michael Bang

AU - Rafner, Janet

AU - Villadsen, Anders Ryom

AU - Wallentin, Mikkel

AU - Sherson, Jacob Friis

AU - players, Skill Lab

N1 - 11 pages, 4 figures, and 2 tables

PY - 2020/9

Y1 - 2020/9

N2 - Psychology and the social sciences are undergoing a revolution: It has become increasingly clear that traditional lab-based experiments fail to capture the full range of differences in cognitive abilities and behaviours across the general population. Some progress has been made toward devising measures that can be applied at scale across individuals and populations. What has been missing is a broad battery of validated tasks that can be easily deployed, used across different age ranges and social backgrounds, and employed in practical, clinical, and research contexts. Here, we present Skill Lab, a game-based approach allowing the efficient assessment of a suite of cognitive abilities. Skill Lab has been validated outside the lab in a crowdsourced population-size sample recruited in collaboration with the Danish Broadcast Company (Danmarks Radio, DR). Our game-based measures are five times faster to complete than the equivalent traditional measures and replicate previous findings on the decline of cognitive abilities with age in a large population sample. Furthermore, by combining the game data with an in-game survey, we demonstrate that this unique dataset has implication for key questions in social science, challenging the Jack-of-all-Trades theory of entrepreneurship and provide evidence for risk preference being independent of executive functioning.

AB - Psychology and the social sciences are undergoing a revolution: It has become increasingly clear that traditional lab-based experiments fail to capture the full range of differences in cognitive abilities and behaviours across the general population. Some progress has been made toward devising measures that can be applied at scale across individuals and populations. What has been missing is a broad battery of validated tasks that can be easily deployed, used across different age ranges and social backgrounds, and employed in practical, clinical, and research contexts. Here, we present Skill Lab, a game-based approach allowing the efficient assessment of a suite of cognitive abilities. Skill Lab has been validated outside the lab in a crowdsourced population-size sample recruited in collaboration with the Danish Broadcast Company (Danmarks Radio, DR). Our game-based measures are five times faster to complete than the equivalent traditional measures and replicate previous findings on the decline of cognitive abilities with age in a large population sample. Furthermore, by combining the game data with an in-game survey, we demonstrate that this unique dataset has implication for key questions in social science, challenging the Jack-of-all-Trades theory of entrepreneurship and provide evidence for risk preference being independent of executive functioning.

KW - physics.soc-ph

KW - econ.GN

KW - q-fin.EC

KW - Cognition

KW - Game-based

KW - citizen science

M3 - Working paper

BT - Measuring Cognitive Abilities in the Wild

PB - arxiv.org

ER -