Motion data transformed: Markerless motion capture brings new insights into old data.

• In Event: Measurement Issues in Developmental Research

Abstract

Motion capture data has long been used to gain insights into motor processes, from tracking infants’ walking strategies (Snapp-Childs & Corbetta, 2009) to gait-analysis of individuals with cerebral palsy (van den Noort et al., 2012). However, research investigating complex psychological processes has not yet fully taken advantage of motion capture due to the challenges inherent in coding motion data by hand and/or the expense of larger scale motion tracking systems.

Advancements in artificial intelligence are removing these barriers. DeepLabCut is an open source software that allows users to perform post-hoc analyses using a neural network which can be trained accurately within a few hours, without specialized equipment (Mathis et al., 2018). After training, participant videos can be run through the trained neural network allowing researchers to gain new insights into old data. We will present two case studies in which we use DeepLabCut to analyze spatial variability in infants’ hand movements to gain insights into the developmental processes of persistence and generosity, respectively.

In the first case study, we applied DeepLabCut to videos collected from a study by Lucca et al. (2020) in which infants (n = 96; MAGE = 18.5; 38 female) watched an adult solve or attempt to solve a means-end problem (pull a rope in order to retrieve an out-of-reach toy) and then were given the opportunity to solve the problem. Unbeknownst to them, the problem was impossible as the toy was affixed to the table. Lucca et al. (2020) found greater persistence by infants after watching an actor struggle and then solve the problem than when the actor easily solved the problem, or failed to solve the problem. Spatial variability in pulling was derived to test the hypothesis that infants will increase variability in pulling attempts after watching an experimenter attempt but fail, compared to watching an experimenter successfully pull the rope. Thus, the application of DeepLabCut will reveal new information regarding infants’ persistence by documenting the strategies underlying infants’ persistence.

Previous work by Lockwood et al. (2017) found that adults are less willing to exert effort to help others, compared to themselves. While young children exhibit prosocial actions, it remains unclear if they exhibit a similar bias. In the second case study we leveraged videos from an ongoing study in which children are told their claps gain rewards for themselves or others. DeepLabCut was deployed to measure the distance between hands during claps to test the hypothesis that children mitigate incurred cost by offering smaller claps for another person. While human coders can count claps, using DeepLabCut we are able to gain fine-grained insights into the cost-benefit analysis involved in young children’s prosociality.

These two case studies showcase how new markerless motion capture technology can be used to harness a previously untapped trove of nuanced, yet objective, psychological data. The lack of specialized equipment needed and the highly customizable nature of the software allows researchers to facilitate important scientific discoveries even while face-to-face data collection is not possible.

Authors

• Hannah Solby, University of Toronto
  Presenting Author

• Mia Radovanovic, University of Toronto
  Non-Presenting Author

• Rachel Horton, University Of Washington
  Non-Presenting Author

• Kelsey Lucca, Arizona State University
  Non-Presenting Author

• Jessica A Sommerville, University of Toronto
  Non-Presenting Author