Poster #118 - Automated Detection of Infant Holding and Carrying Behaviors via Body-Worn Motion Sensors

• In Event: 3-053 - Poster Session 10
  In Poster Session: PS 10 Section - Methods, History, Theory

Integrative Statement

Physical touch is an important modality of coordination between parents and their infants. Physical contact is implicated in bonding and attachment and has important consequences for the development of autonomic nervous system and physical growth (Field, Diego et al. 2010).

This poster will describe the feasibility of using wearable motion sensors to automatically detect and distinguish various types of physical contact between caregivers and their infants. Activity recognition from such “body-worn” accelerometers is common in the field of ubiquitous computing (Bulling, Blank et al 2014), where, for example, similar technology is used to translate motion signals into steps, or to identify episodes of eating over the course of the day.

We will present data from an ongoing study of (current N=10, expected N=24) mother-infant pairs (Mean age= 5.59 months, SD = 2.07 Range=3.5mo-10mo) wearing a motion sensor suite in a video-taped home session, including standard free-play and behavioral reactivity (e.g. Kagan REF) tasks, as well as an “around the house” task where parents mimic typical caregiving activities in various locations around their own homes. This protocol provided a variety combination of natural physical contact and proximity behaviors. Mother and infant participants wore wireless motion sensors capturing continuous accelerometer (25Hz) and barometric pressure (12Hz) datastreams. Trained coders annotated eight states of physical contact and proximity across the entire recorded session (Mean: 52 minutes, SD: 15 minutes): holding, carrying, bouncing, picking up, putting down, touching, hovering and nearby/no physical contact (kappa = .735).
Initially, we attempted to distinguish between all eight physical contact activities at one second resolution using classic activity recognition methods. In particular, we used a supervised learning approach to train various machine learning algorithms (e.g. Random forest and support vector machines) to learn relations between motion signal features (e.g. mean, variance and derivative of motion and BP data at each second) and all annotations split into 1-second bins. However, the predictions were highly segmented and accuracy was unacceptably low. In particular, precision was 18% and recall was 21% for an 8 state classification. However, additional modeling attempts indicate we can achieve high accuracy (upwards of 80%) for a binary classification of holding (a combined state including holding, carrying, bouncing, picking up, putting down annotations) vs. not holding (combining touching, hovering and nearby/no physical contact annotations). Accuracy varies according to the desired temporal precision of the detected activity, which should inform and be informed by the intended use of the algorithms (Bakeman & Quera, 2011). We provide five possible user case scenarios for activity recognition with varying degrees of precision and report accuracy assessments for each one, detailing implications for each scenario. For example, a given model may have unacceptably low accuracy for precise onset and offset for individual holding episodes but accurately predict of the cumulative time infants are held over 24 hours.

Space permitting, we will also describe feasibility of applying our trained algorithm to unlabeled sensor data collected during 24-72 hour home recording sessions in which parents and infants continuously recorded their typical daily home activities using the motion sensors alone (i.e. no video).

Authors

• Yao Xuewen, The University of Texas at Austin
  Presenting Author

• Thomas Ploetz, Georgia Institute of Technology
  Non-Presenting Author

• Mckensey Johnson, The University of Texas at Austin
  Non-Presenting Author

• Kaya de Barbaro, The University of Texas at Austin
  Non-Presenting Author