Discovering Temporal Dependency and Directional Influence among Multimodal Behavioral Variables with Granger Causality

• In Event: 1-044 - Finding Structure in Time: Computational Methods for Characterizing the Micro-dynamics of Interaction

Integrative Statement

The world of child’s development is profoundly multimodal (Smith & Gasser, 2005). Multiple time-locked sensory-motor systems enable the child’s cognitive system to explore and educate itself. In psychology experiments, we often collect multiple behavioral variables both intrapersonally (within the child) and interpersonally (between the child and their social partner). This poses a major methodological challenge to developmental researchers: when multiple variables are interdependent on each other, how to quantify the temporal dependency and directional influence from one variable to another within a system?

Here we propose a novel approach to quantifying the directional influence among multimodal interdependent behavioral variables with Granger causality. Granger causality is a well-established and effective method for the investigation of directional relationships among a set of interdependent variables in many domains (Bressler & Seth, 2011). Granger (1969) formalized the basic idea of causality between signals introduced by Wiener (1956) based on multivariate autoregressive (MVAR) models. For example, X and Y are two interdependent processes in a system, H. If past values of Y contain unique information that helps to predict X above and beyond the information contained in the history of all other variables, then Y is said to Granger-cause X.

In this presentation, I will explain Granger Causality and multivariate autoregressive models both at the conceptual level and within the context of an empirical developmental study. In this study, we invited dyads of infants and parents to participate in a toy-play experiment. The dyad’s eye movements were recorded by head-mounted eye-trackers (Franchak, Kretch, Soska, & Adolph, 2011) at 30 frames per second (see Figure 1a). An additional bird-eye camera captured their manual activities from above at the same sampling rate. Thus, four behavioral variables were collected for each dyad: child gaze, child manual activity, parent gaze and parent manual activity. Then, we used Granger causality to extract the directional influences from each of these four variables to another. Figure 1b shows the collected behavioral streams. Figure 1c shows an example of the quantified interaction at two different ages from the same dyad. This study will showcase how the technique can be used to investigate the development of child-parent multimodal coordination. It shows the promise of using Granger Causality as a general approach to quantifying relations among multimodal behaviors in developmental science.

I will also provide demo cases and visualization to enhance the audience’s understanding of Granger causality. All scripts and Matlab toolbox for computing Granger causality will be provided on Github.

Author

• Tian Linger Xu, Indiana University Bloomington
  Presenting Author