Measuring Emotions in Children: An Analysis of Emotions Observed in Real Time During a Narrative Task

• In Event: 25.023 - Show...and Tell? Using Behavioral Observation to Study Affective Processes in Educational Settings

Abstract

Emotions significantly impact learning, as they direct attention, shape thoughts, and affect student well-being and achievement. Academic emotions are dynamic and interdependent (D’Mello & Graesser, 2012), and are influenced by perceived control and value placed on success (Pekrun, 2006). Positive emotions are favourable on a number of learning tasks, as they broaden attention and facilitate global information processing (Fredrickson, 2001), while negative emotions narrow attention and produce task-irrelevant thoughts (Eysenck, Derakshan, Santos, & Calvo, 2007; Pekrun & Perry, 2014). While there have been extensive developments in the methods used to study emotions, there are many factors that confound measurement with children. Self-reported accounts of emotions rely on working memory and metacognitive awareness of previous states, which continuously develops until adolescence (Lagattuta, 2014). To address these issues, this study utilized intelligent software that recognizes the presence and intensity of facial expressions in real-time. The hypotheses were that negative emotions would inhibit performance on a narrative storytelling task, and negative emotions would be elicited by specific elements of the task.
This sample consisted of 212 students (Mage=8, SD=0.85, 45% Male). Emotions were measured using Emotient (iMotions) software, which utilizes Support Vector Machine (SVM) analytics to recognize patterns in in facial expressions (Kanade, Cohn, & Tian, 2000). The output is a log-likelihood that emotions are being expressed over multiple frames per second (Littlewort et al., 2011). Participants completed a narrative storytelling task, in which a story corresponding to a wordless picture book was spontaneously created. Stories were evaluated based on an adapted coding scheme (Table 1; Reilly, Losh, Bellugi, & Wulfeck, 2004). The task was completed on a computer in front of a web-camera. To eliminate noise from extraneous facial muscle movement, videos were segmented to exclude portions in which the participant was speaking. Emotion scores were averaged across the number of video frames, by each individual page of the book (Mframe= 78, SD=6.8).
Regression analyses were conducted on emotions and overall narrative scores, and a linear mixed effect model (LMM) was conducted to account for the variation within subjects, and pages of the book. Results demonstrated that frustration and anger negatively predicted narrative storytelling (Figure 1). Results of the LMM demonstrated that there was a significant effect of individual book page on emotion scores for frustration and anger (Figure 2). Pairwise comparisons across individual pages revealed that negative emotions were elicited on pages with congruent story components.
This study utilized an unobtrusive approach to measuring emotions in real-time, and found that negative emotions were negatively related to performance. This finding suggests that when using video data, negative emotions are more influential and observable than positive emotions. Multilevel modelling analyses revealed that changes in emotional expressions were congruent with specific content within the book, such that negative emotions were elicited during negatively valenced story components. This finding localizes a specific predictor of negative emotions in reading material. When regulating students’ emotions, scaffolding strategies should be directed towards these parts of the story that are likely to elicit negative emotions and impair performance.

Authors

• Stephanie Buono, University of Toronto

• Earl Woodruff, OISE/University of Toronto