Poster #241 - Triangulation of physiological and observational data to understand cognitive-affective states during learning in Kindergartners

• In Event: 1-169 - Poster Session 04
  In Poster Session: PS 04 Section - Technology, Media & Child Development

Integrative Statement

Affect has been shown to play an important role in performance and learning as it influences cognitive processes (e.g., Kort and Picard, 2002; Goleman, 1995). However, much of classroom-based assessments focus on test scores and reporting achievements. The models and theories that guide the development of adaptive learning systems in the field of Human Computer Interaction focus on cognitive factors (Picard, 2010) making them unable to adapt to real-world situations in which affective factors play a significant role.
Drawing from prior work on physiological and facial markers accompanying increased mental effort (Sridhar et al., 2018; Shi et al., 2007) and wearable sensing, the study aimed to, 1) explore the feasibility of collecting physiological and observational data during learning, 2) identify sensitive markers and 3) triangulate the data sources to better understand cognitive-affective states during STEM (Science Technology and Math) learning. We present our findings from 14 kindergartners (4 – 7 years) as they explored some STEM concepts through a constructionist approach using a Lego STEM kit for 30 minutes. Each participant wore an Empatica E4 wristband that recorded the skin conductance (SC) and heart rate variability (HRV). The sessions were video-taped. Following a baseline measure for 5 minutes rest period, the learning was divided into 3 phases: Connect (participant listened to a story where context for the challenge was provided), Construct (participant constructed a pinwheel using a visual instruction sheet) and Contemplate (participant tested construction and brainstormed on wind energy). This was followed by a set of multiple choice and open-ended questions that tested their overall understanding (QA). The skin conductance and heart rate variability were analysed using MATLAB. The videotapes were coded for two types of behaviour: emotional state and surface behaviour (e.g., curiosity, looking away to disengage from stress, etc.; Eldar et al., 2010; Kort et al., 2002; Leidelmeijer, 1991)
In spite of some individual variations, we found some common patterns with physiological data. Paired t-test revealed that the number of SC responses were significantly higher for the construct, contemplate and question answer phase as compared to the resting baseline (ps <0.01). For HRV, paired t-test showed that the low frequency component modulated by sympathetic and parasympathetic activity was significantly higher (ps<0.05) for the contemplate and the QA phase indicating higher mental effort. No such difference was found for the Connect phase. This concurs with the nature of instructional approach, where it is mainly passive. Since physiology alone does not reveal the valence of the emotion and not all children had overt emotions, we triangulated them for a holistic picture. This revealed emerging patterns that correspond to a participant’s affective state during the learning process (Table 1). This study is a significant step towards identifying a map of cognitive-affective states in STEM learning. Having access to such information helps teachers understand what a child felt during learning journey, possible pain-points in real-time and redesign their instruction/ offer support. Further, it aids design of adaptive learning systems that can sense a learner’s cognitive-affective state and respond appropriately.

Authors

• Priyashri Sridhar, Singapore University of Technology and Design
  Presenting Author

• W. Quin Yow, Singapore University of Technology & Design
  Non-Presenting Author