Assessing the Relation of Passive and Active Data Streams on Performance in Exam Setting

• In Event: Authenticating the Signal: Validating Digital Traces of Student Learning Using Concurrent, Corroborating Data Sources

Abstract

Objective & Framework. This presentation will explore the challenges and opportunities created by examining the independent effect of passively-collected trace and actively collected self-report data on exam performance. As scholars move to align these convergent data streams, we consider the additive value of each aspect of this multimodal approach to data collection. We examine these relations in the context of an exam, as these assessments are frequent and consequential experiences for students (Jamieson, et al., 2016).
Advancements in technology now allow researchers to passively capture objective measures of physiological indicators of Sympathetic Nervous System (SNS) arousal (Electrodermal Activity: EDA; e.g., Järvelä et al., 2016; Malmberg et al., 2019). However, there remains some ambiguity regarding the function of various components of the EDA signal (Setz et al., 2009). Additionally, it is unclear how these different EDA components relate to item-level motivational factors such as self-efficacy and their additive value in predicting performance. Grounded in self-regulated learning (SRL; Greene, 2018) and control-value theory of academic emotions (CVT; Pekrun, 2006), we engaged in a series of studies using two different methods for decomposing phasic EDA, examining both students' average skin conductance response while they engage with each question (mean EDA) and the number of skin conductance response peaks during the question (peak EDA). We evaluated the relation between these two components of EDA, students' emotional regulation, and item-level self-efficacy, as well as the independent effect of each measure on item-level performance during a midterm practice exam.
Methods, Materials, & Results. Participants were recruited from an undergraduate engineering course in the United States. Data were collected during a required practice midterm exam one week before the test (AUTHORS, 2019). Before the test, students self-reported their trait emotional regulation (Gross & John, 2003). EDA data were continually collected from students during the exam, from which item-level phasic mean and peak EDA were extracted (Benedek & Kaernback, 2010). Before working on each test item, students reported their self-efficacy (Bandura, 2006); students' performance was measured at the item level after completing each question. We present data collection procedures in Figure 1. All hypotheses were tested using multilevel logistic regressions (e.g., Raudenbush & Bryk, 2002). We found that both within- and between-subject mean EDA positively predicted student performance independent of students' self-reported trait emotional regulation and item-level self-efficacy. Conversely, we found that both within- and between-subject peak EDA negatively predicted student performance independent of students' self-reported test-level emotional regulation and item-level self-efficacy.
Significance. These findings highlight the additive value of leveraging both passively collected and situated actively reported data in predicting student performance. We found that the two decompositions of EDA differently predicted students' performance, suggesting that each component relates to a discrete aspect of SNS arousal that has different implications for students' motivation, self-regulation, emotions, and performance. The significant influence of both measures of EDA on performance after accounting for the influence of item-level self-efficacy (Marsh et al., 1997) suggests that this passive data stream can also provide unique insight into students' experiences during the examination process.

Authors

• Matthew C Graham, University of Oregon

• Idalis Villanueva, University of Florida

• Jenefer E. Husman, University of Oregon

• Keith Zvoch, University of Oregon

• Md. Tarique Hasan Khan, Utah State University

• Darcie Christensen, Utah State University

• Shawn Lampkins, University of Oregon