Studying Student Affect in Computer-Based Learning Environments With Field Observations and Log Files

• In Event: Interdisciplinary Approaches for Analyzing Data From Multiple Affective Channels With Computer-Based Learning Environments

Abstract

Our laboratory has been conducting research on student affect using two modalities - log files from the interaction of students with computer-based learning environments (CBLE), and quantitative field observations (Baker, Corbett, Koedinger, & Wagner, 2004) - in order to determine how student interactive behaviors in CBLE differ between students with different affective experiences. We have collected coordinated data from these two modalities, where field observations were collected with Android handheld computers (running software designed to facilitate field observation) and then this data was temporally synchronized with student behaviors noted in log files from CBLEs. Thus far, we have collected synchronized data from several learning CBLEs, including Cognitive Tutor Algebra and ASSISTments. For each of these CBLEs, data was collected by two or more field observers, validated to achieve inter-rater reliability over κ = 0.6. Field observations were collected with respect to four affective states: boredom, frustration, confusion, and engaged concentration. These affective states are chosen based on the theoretical framing proposed by Graesser et al., (2006) that these affective states are
particularly relevant to on-line learning, and can be viewed as occupying affective dimensions of engagement (Fredericks et al., 2004).

In order to determine how student affect from field observations correlates to student interaction with the CBLEs, we distilled a set of features that represented various aspects of student behavior within the learning system. The features included indications of the
temporal aspects of student behavior (Baker, Corbett, Roll, & Koedinger, 2008), past detectors of student meta-cognitive behaviors and disengaged behaviors (Aleven, McLaren, Roll, & Koedinger, 2006; Baker, 2007; Baker et al., 2008; San Pedro, Baker, Rodrigo, 2011) and features representing the history of student behaviors, such as correctness or help use (Baker et al., 2008). A range of data mining algorithms were used to determine which student behaviors correlated with the four affective states, with the goodness of algorithms assessed with Cohen's (1960) Kappa and A' (Hanley & McNeil, 1982), cross-validated at the student level. We were able to find combinations of data features that automatically predicted the affective states in each learning environment. These models in turn have been able to predict student performance on standardized
examinations (Pardos et al., 2013) and college attendance (San Pedro et al., 2013a), as well as shedding light on the contexts in which affect occurs (San Pedro et al., 2013b).

These results provide evidence of semantically meaningful behavioral correlates of student affect and demonstrate the value of correlating ground-truth labels of student affect (i.e. field observations) with log file's indicators of student behavior.

Authors

• Ryan Baker, Teachers College, Columbia University

• Sujith M. Gowda, Worcester Polytechnic Institute

• Jaclyn Ocumpaugh, Worcester Polytechnic Institute

• Supreeth M. Gowda, Worcester Polytechnic Institute

• Maria San Pedro, Teachers College, Columbia University