Coherence Analysis for Identifying Self-Regulated Learning Profiles and Changes Over Time

• In Event: Self-Regulated Learning Analytics: Aligning Data and Their Treatment to the Assumptions of Theory

Abstract

Advances in computing devices and software have made it easier for researchers to leverage the power of computer-based learning environments to study aspects of students' self-regulated learning (SRL) behaviours (e.g., Kinnebrew, Segedy, & Biswas, 2014; Sabourin, Shores, Mott, & Lester, 2013; Winne et al., 2006). SRL is a multi-faceted construct involving monitoring one's learning progress, emotional and behavioral control, management of one's learning environment and cognitive resources, perseverance in the face of difficulties, and engaging in social interactions that promote effective learning (Zimmerman & Schunk, 2011). For decades, researchers have recognized academic advantages for learners who are self-regulated (e.g., Zimmerman, 1990), and SRL is particularly important for students working in open-ended computer-based learning environments (OELEs; Land, 2000). OELEs provide a learning context and a set of tools for exploring, hypothesizing, and building solutions to authentic and complex problems. Such environments are demanding; they require students to wrestle simultaneously with their emerging understanding of a complex topic (e.g., ecosystems or macroeconomics), develop and utilize problem-solving skills, and employ SRL processes for managing the complexity and open-ended nature of the learning task. The open-ended nature of the environment provides students with choices on how to proceed, and these choices can reveal information about how students' regulate their learning, including their metacognitive monitoring, control of cognition, and their social interactions (e.g., help seeking behavior).

We present our work in applying coherence analysis (CA; Segedy, Kinnebrew, & Biswas; 2015), a novel approach to analyzing and interpreting student behavior in OELEs. From an information-processing perspective congruent with Winne and Hadwin's model of SRL (Winne & Hadwin, 1998; Winne 2001), CA focuses on the learner’s ability to seek out, interpret, and apply information encountered while working in the OELE. By characterizing behaviors in this manner, CA provides insight into students' metacognitive monitoring and control, as well as the extent to which they understand the nuances of the learning task. We apply CA to logged data of student actions during a recent classroom study with the Betty's Brain OELE (Leelawong & Biswas, 2008). In Betty's Brain, students have the task of teaching a virtual agent, Betty, about a science phenomenon by constructing a causal map. Students acquire the necessary domain knowledge by reading hypertext resources and teach this information to Betty. To assess their evolving causal maps, students can ask Betty to answer cause-and-effect questions and have her take graded quizzes (answering sets of questions generated and graded by a Mentor agent).
In this analysis, we apply clustering to student behavior data described by the CA measures in two (consecutive) units in Betty's Brain, identifying groups of students with similar behavioral and metacognitive characteristics. Results of this analysis illustrate the effectiveness of CA in 1) identifying common behavior profiles that are linked to metacognitive processes (or the lack thereof) and 2) evaluating shifts in self-regulatory behavior during and across two science units. Overall, these results provide insight into students' SRL processes and suggest targets for adaptive scaffolds to support students' development of open-ended problem-solving skills.

Authors

• Gautam Biswas, Vanderbilt University

• John Kinnebrew, Vanderbilt University

• James Segedy, Vanderbilt University