Assessing Self-Regulated Learning: A (Meta)Cognitive Modeling Approach

• In Event: Integrating Different Approaches to Investigating Self-Regulated Learning

Abstract

The main objective of our research is to help students be effective self-regulated learners, specifically, more effective help seekers, an important strategy for self-regulated learning. We focus on learning with an intelligent tutoring system (ITS), a type of computer-based learning environment that supports the learning of a complex cognitive skill (e.g., solving geometry problems) through tutored problem-solving practice. ITSs have been shown to lead to improved learning outcomes in a range of domains, including mathematics, physics, and computer programming.

ITSs provide “next step help” at any juncture in a student’s problem-solving process. However, students tend not to use these help facilities very effectively, use help in executive ways, or avoiding help altogether. It is likely that students would achieve more robust learning outcomes if they were to seek help more effectively. We view help seeking as an SRL skill and ask whether the same instructional approach that has proven to be effective for domain-level cognitive skills, tutored problem solving, can be effective for SRL skills.

To answer this question, we modified the tutor so it simultaneously tutors students with respect to geometry problem solving and with respect to help-seeking strategies. To provide automated tutoring on help seeking, we created a production-rule model of help seeking that captures the recurring decision students face, namely, whether to seek help or persist in trying to generate the given problem step independently. The model’s production rules identify the specific conditions under which seeking help is likely to be effective during the learning process. In this model, the decision to seek help on a given step is based on feeling-of-knowing judgments, judgments-of-learning, the number of errors made, and the amount of help received. The model also contains production rules that capture ineffective help-seeking behavior, which enable the tutor to respond to specific occurrences with specific feedback messages.

The model was validated by showing that it converges with other methods of help seeking. In an experiment with 4 geometry classes in a vocational school, involving 67 students, we found that automated tutoring of help seeking, based on this model and integrated with geometry tutoring, led to a lasting improvement in students’ help-seeking behavior.

Cognitive modeling has a long tradition in information-processing approaches of cognitive science and in research in cognitive architectures. The production rule formalism has been used in theories of SRL (e.g., Winne & Hadwin, 1998; 2008) but computer-executable models have not been used to directly assess student behavior. The use of rule-based cognitive modeling is likely to facilitate the theoretical unification between SRL and cognitive approaches to skill acquisition.

While the model provides a practical means of assessing and improving students’ help-seeking skill, it is also an attractive tool for research. This approach is compatible with an event-based perspective on SRL, as it provides an action-by-action account of help seeking. It is equally compatible with trait-based approaches to SRL, in that the skills in the model represent (somewhat stable) traits that predispose individuals to seek help (or not) in particular kinds of situations.

Authors

• Vincent Aleven, Carnegie Mellon University

• Ido Roll, The University of British Columbia

• Bruce McLaren, Carnegie Mellon University

• Kenneth R. Koedinger, Carnegie Mellon University