Cued Retrospective Reporting: A Method for Measuring Self-Regulated Learning

• In Event: Innovations in Researching Regulation of Learning in Solo and Collaborative Tasks

Abstract

This study examines Cued Retrospective Reporting (CRR), as a method for measuring Self-Regulated Learning (SRL) skills in a cognitively complex domain. In CRR, eye-movements of participants are recorded during task performance. After performing a task, participants review the recordings of their own eye movements superimposed on the stimulus and verbalize the thoughts they had while performing the task (Van Gog, Paas, Van Merriënboer, & Witte, 2005). By using CRR as a tool for measuring meta-cognitive and self-regulatory processes in SRL, we expect to overcome two shortcomings of other SRL measures (e.g., concurrent reporting, questionnaires). First, CRR does not affect the actual task performance since the reporting takes place after the task performance and eye-tracking can be done in a totally on obstructive way. This is an important fact in a domain where distraction of the task is undesired (e.g., Air Traffic Control; ATC). Second, using eye-movements makes the cue personalized and distinct minimizing fabrication of socially desirable answers. The CRR was validated in cognitive complex domain by triangulating with other SRL related measures.

Eighteen ATC trainees participated in 60 minutes individual sessions. The main task was a 12 minute ATC exercise. Eye-movements were recorded during performance. After task performance, participants self-assessed their task performance. A coach assessed participant performance. Next, the participant performed a CRR on 75% speed to give more time for verbalizing the thoughts. Prior to the main task, participants completed: (a) the PRO-SDL scale (Stockdale & Brockett, 2010) measuring degree of taking initiative, control, and participants’ motivation, and (b) the STPQ scale measuring self-efficacy on task performance (Lodewyk & Winne, 2005).

Utterances from CRR revealed SRL skills such as: orientation, planning, monitoring, adjusting, and assessment skills (Zimmerman, 1989). The mutual order SRL skills use and the relative frequency and moment of use was captured with CRR. Findings: (a) provide insight about the potential for using CRR as measurement tool for SRL skills in cognitive complex training domains, and (b) indicate participants focus on cognitive aspects of the training task and that they are hardly aware of their own learning needs and learning opportunities regarding task performance. Findings stress the need for developing methods and learning environments to foster the development of SRL (Dinsmore, Alexander, & Loughlin, 2008). Finally, for developing training programmes, CRR can provide coaches information on trainees’ individual levels of SRL skills. This may generate knowledge for scaffolding instructional support for the trainees’ SRL development.

Authors

• Ludo Van Meeuwen, The Open University of the Netherlands

• Saskia Brand-Gruwel, The Open University of the Netherlands

• Paul A. Kirschner, Open University of the Netherlands

• Jeano de Bock, Air-Traffic Control, the Netherlands

• Jeroen J.G. Van Merrienboer, Maastricht University