New Literacies, New Challenges: Modeling Self-Regulated Learning and Epistemic Cognition Using Think-Aloud Protocol Data

• In Event: Self-Regulated Learning Analytics: Aligning Data and Their Treatment to the Assumptions of Theory

Abstract

Emergent and rapid changes in learners’ use of computer-based learning environments (CBLEs), such as the Internet, have led to commensurate increases in the demands placed on these learners to manage vast amounts of information available in these contexts (Leu, Forzani, Rhoads, Maykel, Kennedy, & Timbrell, 2015). New Literacies research focuses on the dynamic and multifaceted skills, strategies, and practices required to meet these demands (Leu, 2010; Leu, Kinzer, Coiro, Castek, & Henry, 2013). As learners navigate multiple, non-linear texts, they must locate and evaluate relevant sources, synthesize information, and communicate solutions to complex problems (Castek & Coiro, 2015). Greene, Yu, and Copeland (2014) have argued that two critical aspects of New Literacies are self-regulated learning (SRL; Winne & Hadwin, 2008; Zimmerman, 2000) and epistemic cognition (Chinn, Buckland, & Samarapungavan, 2011; Greene, Azevedo, & Torney-Purta, 2008; Hofer & Pintrich, 1997). SRL involves the adaptive monitoring and control of cognitive, metacognitive, motivational, and affective aspects of learning (Greene & Azevedo, 2007; Winne & Hadwin, 2008). As students engage in SRL over the course of learning (e.g., task definition, planning, strategy use, and adaptation), the ways they acquire, use, validate, and communicate knowledge (i.e., epistemic cognition) influence their choice of resources, and how they engage with, and integrate, them (Muis, 2007). New Literacies, including SRL and epistemic cognition, are best captured via concurrent methods (e.g., think-aloud protocols, trace logs), but finding the signal amongst the noise of the large-scale datasets that result from such data collection can be challenging (Ben-Eliyahu & Bernacki, 2015). Our work has been focused on testing the predictive validity of various methods of data reduction and aggregation with think-aloud protocol (TAP) data (Ericsson & Simon, 1983).
In this study, 53 undergraduates independently participated in a 30-minute learning task about a complex science topic: whether multivitamins are effective for normal, healthy adults. A pretest was used to assess prior knowledge. During the learning task, participants were free to search anywhere on the Internet, but were also provided a list of relevant Internet sites. They were asked to verbalize their thoughts (i.e., TAP) as they learned and navigated the Internet. Following the learning task, participants wrote a posttest essay on the topic of multivitamin effectiveness. Learner verbalizations were transcribed and coded for self-regulated learning and epistemic cognition processes using a coding scheme derived from prior published work (e.g., Azevedo & Cromley, 2004; Greene et al., 2014).
Word limits preclude a full review of all of the methods and results that will be presented in the poster. Briefly, we investigated a number of techniques for capturing, aggregating, and testing TAP data. We found that data-driven analysis techniques resulted in the best fitting model, compared to other methods, revealing key SRL and epistemic cognition processes that predicted posttest performance, after controlling for prior knowledge (R2 = .447, p < .001). Our results have implications for how to best balance the fidelity and resource demands of TAP methods for capturing New Literacies (e.g., SRL and epistemic cognition), and how to help learners successfully use CBLEs.

Authors

• Jeff A. Greene, University of North Carolina - Chapel Hill

• Dana Z Copeland

• Victor M Deekens, University of North Carolina - Chapel Hill

• Seung Yu, University of North Carolina