Processing of Self-Regulated Learning Feedback: Insights from Student Reflections Using Sentiment Analysis

• In Event: Building Theory-Informed and Context-Sensitive Natural Language Processing to Capture Self-Regulated Learning in Action

Abstract

Objectives and Theoretical Framework
Self-regulated learning (SRL) is positively linked to measures of college academic success (Tuckman & Kennedy, 2011), thus resulting in the development of SRL interventions at the postsecondary level (Jansen et al., 2019; Authors, 2023a). Interventions that include a feedback component (e.g., teacher-provided feedback on students’ self-regulation) are associated with greater intervention effects (Theobald, 2021). Despite the known advantages of feedback (Wisniewski et al., 2020), less is known about how feedback is processed within these contexts. Students’ mindset beliefs might provide insight into feedback processing. While prior work has suggested a growth-mindset is predictive of both SRL (Authors, 2023b; Hertel et al., 2024) and positive emotional sentiment (Topkoc, 2024), mindset and sentiment have not been studied concurrently. Identifying a potential connection then may help identify mechanisms through which students interpret and process feedback. We thus examined students’ written reflections in response to feedback about their SRL behaviors and strategy use in a learning-to-learn (L2L) course. We applied sentiment analysis, a natural language processing (NLP) technique, to a) understand what students wrote about, b) extract emotional sentiment from their writing and examine how sentiment changes over time, and c) examine how emotional related to students’ self-reported mindset at the start of the L2L course.
Method
At the beginning (T1; Week 2) and end (T2; Week 14) of one semester, undergraduates in an L2L course (N=405) completed surveys assessing their SRL strategy use, behaviors, and mindset. Immediately after each survey, students were provided with automatically generated feedback on their survey responses (e.g., less/more adaptive strategy use) and asked to write a brief reflection. We collected SRL and mindset variables from the surveys, along with students’ written reflections.
Results
Text data was analyzed using sentimentr (v2.9.0; Rinker, 2021) R package. Across the two time points, students’ responses exhibited different patterns of positively and negatively valenced terms (Figure 1). The distribution of positively- and negatively-valenced sentences in students’ responses also varied across time points (Figure 2). The average sentiment at each time point was also calculated; sentiment from reflections T2 was significantly more positive than reflections from T1 (Figure 3). To test the extent to which emotional sentiment varies as a function of self-reported mindset beliefs, T2 sentiment was regressed on T1 mindset, controlling for T1 sentiment (Table 1); results indicate T1 mindset did not predict T2 sentiment.
Discussion
This study examined students’ implicit emotional response as expressed in their written reflections following feedback on their SRL within a L2L course through the application of NLP to large amounts of text data. We found distinct patterns in students’ text from the beginning to the end of the semester in terms of the content and emotional sentiments within their writing. While students felt more positively about their SRL feedback across the semester, we did not find significant relations between sentiment and mindset, indicating further need to examine relationships between sentiments and changes in strategy use, motivational beliefs, and relationships to more online measures of SRL, among other potential directions, within L2L interventions.

Authors

• Huy Nguyen, The Ohio State University

• August Masonheimer, Purdue University

• Christopher A. Wolters, The Ohio State University