Using Educational Data Science to support Adaptive Learning Technologies for Self-Regulated Learning

• In Event: Building Theory-Informed and Context-Sensitive Natural Language Processing to Capture Self-Regulated Learning in Action

Abstract

Purpose and Framework
Societal megatrends like globalization and digitization have necessitated innovative approaches in schools to promote sustainable learning, emphasizing lifelong learning for a more equitable society (UNESCO, 2022). Self-regulated learning (SRL) is crucial for this and is increasingly supported by digital tools (Broadbent & Poon, 2015). SRL is an adaptive process where learners analyze tasks, set goals, and use strategies to achieve them, with motivation and emotional factors being crucial. The process is monitored using metacognitive strategies, which vary depending on the task and context (Greene et al., 2021). Continuous assessment of SRL skills through coaching and feedback is essential (Butler & Winne, 1995; Tempelaar, 2020).
In this context, Adaptive Learning Technologies (ALT) can offer personalized learning support by responding to individual learner needs. These systems can intervene and provide support at critical moments, thereby enhancing SRL strategies (Azevedo & Gašević, 2019; Khalil et al., 2024). Combining ALT with educational data science, including Large Language Models (LLM) and Natural Language Processing (NLP), has already provided promising results in adaptively fostering SRL (Authors, date; Pijeira‐Díaz et al., 2024). Yet, despite these promising results, many questions remain about how to improve ALT for a more comprehensive analysis of learner input and individualized support. Future research needs to develop methods to further improve the accuracy and efficacy of SRL interventions, especially with new LLMs such as ChatGPT (Ng et al., 2024). To this end, we are investigating the following questions:

1. To what extent can LLMs be used to diagnose SRL needs more accurately?
2. How can LLMs be configured to generate more customized feedback promoting SRL?

Method
This research is grounded in a Swiss project that collaborates with upper secondary school students and teachers to develop a tailored digital tool aimed at offering personalized and timely support in SRL. The collected data is based on individual interviews that were conducted with a sample of 25 students (13 female, 11 male, 1 non-gendered). Based on opinion mining (Liu, 2012; Varathan et al., 2017) and LLM prompt engineering (Bozkurt & Sharma, 2023; Knoth et al., 2024), we analyzed the transcribed interviews using different SRL dimensions (MSLQ; Pintrich, 2004). This analysis aimed to identify learners' needs and suggest tailored SRL strategies based on these identified needs.

Results
Based on our preliminary results, we find supportive evidence for actively using system message prompts and few-shot prompting to better contextualize feedback and address potential sources of bias in diagnosing SRL needs. By processing the input with NLP methods from opinion mining, we are able to provide the LLMs with more nuanced data, thus increasing the adaptivity of the ALT. The results show that the analysis of SRL needs (Figure 1a) and the feedback based on it (Figure 1b) can be more personalized. At the same time, similarities between learners can be identified.

Significance
This study improves SRL interventions by using LLMs and NLP to provide personalized feedback and support, enhancing learners' self-regulated learning by advancing adaptive learning technologies in secondary education.

Authors

• Mathias Mejeh, Fachhochschule Nordwestschweiz

• Martin Rehm, University of Southern Denmark