Technology-mediated Self-assessment and Metacognitive Outcomes: A Systematic Review

• In Event: Assessment Literacies in a Post-Pandemic Digital Age: An Exploration Through Six Systematic Reviews

Abstract

Self-assessment has emerged as a crucial formative process in education, demonstrating significant impact on students' learning and achievement outcomes (McDonald & Boud, 2003; Yan et al., 2022). Research indicates a bidirectional relationship between self-assessment and metacognition, showing that self-assessment strategies directly improve learners’ metacognitive skills (Ibabe & Jauregizar, 2010; Siegesmund, 2016; Mendoza & Yan, 2021). As technology increasingly permeates educational practices, it is reshaping how self-assessment is implemented, potentially modifying its impact on metacognitive outcomes (Ibabe & Jauregizar, 2010; Roll et al., 2011). This systematic review synthesizes the current empirical evidence on the influence of technology-mediated self-assessment (TMSA) on students' metacognitive outcomes, focusing on self-testing skills, metacognitive awareness/knowledge, and testing strategies.

Our review addresses two key research questions: (1) What is the overall effect of TMSA on students' metacognitive outcomes? (2) How do different TMSA implementations impact specific metacognitive skills? The literature search was conducted in ERIC, PsycInfo, and Web of Science, identifying 16 studies that met all inclusion criteria.

The findings suggest that TMSA can enhance learners' capacity to reflect on and evaluate their learning processes, thereby enriching their metacognitive strategies (Elsner & Großschedl, 2023; Huang et al., 2023; Max et al., 2022; Mou, 2023). Improvements were observed in self-monitoring, self-evaluation, reflection, and comprehension monitoring. The effectiveness of TMSA interventions appears to be enhanced when combined with additional supports such as worked examples, tutoring, or learning diaries (Schmidmaier et al., 2011; Theobald & Bellhäuser, 2022; van den Boom et al., 2007).

The studies employed various TMSA approaches, including multimedia self-assessment tools, prompting systems, intelligent tutoring systems, and online feedback mechanisms (Bakx et al., 2002; Ibabe & Jauregizar, 2010; Kauffman et al., 2008; Yan et al., 2022). TMSA implementations utilizing real-time feedback and interactive platforms tend to show more consistent positive effects on metacognitive outcomes (Say et al., 2023; Theobald & Bellhäuser, 2022). Studies incorporating self-reflection prompts and structured learning diaries within TMSA systems reported significant improvements in students' metacognitive awareness and self-regulation strategies (Mou, 2023; van den Boom et al., 2007).

The review highlights TMSA's potential to support metacognition, enhancing students' ability to plan, monitor, and evaluate their learning processes (Huang et al., 2023; Max et al., 2022; Elsner & Großschedl, 2023). While the majority of studies report positive outcomes, some show mixed effects, underscoring the complexity of designing effective TMSA. This review emphasizes the need for more rigorous research designs and standardized measures of metacognitive outcomes (Panadero et al., 2017; Yan et al., 2021; Yan et al., 2022).

This review contributes to the emerging research on self-assessment and technology-enhanced learning, providing a focused analysis of TMSA's impact on metacognition. Our findings have important implications for leveraging technology to foster students' metacognitive skills and self-directed learning capabilities (Panadero, 2017; Yan et al., 2022). As technology evolves, including the emergence of generative AI (Urban et al., 2023), this review provides a foundation for understanding how TMSA can be optimally designed to support metacognitive development and prepare students for lifelong learning in an increasingly digital world (Yan et al., 2023).

Author

• Norman B. Mendoza, The Education University of Hong Kong