AI Feedback: Moving Past the Hype toward Effective Practice

• In Event: Blending Human and AI Feedback: Impacts on Student Engagement and Performance

Abstract

Objective
New and exciting educational technologies frequently emerge, each promising to transform the way we teach and learn. The new kid on the block is Artificial Intelligence (A.I.) which has brought about more platforms to give automated instantaneous feedback to student writing. However, does the use of Artificial Intelligence in feedback (AIF), like many previous educational technologies before it, live up to its expectations? This paper reports on the use of one AIF platform in an after-school tuition program offered to students from financially disadvantaged families. The two research questions that guided the study were (a) What are the students’ experiences of AIF? (b) How did AIF affect student outcomes?

Methods and Data Source
This two-year case study sought to examine the impact of this intervention on 21 students (aged 13-14). These participants, from different schools, were enrolled in the programme that aimed to improve their English language skills with weekly 2-hour sessions. These sessions incorporated the use of an AI-based application (Scribo) that can quickly analyse submitted work for technical aspects such as grammar, word choice, and punctuation. This article reports findings from interviews with both students and their tutors. The qualitative data were analysed using Bandura’s model of triadic reciprocality comprising environmental, personal, and behavioral factors.


Results and Significance
The students reported that the AI feedback (AIF) tool was easy to use and beneficial to their writing. The just-in-time feedback, along with teacher guidance, resources, and peer feedback, created a positive learning environment where students felt guided and encouraged to improve. Editing their work soon became a habit, with students even enjoying the process, treating it as a challenge to achieve higher AIF scores. The whole experience demonstrably boosted students' self-efficacy in writing, with AIF scores providing tangible evidence of progress, increasing their confidence. This newfound confidence extended to their schoolwork, where they noticed fewer errors and received positive feedback from teachers, further validating their improvement.
Bandura’s social cognitive self-regulation theory adopted for the study proved helpful as both as a theoretical lens and as an analytical tool for understanding the complex processes involved in AIF. It highlights to both the policy maker and the classroom teacher specific areas to work on to effect more positive outcomes from the use of AIF. For the policy maker, the study highlights that providing equal access to quality resources like AI, coupled with effective pedagogy, can potentially mitigate educational inequity driven by socioeconomic differences. Though this study was sited in an after-school programme, the findings are consistent with another study conducted in five mainstream schools. Teachers with large class enrolments can find comfort that AIF tools like Scribo can engage students to independently seek and act on feedback.
The integration of AI feedback (AIF) within a structured feedback pedagogy (pre-feedback, process, post-feedback), shows promise in enhancing learners' motivation and skills for self-improvement. These competencies are crucial for all students, particularly those from disadvantaged backgrounds, highlighting the potential of AIF in promoting equitable educational outcomes.

Author

• Hui Yong Tay, National Institute of Education - Nanyang Technological University