Interact More, Learn More? A Randomized Controlled Trial of GenAI Tutors on Student Learning

• In Event: AER Roundtable Session Friday 1:45 pm - Gold 4
  In Roundtable Session: Human–AI Collaboration and Dialogic Learning

Abstract

This study examined the effects of a retrieval-augmented generation (RAG) GenAI tutor on university students’ course interest, self-efficacy, and academic performance in an introductory marketing course. Using a randomized controlled trial, we compared a GenAI-supported group with a control group that received conventional instruction. No significant differences were found in the outcomes measured. Further observations in the treatment group showed how often students interacted with the Tutor did not affect the same outcomes. However, engagement with the Tutor was higher for the face-to-face and non-working students, whereas previous GenAI experience had no significant relationship with this engagement. These findings suggest the effectiveness of GenAI should be weighed against conventional approaches. Implications for reflective integration and research will be discussed.

Authors

• Weijiao Huang, University of Hong Kong

• Andrew Thoeni, University of North Florida

• Luke K. Fryer, University of Hong Kong