Faculty-Led GenAI Interventions and Student Learning: A Meta-Analysis Using Hierarchical Linear Modeling

• In Event: Future-Oriented Dialogues on AI in Education

Abstract

This meta-analysis investigates the effectiveness of generative AI (GenAI) interventions implemented by university faculty to enhance student learning. Using hierarchical linear modeling, 140 effect sizes from 20 empirical studies were analyzed to examine how methodological and contextual variables influence outcomes. The mean of the effect size (Cohen’s d) is .59, the minimum is -1.08, and the maximum is 9.85 (SD = 1.43). The results indicate that pre- and post-test designs, statistical association methods, and face-to-face delivery are associated with significantly higher effect sizes. Blended learning environments showed reduced effectiveness. Practical implications include improved instructional design for AI-enhanced education. Future research directions emphasize the need for more rigorous, cross-contextual studies to guide ethical and effective GenAI use in higher education.

Authors

• Katherine Jiawen Ren, University of North Carolina - Charlotte

• Ayesha Sadaf, University of North Carolina - Charlotte

• Judson Macdonald, University of North Carolina at Charlotte

• Delandrus Lenet Ieashea Seales, University of North Carolina - Charlotte

• Ji Yae Bong, University of North Carolina - Charlotte

• Chuang Wang, University of North Carolina - Charlotte