Evaluating the Performance of ChatGPT for Selecting Lesson Segments That Support Productive Coaching (Poster 5)

• In Event: AERA Poster Session Sunday 11:40 a.m.
  In Poster Session: Division K Section 5 Poster Session: Teacher and Teacher Educator Learning

Abstract

In the present study, we investigate the potential of generative AI (ChatGPT 4) for choosing segments from teachers’ videoed classroom discussions to discuss in a post-conference. Drawing on data from a study of a successful, video-based coaching program, we compare segments chosen by GPT 4 and by an expert coach. Our emerging results suggest that ChatGPT has significant potential for selecting coachable moments to support a productive coaching conversation. Over half of the ChatGPT selected segments were rated the same or close to the quality of the expert coach. Reasons for lower ratings of ChatGPT-selected segments included being overly long, too short, or showing multiple discrete interactions rendering it difficult to identify a focal issue for discussion.

Authors

• Lindsay Clare Matsumura, University of Pittsburgh

• Marguerite E. Walsh, University of Pittsburgh

• Tianwen Li, University of Pittsburgh

• Dena Zook-Howell, University of Pittsburgh

• Richard J. Correnti, University of Pittsburgh

• Diane Litman, University of Pittsburgh