The Effect of an Assessment Training Video on Music Educators’ Rating Accuracy of Singing Performance

• In Event: AERA Roundtable Session Saturday 1:45 pm - Gold 4
  In Roundtable Session: From Mentorship to Classroom Innovation: Supporting Teachers’ Assessment Literacy

Abstract

To develop a target assessment literacy training tool, we examined the effectiveness of a training video on improving music educators’ rating agreement using a singing performance rubric. 11 K-5 general music educators were randomly assigned to treatment or control groups and rated 50 singing samples using an analytic rubric with tonal, rhythm, and vocal quality dimensions. The treatment group received a training video addressing rubric criteria, types of rater bias, and bias-reducing strategies. Exact agreement rates increased for all participants, with the treatment group improving dramatically compared to modest control group improvements. ANCOVA results showed significant improvements in agreement across all dimensions, controlling for baseline. The video training effectively enhanced rater agreement for this rubric.

Authors

• Kathleen Arrasmith, University of South Carolina

• Xiaobo Wei, University of South Carolina

• Ashlee A. Lewis, University of South Carolina