Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Browse By Descriptor
Search Tips
Annual Meeting Theme
Exhibitors
About Philadelphia
About AERA
Personal Schedule
Sign In
X (Twitter)
Teacher evaluation has been an increasing focus since the inception of Race to the Top. As such, states and school districts around the country have been creating new systems to assess teacher quality. In doing so, the newly created instruments need to be assessed for inter-rater reliability. Past reliability studies have typically provided a percent of overall agreement and some variation of Cohen’s Kappa as evidence of inter-rater reliability. While this is commonplace, Kappa statistics have been shown to suffer from a paradox which causes the estimate to be artificially low. Evaluation data from a school district are used to both illustrate the paradox and to provide support for the inclusion of Gwet’s AC1 statistic into educational reliability studies.