The Importance of Measuring the Reliability of Accountability Scores for School Comprehensive Support and Improvement and Additional Targeted Support and Improvement Identification (Poster 7)

• In Event: AERA Poster Session Seven
  In Poster Session: Division H Research, Evaluation, and Assessment in Schools: Accountability in Schools

Abstract

Estimating the precision of school accountability scores is complex, as the scores are composites of school- or subgroup-level scores. Nevertheless, accountability scores have technical properties and error similarly to test scores. Calculating their reliability can explain the extent to which identification occurs by chance. Using Generalizability theory to initiate Monte Carlo and bootstrap sampling in simulation, this study estimates the variance of school accountability scores for a specific state, finding that the generated true CSI identification cutoff score is over one standard deviation away from the average cutoff score across simulated repetitions. Additionally, as the minimum count required for inclusion of a student group in an ATSI calculation increases, lower performing schools are less likely to be identified for ATSI.

Authors

• Lily An, Harvard University
  Presenting Author

• Brian Gong, Center for Assessment
  Non-Presenting Author