Exploring How Methodological Decisions Affect the Variability of Schools Identified as Beating the Odds

• In Event: 35.027 - School-Wide Innovation: The Impact of Structure, Design, Methodology, and Data-Based Decision Making

Abstract

A number of states and school districts have identified schools that “beat the odds (BTO),” or perform better than expected given the populations they serve. But how stable are the identification results? Based on one state’s experience, this paper examines how a list of schools labeled as BTO changes depending on the choice of statistical methods and technical specifications applied. The study utilizes the state-wide school- and student-level data from K–12 public schools in the study state. By demonstrating the extent of variation that can be expected in the identification results, the paper highlights the importance of careful consideration of the selection criteria for BTO schools, as well as technical decisions in operationalizing these selection criteria.

Authors

• Yasuyo Abe, IMPAQ International, LLC

• Phyllis Weinstock, IMPAQ International

• Coby Meyers, University of Virginia