Does Classroom Quality Change Over the School Year?: Implications for Accountability in Early Education

• In Event: 1-216 - Holding Early Childhood Education Accountability Systems Accountable: Findings from Classroom, State, and National Level Analyses

Integrative Statement

Early childhood education accountability policies, like state-level Quality Rating and Improvement Systems (QRIS) and the Head Start Designation Renewal System (DRS), increasingly rely on observational measures of classroom quality to monitor and evaluate programs. These systems use observations of classrooms, conducted for different programs at different times during the school year, to measure average classroom quality. The assumption underlying this approach is that observed classroom quality is not systematically higher or lower at different times during the school year. The present study tests this assumption by exploring (1) whether and how observed quality, as measured by the Classroom Assessment Scoring System (CLASS; Pianta, LaParo & Hamre, 2008), changes over the course of the academic year within classrooms, and (2) the amount of within- vs. between-teacher variability in CLASS scores, taking into account the amount of variability explained by natural growth in quality over time. Based on our findings, we quantify how likely early childhood programs are to be misclassified under QRIS and Head Start DRS due to growth and variability of CLASS scores.

Data come from the National Center for Research on Early Childhood Education Teacher Professional Development Study, conducted between 2007 and 2011 in nine cities across the U.S., which provided professional development to teachers on classroom instruction (Pianta et al., 2017). Our analysis draws on 136 preschool teachers in the control group who were required to regularly submit videos of their classroom instruction throughout two academic school years. All videos submissions were double-coded by certified raters and scored according to the CLASS.

We employ conditional growth curve analysis to explore classroom-level changes in observed quality over the school year. Specifically, we estimate growth curves separately by CLASS domain, considering seasonal patterns of growth, as well teacher- and classroom-level factors that predict growth patterns over the school year. Next, we will decompose variation in CLASS scores within and between teachers, accounting for natural growth in observed classroom quality over time. Lastly, we will employ a generalizability (G) study/decision (D) study framework to quantify implications of this variation for accountability policies. We focus specifically on the Head Start DRS, which uses programs’ CLASS scores as a criterion for renewed funding, and state QRISs that rely on CLASS thresholds to generate public ratings of program quality.

Preliminary results reveal meaningful change in observed classroom quality over the academic school year. Teachers’ Instructional Support scores evolve the most over time, ending higher than they begin at the start of the year, with suggestive evidence of seasonal dip in the winter months. Emotional Support scores display a similar U-shaped pattern of change, while teachers’ scores in Classroom Organization follow the opposite trajectory, peaking in the middle of the year. Future analyses will build on these models to explore predictors of trajectories of growth. We will also decompose the variance in CLASS scores within- and between-teachers taking into account patterns of growth and quantify implications for specific accountability policies. Implications for the design and implementation of ECE accountability will be discussed.

Authors

• Olivia Healy, Northwestern University
  Presenting Author

• Kathryn Gonzalez, Harvard University
  Non-Presenting Author

• Luke Miratrix, Harvard University
  Non-Presenting Author

• Terri Sabol, Northwestern University
  Non-Presenting Author