Teacher-reported and Observed Instruction in Early Childhood: Which Measure Better Predicts Children’s Development?

• In Event: Teacher Practices and Individual Student Experiences: Measuring Influences on Pre-Kindergarten through Third Grade Students

Abstract

Children’s exposure to advanced instruction in math and literacy, as reported by teachers, is associated with gains in children’s academic skills in kindergarten (Bassok et al., 2016; Engel et al., 2016). However, teacher reports may be subject to social desirability and recall biases, as teachers generally report on their activities across the full academic year. To our knowledge, there is no evidence on the extent to which teacher reports of advanced instruction correlate with their observed classroom practices. Moreover, there is little work that has considered whether reports versus observational measures of advanced practices are comparable predictors of gains in children’s academic skills. Additional evidence on these topics can help inform decision-making related to measurement of instruction in early education settings.
We used data from prekindergarten and kindergarten students (N = 263 Pre-K; N = 359 kindergarten) and teachers (N = 39 Pre-K; N = 46 kindergarten) in the Boston Public Schools. In the spring of each year, teachers reported on their math and literacy practices using items drawn from the ECLS-K Spring Classroom Teacher Questionnaire (NCES, 2011). We coded how advanced these practices were, based on the expected grade level of the activities in the Common Core Standards for English Language Arts and Math and the Massachusetts State Standards, and estimated the average grade level of instruction for each domain in PreK and kindergarten. Videotapes of instruction were also used to code children’s classroom experiences on the Individualizing Student Instruction (ISI; Connor et al., 2009). In the current study, we aggregated child-level measures to the classroom level in order to operationalize the amount of time that children in classrooms were engaged in math, language, and literacy activities that were expected for each grade-level.
We then used bivariate correlations to explore the extent to which the grade levels of teacher-reported language/literacy and math activities were associated with the amount of observed language/literacy and math instruction appropriate for the grade level. Then, we estimated linear associations between these measures, and gains in children’s language (Peabody Picture Vocabulary Test-IV), math (Woodcock-Johnson Applied Problems, REMA), and literacy (DIBELS subscales, kindergarten only) skills. The models adjust for child and classroom level covariates, including classroom instructional quality. See Table 3 and 4 for model results.
We found that teacher reports of more advanced instructional content were not associated with their observed classroom practices. Further, as shown in Table 3, our teacher-reported measures were more predictive of gains in children’s skills than our observational measures. Interestingly, our observational measure showed negative relations with gains in children’s literacy skills in kindergarten on some outcomes. We will discuss implications of these findings for measurement in early education settings.

Authors

• Paola Andrea Guerrero-Rosada, Texas A&M University - College Station
  Presenting Author

• Christina J Weiland, University of Michigan
  Non-Presenting Author

• Meghan Patricia McCormick, MDRC (Manpower Demonstration Research Corporation) - New York City
  Non-Presenting Author

• Michelle F. Maier, MDRC (Manpower Demonstration Research Corporation) - New York City
  Non-Presenting Author

• Catherine Snow, Harvard University
  Non-Presenting Author

• Joann Hsueh, MDRC (Manpower Demonstration Research Corporation) - New York City
  Non-Presenting Author

• Jason Sachs, Boston Public Schools
  Non-Presenting Author