Student Sensitivity to Changes in Teacher Practice

• In Event: New Approaches to the Study of Engaging and Motivating Classroom Instruction

Abstract

Purpose

The purpose of this investigation was to explore within-student variation in ratings of teacher practice, as well as whether engagement and achievement predict variation.

Conceptual Framework

The use of student reports of classroom instruction in research satisfies two purposes – 1) gathering individual perceptions of instruction (Ruzek & Schenke, 2020), and 2) assessing between-teacher differences in instruction via aggregation (Lüdtke et al., 2009). Accuracy in the latter partially depends on whether students perceive meaningful gradations in their teacher’s instruction. To better understand students’ sensitivity to changes in instruction, we examined within-student variability in their reports of science instruction collected over 17 consecutive days.

Methods

The 218 participating students were in high school (57% female, 60% Students of Color, 42% eligible for free/reduced-price lunch). Students reported perceptions of their teachers’ practice following each class for an instructional unit. This analysis focused on students’ daily reports that their teachers provided choice opportunities in class (α = .87) and identified the usefulness, importance, or relevance of activities (α = .86).

To understand the extent to which students perceived differing daily levels of each construct, we estimated multivariate mixed effects location-scale models (MELSM; Hedeker & Nordgren, 2013) using the R package brms (Bürkner, 2018). These models allow for random intercepts that account for dependencies in reports from the same student and from students in the same classroom, and critically, allow for separate random intercepts for each student’s residual variance (i.e., σ in Gaussian models). Practically, σ represents student differences in consistency/fluctuations in their choice and rationale reports. We additionally predicted σ to identify whether a student’s behavioral engagement (α = .80) and a student’s most recent grade in science from the prior unit were associated with within-person variance. See Tables 1 and 2 for respectively, descriptive statistics and correlations.

Results

Initial univariate models were estimated for choice and rationale. These models indicated that within-student variance was similar in size for choice and rationale. A multivariate MELSM, in which choice and rationale were modeled simultaneously, indicated that students who were more variable reporters of choice also tended to be more variable reporters of rationale (r = .50, 95% CI = [0.33, 0.66]). Finally, we added predictors to the multivariate MELSM, finding that more behaviorally engaged students tended to not only have higher mean reports of choice and rationale (β[SE]s of .30[.06] and .46[.07], respectively), but also tended to be more variable in their reports of choice (β[SE] = .13[.04]) but not rationale (β[SE] = .03[.04]). In contrast, within-student variance in choice and rationale reports were unrelated to prior unit grade.

Discussion

We find that some students are more variable in their day-to-day reports of their teacher’s provision of choices and rationales for learning material. Students who were more variable in reporting choice also tended to be more variable in reporting rationale, suggesting a consistency that may indicate greater sensitivity to the day-to-day changes in a teacher’s instruction. This is further suggested by the fact that these students tended to be more highly engaged.

Authors

• Erik Ruzek, NWEA
  Presenting Author

• Erika A. Patall, University of Southern California
  Non-Presenting Author

• Donald R. Williams, Northwest Evaluation Association
  Non-Presenting Author