Multilevel Weighted Analyses of NAEP Teachers and Students

• In Event: Feasibility of Teacher-Centric Analyses Using NAEP Data

Abstract

Similar to the previous presentation, the purpose of this presentation is to demonstrate the utility of NAEP teacher sampling weights. This section addresses the following research questions:
1. Is there sufficient between-teacher variation of student achievement to justify using a multilevel analysis?
2. What can a two-level model reveal about the effects of teachers’ characteristics on student achievement?
3. How do the results of the multilevel analysis compare to results of parallel single-level analyses, using student-level data and student-level weights?
Our analysis uses hierarchical linear modeling (HLM) and data from the 2013 fourth grade mathematics assessment. The HLM analysis includes three models, focusing on three sets of predictors: teachers’ qualifications, teachers’ use of effective teaching practices, and teachers’ race/ethnicity. The HLM models examine whether, after controlling for other factors, student achievement tended to vary as a result of these predictors. In addition, the three models examine whether there was any additional effect of these predictors on the achievement of Black students in particular.
The intraclass correlation indicates that 37% of variability in student achievement occurs between teachers. Results of the three HLM models are shown in Table 10. All three models include the same school characteristics (locale, percent non-white enrollment, and percent NSLP-eligible enrollment) and the same student characteristics (sex, NSLP eligibility, and race/ethnicity). Model 1 also includes measures of teachers’ qualifications (master’s degree, certification, and years of experience). Model 2 includes teaching practices (standardized versions of the differential instruction and formative evaluation composite variables). Model 3 includes teachers’ race/ethnicity. In each model, the same teacher-level covariates are utilized as predictors of the black student achievement slope.
Model 3 examines the relationship between teachers’ race/ethnicity and students’ mathematics achievement. Holding all other factors constant, and in comparison to white teachers, Asian teachers were associated with student achievement scores that were 4.5 points higher on average, while Black teachers were associated with sores that were 2 points lower on average. Black teachers were also found to have an important effect on the performance of black students in particular. While our models did not explore causality, one possible explanation for these results is that Black teachers are more likely to instruct at-risk students with below-average performance, while Asian teachers are more likely to work in classrooms with above-average students. Moreover, we found that Black students working with black teachers tended to score about 4 points higher than black students of white teachers, all else being equal. Thus, our analysis appears to confirm prior research that black students tend to perform better when taught by black teachers (Dee, 2005).

Author

• Austin Lasseter, American Institutes for Research