Exploring relationships between malleable school-level factors and high-achieving black students

• In Event: Using NAEP Data to Investigate Student Differences in Education

Abstract

Recurrent narratives focusing on achievement gaps between Black students and other groups of students belie the fact that there is substantial overlap in the achievement distributions of Black students and other students. In fact, many Black students perform better than the National Assessment of Educational Progress (NAEP) average (Cai, 2020). Unfortunately, studies focusing on achievement gaps can have the effect of reinforcing racial deficit narratives that can damage students’ sense of self-worth and lower their academic self-concept (Steele & Aronson, 1995; Warikoo et al., 2016).

This research centers on high-achieving Black students and explores the factors associated with their high performance. Using samples of Black students from 2019 NAEP reading and mathematics public school data, we explore relationships between malleable school-level factors and the likelihood of performing above the national average on NAEP. Specifically, we ask:
What malleable school-level factors are associated with the likelihood of Black students achieving higher than the national average on NAEP reading and mathematics?

To answer this question, we first define high achievement as performing above the national NAEP average separately for math and reading and then fit multilevel logistic regression models, following an iterative model-building procedure that involves:
1) Fitting unconditional (null) models on data that include an indicator of student-level achievement as the outcome, with schools modeled as random effects at level-2;
2) Entering student-level control variables at level-1 that represent students’ demographic characteristics associated with achievement;
3) Entering school-level variables of interest at level-2 that represent more malleable factors that can be influenced by policy reforms.

This model-building procedure facilitates a deliberate approach to controlling for potential confounds, including school effects and student-level demographic characteristics that could lead to spurious conclusions about relationships of interest if they were left unmodeled. The control variables include a measure of students’ socioeconomic status, English learner status, and disability status. We also conduct sensitivity analyses to evaluate whether our results might be substantively influenced by discretionary modeling decisions. These additional analyses include modeling state-effects, both as fixed-effects with the two-level models and random effects in three-level models.

Preliminary results from analysis of grade 8 mathematics data suggest that various factors related to teachers’ instructional practices and school climate are associated with the likelihood that Black students achieve above the national average, even after controlling for demographic characteristics that are strongly associated with achievement. Among the factors related to instructional practices, we find relatively strong evidence suggesting that Black students are more likely to achieve above the national average when teachers do not report that they set different achievement standards for some students.
 
Results from Table 1 indicate that Black students are about 30 percent more likely to achieve above the national average in grade 8 mathematics for every one-point decrease on the 4-point Likert scale used to record teachers’ responses to whether they set different achievement standards for some students. Causality, however, cannot be assumed in the absence of an experimental design.

Authors

• David Bamat, American Institutes for Research

• Rebecca C. Shipan, American Institutes for Research

• George W. Bohrnstedt, None

• Alvaro Cruz, American Institutes for Research

• Winifred Wilberforce, The Ohio State University