Investigating the Student Growth Across Subgroups From 2015 to 2016 in the AzMERIT Statewide Assessment

• In Event: 41.083 - Roundtable Session 15
  In Roundtable Session: 41.083-17 - Roundtable: Examining Construct-Irrelevant Factors Impacting Test Score Interpretations in Statewide Assessments

Abstract

The AzMERIT assessments report student test scores on a vertical scale, allowing families and teachers to make inferences about student growth across school years. The validity of test score interpretations about student growth over time depend strongly on the vertical linking design used to develop the vertical scale. But even when test score interpretations are appropriate to the scaling design, it is important to examine whether student gains may be interpreted consistently across subgroups or whether differential gain rates across subgroups limit the inferences that can be made about test score gains over time. To address this issue, we examine rates of student growth across student gender, ethnic, students with disabilities (SPED) and English language learners (LEP).
To evaluate differential growth across ethnic subgroups, a series of regression analyses were conducted to predict 2016 test scores from 2015 test scores, controlling for demographic subgroup membership. To compare ethnic subgroup performance, we created six dummy variables contrasting white students with each of other ethnic groups (e.g., white/Hispanic, white/African American). Gender was coded 1 for female. For SPED and LEP, SPED and LEP students were coded for 1, respectively.
The 2015 test scores were centered on the population mean so that the intercept represents the mean performance on the 2015 assessment. The slope represents the association between 2015 and 2016 test scores, controlling for demographic subgroups. A positive slope indicates students with higher test scores in 2015 have higher average levels of test scores in 2016.
Although many individual effects attained conventional levels of statistical significance due to large sample sizes, we focus here only on highly significant effects that are associated with more practically significant effect sizes and that may point to trends across grade level and/or subject area assessments. For all grades and subject areas, higher scores on the 2015 assessment were associated with greater growth on the 2016 assessment. Also generally consistent across grades and subjects, SPED students showed lower rates of gains than non-SPED students. Similarly, LEP students showed lower rates of gains than non-LEP students, but only during the elementary and middle school years. By high school, there were no differences in the gain rates between LEP and non-LEP students.
Differential gains by gender were observed, but they were not consistent across grades. Females showed lower rates of gain than males in both ELA and Math from grade 4 to 5 and from grade 8 to 9. A lower rate of gain for females was also identified in ELA from grade 6 to 7. However, females showed higher rates of gain than males in ELA from grade 5 to 6, and in math from Geometry to Algebra II.
With respect to differential gains across ethnic groups, Hispanics and American Indian generally showed somewhat lower rates of gain than whites across grades and subjects. African American students showed somewhat greater rates of gain than whites in ELA and math during elementary school years, but by the end of middle and high school years, the pattern was reversed.

Authors

• Myung Hee Im, American Institutes for Research

• Irene Hunting, Arizona Department of Education

• Stephan A. Ahadi, American Institutes for Research