Investigating the Effects of Direct Linguistic Support Accommodations for English Language Learner Students on a Mathematics Assessment

• In Event: Validity of Accommodations for English Language Learners in NAEP

Abstract

Previous literature suggests that providing accommodations specific to ELL students’ needs, i.e., language barriers, would be beneficial for them in order to demonstrate their knowledge and skills in content assessments (Abedi, 2007; Rivera et al., 2006). Prior studies also found that linguistic features of math problems can interfere with ELL students’ ability to solve the problems (Spanos et al., 1988), supporting the provision of language-related accommodations for ELL students. However, only a few empirical studies are available to prove the effect of these accommodation types. If any, results have been inconclusive without sufficient explanations.
The present study investigated the effectiveness and validity of direct linguistic support accommodations for ELL students in taking a math test. This study focused on two particular accommodations for Grade 8 ELL students: read aloud of the entire test and glossary. These two accommodations are commonly allowed for many states’ math tests as well as for the National Assessment of Educational Progress (Rivera et al., 2006; Wolf et al., 2008). Examining the effectiveness and validity of the two accommodations will provide valuable information for a broad group of policymakers and practitioners.
This study employed both quantitative and qualitative methods to investigate the accommodation effects and students’ use of the accommodations. Quantitatively, a randomized experimental design was utilized where ELL and non-ELL students were randomly assigned into one of three conditions (i.e., read aloud, glossary, and no accommodation). A total of 605 students from two states participated in this study (313 ELL, and 292 non-ELL). An 8th-grade math test aligned with states’ math standards was developed. Regression analyses were conducted to examine whether ELL students performed differently depending on the given condition. Non-ELL students’ performance was also examined to test our hypothesis that a given accommodation would not affect non-ELL students’ performance on the test. Qualitatively, a student “think aloud” and retrospective interview was conducted with a sample of 68 ELL students in order to investigate the students’ problem-solving processes when a glossary was provided.
The results showed that the glossary accommodation had no significant effect on ELL students’ performance. The effect of the read-aloud accommodation was mixed. While State X ELL students had no statistically significant difference across conditions, State Y ELL students under the read-aloud condition performed significantly better than the sample ELL students with no accommodation. A notable finding was the significant interaction effect of the given accommodations and students’ prior content knowledge. A closer look at the think-aloud data indicated that while students were able to comprehend the text, they often lacked the prior content knowledge necessary to answer the items correctly. In this paper, we present these findings and others in more details and discuss practical implications for a valid and effective use of accommodations for testing.

Author

• Mikyung Kim Wolf, ETS