Fairness of Language Tests for Students From Various Linguistic Backgrounds

• In Event: Students’ Diversity and Test Fairness

Abstract

Guaranteeing for valid and fair assessments of linguistic skills for students from various linguistic backgrounds is one of the most challenging issues confronting researchers, teachers and politicians. The obstacle of treating every test taker fairly becomes even higher in regard to the growing diversity among students in educational systems. In standardized tests native speaking students often perform significantly higher than students communicating in a language other than the host language at home. It is assumed that the observed achievement gaps aren’t just the result of differences in linguistic competencies but arise in part due to unfair test items. To evaluate the fairness of a commonly used language test three aspects are addressed in this study; equal precision of measurement and similarity of the factor structure of the test across examinee groups, and differential item difficulty due to group membership.

In international student assessments differential advantages for students in reading tasks from their own language and/or cultural area constraint the comparability of the collected data. The existence of comparable effects among different groups, who live in the same local zone, but have different linguistic backgrounds, has been shown for English Language Learners (ELLs) compared to native English speakers or non-ELLs (e.g., Young et al., 2009). Studies examining DIF have yielded that items, which exhibited DIF against ELLs, are often high degree language complexity items (e.g., Martiniello, 2009). Existent studies address the impact of construct irrelevant language difficulty on test outcomes. In language tests the language complexity is part of the construct which forces researchers to identify potential sources of unfairness which are other than language complexity. Moreover it is of great interest to investigate whether the findings for ELLs can be replicated for comparable subgroups in other contexts, to explore and offer new perspectives regarding the problem of unfair tasks in skill-based assessments.

The study included 288 4th graders recruited in their primary schools in Germany. The students worked on a commonly used German vocabulary test. The sample consisted of 81 students communicating in German at home and 207 students who spoke both German and another language at home. To identify the internal consistency across the examinee groups, Cronbach’s alpha and related alternatives were calculated. Unidimensionality was analyzed using confirmatory factor analysis and nonparametric approaches. Mantel and Haensel’s approach was used to detect DIF items. To corroborate the findings, Lord’s IRT-based approach was applied additionally. All statistical analyses and graphics were produced in R.

The data suggest the presence of test unfairness: The test is less reliable for the focused group, and the factor structure differs among examinee groups. The two DIF approaches detected items favoring the students communicating in German at home. The identified items indicate that unfairness may occur because of different previous knowledge between examinee groups.

In conclusion, the study points to previous knowledge as another significant impact factor along with linguistic complexity. This contributes to the current and important objective of research and educational practice to ensure fair tests for every student.

Authors

• Franziska Schwabe, Technical University of Dortmund

• Miriam Marleen Gebauer, Technical University of Dortmund

• Wahiba El-Khechen, Technical University of Dortmund

• Ali Ünlü, Technical University of Dortmund

• Nele McElvany, TU Dortmund University