Impact of Language on Validity and Fairness of Educational Test Scores: The Case of the United States and the Basque Country in Spain

• In Event: Students’ Diversity and Test Fairness

Abstract

In our globalized world, educational testing takes place in linguistically complex contexts that affect the validity of inferences based on test scores for some groups. In this paper we discuss validity issues derived from the linguistic interactions taking place in the context of educational assessment. The linguistic spheres relate to the language of instruction, testing, and the dominant-language of students.

Specifically, this paper investigates the validity of large-scale mathematics assessments administered in “officially mandated” monolingual and bilingual educational contexts. Differential item functioning procedures are employed to examine the impact of the language of testing on the fairness of fourth-grade mathematics achievements tests for subgroups for whom the language of testing does not coincide with the language of instruction or the student-dominant language.

Two cases are discussed. The first case illustrates a monolingual instructional setting in an “English-only” state of the USA. The second is a bilingual educational context in the Basque country in Spain. In the Monolingual Instructional Setting of the “English-only state”, English is the only official language in schools across the state. In consequence, classroom instruction is only provided in English. Educational testing is delivered in English only. However, students come from diverse linguistic backgrounds.

In the Multilingual Educational Context (Basque country), the education system operates with two official languages (Castilian and Euskera).

In both contexts, we analyze fourth grade mathematic assessments. DIF detection samples include about 70,000 fourth graders in the US and 3,000 students (ages 10-12) in the Basque country. For each of the tests, Differential Item Functioning was conducted for the following reference and focal groups. The Reference Groups for estimating DIF include students for whom the student-dominant language corresponds to the language of instruction and the language of the test, in both the US and the Basque contexts.
In the US, the Focal Group is the group of students for whom the student-dominant language (one of forty languages other than English) does not correspond to the language of instruction (English) and the language of the test (English). In the Basque country, the Focal Group is the group of students for whom the dominant language (Euskera) DOES correspond to the language of instruction (Euskera); but differs from the language of the test (Castilian).

In both contexts, we found high percentage of ítems with DIF (greater than 20%) when the language of the test differed from the students’ dominant language. In the US case, the size and direction of DIF was associated with the linguistic complexity of the language in the items. The more complex the item language the greater its difficulty for students not proficient in English compared to native English speakers with equal mathematics scores.

To conclude, a validity framework relating language of instruction, language of testing, and dominant language of students in the assessment of mathematics will be discussed.

Authors

• Maria Martiniello, Tomas Rivera Policy Institute

• Paula Elosua, University of the Basque Country