Effectiveness, Validity, and Comparability of Accommodated and Nonaccommodated Assessments for English Language Learner Students in NAEP

• In Event: Validity of Accommodations for English Language Learners in NAEP

Abstract

Federal legislation (e.g., NCLB) mandates inclusion of all students including English language learners. The National Assessment of Educational Progress, referred to as the Nation’s Report Card encourages inclusion policy by providing accommodations for ELL students. Many different accommodations are used in NAEP (see, for example, Abedi & Hejri, 2004). These accommodations include small group testing, one-on-one testing, read aloud and extension of testing time, large print, bilingual dictionary and computer testing. Studies on the effectiveness and validity of these accommodations are quite limited. The existing NAEP data many not help address effectiveness and validity of accommodations since ELL students are not randomly assigned to the accommodated and non-accommodated condition. As elaborated in the literature (e.g., Abedi, 2007) to examine validity and effectiveness of accommodations ELL and non-ELL students must be randomly assigned to accommodation conditions where a group of students receive accommodations (the treatment group) and another group of students are tested under standard testing condition where no accommodation is provided. In this session, we will discuss a study in which effectiveness and validity of accommodations were tested in a randomized field study.

Methodology
Validity and effectiveness of a computer-based accommodation system were examined through two different approaches, (1) an experimental approach in which both ELL and non-ELL students were tested under accommodated and non-accommodated condition and (2) a criterion-referenced approach in which accommodated and non-accommodated assessments were compared in their structural relationship using multiple group confirmatory factor analyses approach.
In this study performance of both ELL and non-ELL students in Grades 4 and 8 were compared under accommodated and non-accommodated testing conditions. The accommodations included a computerized administration of a math test with a pop-up glossary, a customized English dictionary, extra testing time, and small-group testing. Extra time and small-group testing were used for Grade 4 students only. A reading l composite score was used as a covariate. Results supported the use of computer-based accommodations and indicated that computer testing was the most effective accommodation in providing valid and accessible assessments for ELL students for both Grades 4 and 8. It is an alternative test item delivery and an easy-to-access gloss of non-math lexicon. This accommodation did not impact the validity of assessments.
In this symposium, we will present a demonstration of the computer-based accommodation that was used in this study and discuss the outcomes of this study. The focus of this presentation will be on the comparability of accommodated and non-accommodated assessments. If accommodations alter the focal construct, then the accommodated assessment outcomes many not be aggregated with non-accommodated assessment outcomes.

Author

• Jamal Abedi, University of California - Davis