English Learners' Difficulties in Solving Mathematics Items Aligned With the Common Core: Implications for Assessment Accessibility

• In Event: Validity Issues in Assessing English Learners

Abstract

With recent reform efforts to prepare all students for college and careers, new academic standards such as the Common Core State Standards (CCSS) have started to be implemented in K-12 schools. Some major characteristics of the CCSS include increased academic rigor and language demands across all disciplinary areas (Hakuta, Santos, & Fang, 2013). The expectations for students’ academic language skills are not only manifested in English language arts but also in mathematics. The eight standards for CCSS-Mathematics Practice (e.g., making sense of problems, constructing viable arguments and critiquing the reasoning of others) involve an increased language demand for students (Moschkovich, 2012). Inarguably, these new academic standards will be extremely challenging for English learners (ELs) who are still developing their English skills while learning rigorous academic content.

The validity of content assessments for ELs will become a more complicated issue as language demands can form part of the construct. While the consortia delivering Common Core assessments have developed new accessibility and accommodation guidelines, it is important to provide empirical evidence about the sources of difficulties ELs experience in the new CCSS-based content assessments. This study is part of a large-scale, multi-year research project funded by the U.S. Department of Education with an overall goal of examining the effectiveness of accommodation types for ELs in mathematics assessments. The purpose of this study is to investigate ELs’ processes and strategies in solving CCSS-based mathematical items. Specifically, this study aims to identify the types of difficulties ELs encounter in solving these items, which is essential to identify potential accommodation types.

This study undertook small-scale cognitive labs consisting of 20 Grade 8 students who also participated in the larger pilot testing with 221 ELs and 412 non-ELs. Efforts were made to sample students with a range of English language proficiency including non-ELs. The sample included 11 ELs and nine non-ELs from one urban school. Students were asked to complete five items selected from the pilot test form, covering a range of content difficulty, linguistic complexity, and graphic representation. During the cognitive labs, students performed a think-aloud and were then asked questions about each item by an interviewer. The interviews were recorded and transcribed. The transcripts were analyzed based on a coding scheme focusing on (1) comprehension of item language, (2) problem-solving skills, (3) item difficulty, (4) precision in mathematical explanation, and (5) opportunity to learn. NVivo software was used to organize the data.

The results suggest that students generally perceived greater difficulties for CCSS-based items partly due to the large amount of language required to process the items. For example, the item with the highest percentage of correct responses required students to solve a simple equation, which imposed little-to-no language processing demands. One notable finding was students generally lacked strong foundational skills (e.g., multiplication with several digits, percentage calculation). In this presentation, we will share the results in detail as well as implications for the development of appropriate accessibility and accommodation features in assessing ELs.

Authors

• Mikyung Kim Wolf, ETS

• Danielle Guzman-Orth, Educational Testing Service

• Jennifer Wain, Educational Testing Service

• Jamal Abedi, University of California - Davis

• Florencia Tolentino, ETS