The Effects of Changing a Threshold-Based Policy for Reclassifying English Learners: A Difference-in-Regression-Discontinuities Approach

• In Event: Excellence in Education Research: Early Career Scholars and Their Work

Abstract

Language-minority students who enter the school system as English Learners (ELs) are expected, during their time as students, to be reclassified as Fluent English Proficient (R-FEP). Policy considerations tend to focus on the speed with which ELs are reclassified, with the specific goal of reaching R-FEP status quickly, often within a predefined number of years. To reach R-FEP status, ELs often must attain test-based thresholds set by policymakers. Because reclassification can affect student achievement through altering the instructional setting, recent studies have used regression discontinuity designs (RDD) to investigate threshold-based reclassification effects (e.g., Matsudaira, 2005; Robinson, 2011; Slama, 2012; Thompson, 2012; Umansky, 2012). In this paper, we extend this work to examine the effects of policy changes in reclassification thresholds. We implement a series of “difference in regression discontinuity designs” to estimate these policy effects, and in the process, we also estimate the RDD-based effects (based on RDD approaches for multiple rating scores because reclassification requires attaining several test-based thresholds; see Reardon & Robinson, 2012; Wong, Steiner, & Cook, 2013).
We used eight years of student-level administrative data from the Los Angeles Unified School District spanning a period of two different reclassification test policies: In 2007, the California Department of Education introduced a revamped English language development test with more linguistically complex items, intended to assess students’ proficiency with academic English used in English-only classrooms. Before 2007, the threshold required for attaining reclassification eligibility was implicitly lower and contained fewer linguistically complex items. We examine the effects of this state-level exogenous policy change on ELs by simultaneously estimating RDD-based effects prior to 2007 and again after 2007.
Prior to the 2007 test change, we find that reclassification associated with attaining the policy-based threshold resulted in negative effects on subsequent English language arts tests of about one-third of a standard deviation. These negative effects suggest that the reclassification criteria were not stringent enough and that ELs were being reclassified as R-FEPs when they were still benefitting from the EL instructional setting. These negative effects were found in both elementary and secondary school. Following the 2007 test change, we find no evidence of a reclassification effect on achievement. Importantly, there are significant differences between the pre-2007 and post-2007 effects, suggesting that the test-policy change itself had an effect. In this case, the test-policy change was fortuitous because it replaced the negative (i.e., disruptive) achievement effects with null (i.e., non-disruptive) achievement effects for students on the cusp of threshold-based reclassification, suggesting that the revamped English language development test more accurately assessed students’ readiness for reclassification.
In our main analyses, as well as across several different specification checks (some of which are novel approaches), we find consistent evidence that a policy change in requirements for EL reclassification had significant effects on students’ achievement trajectories in both elementary school and secondary school. This research has implications for various policies affecting ELs, including the increasing centralization of criteria for determining EL status and the establishment of reclassification criteria under new Common Core State Standards-aligned assessments.

Author

• Joseph P. Robinson-Cimpian, University of Illinois at Urbana-Champaign