Evaluating Criteria for English Learner Reclassification: How Do We Know if the Chosen Reclassification Policy Is Appropriate?

• In Event: How, When, and Why Are English Language Learners Reclassified as Fluent English Proficient? New Methods for Establishing and Examining English Language Learner Assessment, Accountability, and Equity Policies

Abstract

Objective
When English learners (ELLs) are reclassified as fluent English proficient (R-FEP), often their instructional setting changes, including a significant reduction in or elimination of English language development services. Depending on the child’s language skills, this instructional change could hinder future development or provide needed opportunities for learning advanced material. By establishing assessment-based guidelines for reclassification, policymakers have tremendous influence on when these settings change. This paper highlights this policy lever for guiding reclassification decisions and identifies a method for rigorously evaluating whether the threshold for transitioning between settings is appropriate. This method—binding-score regression discontinuity (RD) with an instrumental variable (IV)—was then implemented to obtain unbiased effects of reclassification on academic outcomes for students on the cusp of meeting reclassification criteria to provide credible policy recommendations for maintaining or shifting assessment-based reclassification thresholds. The method detailed in this paper can be used by policymakers to evaluate their own assessment-based guidelines.

Methods
The goal is to examine the effects of reclassification at the district’s chosen threshold for reclassification. A rigorous quasi-experimental method—binding-score RD-IV—that can account for hidden biases that a standard regression approach may miss is proposed and applied to evaluate the effects of one district’s specified thresholds on various student outcomes (current and subsequent English language arts achievement, attendance, and advanced course-taking) and to illustrate how its estimates compare to those of other approaches (e.g., a standard regression approach), which condition on observable characteristics. The paper concludes with a discussion on how the methods can be extended to study cases of heterogeneity in reclassification effects.

Data
The longitudinal, student-level datasets for the analyses come from a large urban school district in California, with ELLs constituting about one-third of the district’s student population. Variables include school year, grade level, race, gender, special education status, English learner status, attendance, grade point average, CELDT scores and CST scores. The availability of multiple years (from 2001-2002 to 2006-2007) and cohorts of data, spanning 3rd through 11th grades, permits tracking individual students while accounting for prior achievement and changing statuses (e.g., changing from ELL to R-FEP status). Additionally, course history files allow for examining changes in advanced coursework (i.e., college-preparatory courses) that occur when a student is reclassified.

Results
The results for the effects of reclassification in high school are striking: Using the conventional approach (i.e., a standard regression model), it appears that reclassification results in significant positive outcomes of about +0.16 standard deviations on next year’s ELA test. However, such models cannot account for selection bias that is unobserved by researchers. The evaluation models proposed (binding-score RD-IV), which can account for selection bias, reveal that reclassification results in significant negative outcomes of −0.31 standard deviations for ELLs just barely meeting the reclassification criteria. Thus, the district’s chosen reclassification point is not appropriate because it does not ensure a smooth transition for high school ELLs; and there is sufficient evidence to pursue efforts to adjust the criteria levels and/or modify the curricula to better ensure a smooth transition.

Author

• Joseph P. Robinson, University of Illinois at Urbana-Champaign