Moving Toward a Common Definition of English Learner: Validation of a Language Use Survey Protocol

• In Event: Strengthening the Validity and Utility of Home Language Surveys to Identify Potential English Learners

Abstract

Federal policy and case law require schools to provide English language educational support services to students for whom English language proficiency constitutes a barrier to academic achievement. Identifying English Learners (ELs) is foundational to providing this required support. A common identification process is to administer a Home Language Survey (HLS) to all students at initial school enrollment, with indications of non-English use at home triggering administration of an English Language Proficiency (ELP) screener. Research has identified many issues with current HLSs, including significant variance in questions between HLSs, and lack of nuance leading to inappropriate screening decisions (Linquanti & Bailey, 2014).

The 35-state ASSETS (Assessment Services Supporting ELs through Technology Systems) Consortium’s EL Definition Subcommittee, comprised of state education agency representatives and researchers, has been working in conjunction with WIDA and other multi-state assessment consortia to establish a common HLS protocol. The committee’s work is grounded in recent research (Bailey & Kelly, 2013; Linquanti & Cook, 2013; Linquanti & Bailey, 2014) supporting a standardized method for identifying ELs, and has been focused on using data-driven metrics for HLS design and improvement. Model survey language has been drawn from these publications, including insights on substantially more nuanced questions and processes which could lead to more accurate initial screening decisions.

A major design consideration is that an HLS has to be streamlined and easy to administer, but nuanced enough to produce a reliable decision to screen (or not). Brevity is required as the HLS is administered to every student, while the nuance must address such students as recent immigrants, those exposed to multiple languages, international adoptees, and others with complex linguistic backgrounds. These challenges led to the design of a hybrid EL identification protocol: An HLS is used to collect initial data, which in turn determine whether further, more probing questions are needed. This protocol is called a Language Use Survey (LUS).

WIDA researchers have designed a multi-stage validation process to determine how well the LUS determines whether or not to screen students for initial English language proficiency. The first stage of the validation process involved a cognitive lab designed to answer the following:

-Is this tool able to adequately determine which students need to be screened for ELP?
-Is it usable by a diverse group of educators and school personnel working in a variety of educational settings?
-Can we demonstrate that the process and questions yield reliable results?

Educators were asked to review profiles of hypothetical students with complex linguistic backgrounds and answer LUS questions as if the information had been given by a parent or guardian. More than 100 educators participated in site-based and online labs, and analysis of these labs shows 85% agreement in determining which students should be screened.
Areas of disagreement and participant feedback indicated clear ways to improve the LUS. These findings will be presented along with a framework for implementing the next stage in the LUS validation process, one designed to provide empirical evidence of validity and reliability.

Authors

• Audrey Lesondak, Wisconsin Department of Public Instruction

• Howard Gary Cook, University of Wisconsin

• Jesse Roberts, WIDA Consortium