The Pragmatic Arrangements of English Language Proficiency Standards and Assessments: Content Analyses and Teacher Perceptions

• In Event: Validity and Utility of English Language Proficiency Assessments: New Evidence in the Common Core Era

Abstract

Purposes

This study was part of a larger project to identify the essential language knowledge and skills that may assist English learner (EL) students to access academic curricula. Specifically, we examined how expectations for students using language to interact with others (i.e., pragmatic language) are expressed in fifth-grade English language proficiency (ELP) standards, related ELP assessments, and by teachers of EL students.

Perspectives

The latest generation of ELP assessments are designed to measure the language demands and range of contexts in which language is used in school. While attention has been paid to linguistic forms and functions (e.g., linguistic complexity, vocabulary choices, explanation, argumentation), less attention has been paid to the nature of pragmatic demands placed on students. For example, pragmatic demands might include whether students are expected to interact with a range of different audiences, with different configurations of participants (e.g., one-on-one, small group), and be familiar with conversational and monologic skills. These skills and knowledge may not be readily acquired without opportunities to experience a range of interactions and instructional contexts and thus cannot be ignored if students are to put their formal language skills to use to learn effectively.

Method

Researchers conducted content analyses of two sets of ELP standards and related assessment materials for two states. The categories of analysis included (1) modes of expressive and/or receptive language, (2) audience (e.g., other students, teachers), (3) one-to-one, one-to-many interactions, (4) conversational turn-taking, (5) register (e.g., degree of formality, including academic or social usage), and (6) use of various resources (e.g., use of L1, visual representations). Additionally, several focus groups were convened to gather the expectations of 26 teachers on pragmatic skills from standards and their instruction.

Results

Both sets of standards included interpretive, productive, and collaborative modes as key communicative purposes, and students were expected to answer questions and support their responses with details, modify language for different registers and use non-linguistic resources (e.g., graphic organizers). Pragmatic expectations in standards of State A had some similarities to their ELP assessment, including monologic interactions (e.g., student presentations) and reliance on non-linguistic resources across the modalities. The standards also highlighted a wide range of dialogic configurations (e.g., dyads, small groups, whole class), whereas ELP assessment stimuli/items less so. Peers were an overtly-mentioned audience in State B’s standards and a range of dialogic configurations (including collaborations) were expected; the ELP assessment focused on teachers as audience and student-teacher conversations. Both sets of standards and assessments placed a heavy emphasis on non-linguistic resources. Focus groups revealed that many teachers utilized non-linguistic resources to foster communication and several teachers were aware of the challenge of whole-class interactions. However, some doubted ELP assessments could, as they are, “assess the communication skills that we practice in class.”

Significance

In order to comprehensively identify the language demands of school, efforts by states and consortia should consider pragmatic features in their articulation of alignment between ELP standards and assessments and explore social-communicative aspects of language demands that may offer valuable targets for future ELP assessment.

Authors

• Alison L. Bailey, University of California - Los Angeles

• Mikyung Kim Wolf, Educational Testing Service

• Anahit Pogossian, University of California - Los Angeles

• Laura Ballard, Educational Testing Service