Using Conversation-Based Tasks to Diagnose the English Language Skills and Math Knowledge of Middle School English Learners

• In Event: 49.057 - Conversation-Based Assessments

Abstract

English learners (ELs) are taxed with the double challenge of simultaneously learn how to effectively use a second language in an academic setting while learning the knowledge and skills in multiple disciplines through that second language. Now with the adoption of rigorous college-and career-readiness standards (e.g., Common Core State Standards), there is increased demand for students to acquire advanced levels of language competency in order to acquire content knowledge and skills. Moreover, scholars argue that there is a strong interdependence between language and achievement in mathematics and that there is a mathematical register comprised of special linguistic features that must be mastered by students who are studying mathematics (Schleppegrell, 2007; Spanos et al., 1988). This interdependence potentially increases the already difficult task ELs face to learn content knoweldge, as well as the difficult task teachers face when identifying methods of instruction and intervention to help ELs who are struggling.
Consequently, there is a clear need for more emphasis on assessments that can help screen EL students for content and language difficulties and pinpoint areas for instructional and additional practice. The purpose of this investigation was to explore the usability and authenticity of a technology-enhanced conversation-based diagnostic assessment to provide information about English learners’ language proficiency and mathematics content knowledge.
For this study, development focused on an interactive assessment task, or trialogue task. Trialogues are interactive conversations between a test taker and two virtual characters, and are used to create engaging realistic environments and to facilitate learning by supporting human-like interactions with computer-based systems. The purpose of this task is to simulate authenticity of a classroom environment small-group setting while eliciting language and math content knowledge to identify strengths and weaknesses in basic academic English language proficiency and mathematics content skills.
Unique features of the conversation based assessment, or trialogue task, and usability study results will be shared in the presentation. Twenty middle school students (ELs and Native English speakers) will be recruited to participate in 1:1 cognitive laboratories using usability and think aloud item protocols to elicit feedback from the students and to examine how they interact with this technology enhanced conversation-based assessment. A priori and open coding techniques will be used identify emergent themes in the student feedback. Codes will focus on major themes of perceived task difficulty- language, perceived task difficulty- mathematics problem solving, and task authenticity, as well as motivation and engagement.
Implications for current assessments will be discussed, as many assessments available on the market do not meet the needs of struggling students who are learning rigorous content instruction in their second language. The prevalence of these needs will only increase given the high number of EL students in today’s classrooms, and teachers’ needs to facilitate their students’ understanding of the rigorous content standards, by better understanding the source of their students’ difficulties.

Authors

• Alexis Lopez, Educational Testing Service

• Danielle Guzman-Orth, Educational Testing Service