Examination of Illustrations Glossaries for Deaf and Hard-of-Hearing Students

• In Event: 66.049 - Designing Illustrations Glossaries for Special Populations in a National Large-Scale Mathematics Assessment

Abstract

This study investigates how students who are deaf or hard of hearing (D/HH) interacted with embedded Illustrations Glossaries created for a nationally administered mathematics content assessment. In this presentation, we will share empirical evidence from cognitive labs conducted with D/HH students to showcase how D/HH students interpreted the Illustrations Glossaries and if the illustrations helped students gain access to the tested content. We will also share usability evidence from the students’ experiences.

Depending on the individual needs of D/HH students, accommodations may be necessary to support the student gain access to the tested content (Cawthon, 2015). There has been some evidence to indicate the use of carefully designed illustrations for English learners supports (Authors, 2014), but limited research to apply these accommodations to students who may also benefit from a non-traditional accommodation. This study examines the benefit of Illustrations Glossaries to provide support for D/HH students that would not benefit from American Sign Language accommodations.

We conducted cognitive labs with D/HH students at grades 4, 7, and 11. We specifically targeted D/HH students who were at beginning levels of ASL or who used other sign systems. In total, 13 students participated in the study. Their communication systems ranged from ASL to Spoken English to Pidgin Signed English (or any combination thereof). In instances that students could not be recruited to participate due to the strict inclusion criteria, students from neighboring grades were invited to participate (e.g., no D/HH 4th graders with the specific communication system were available, so a 5th grader that fit the rest of the criteria was invited to participate instead).

Demographic data was collected, as well as information about students’ communication system(s). Data was collected with one-on-one cognitive laboratories retrospective questioning to investigate how students interacted with the illustration based glossaries. We analyzed the qualitative data to explore themes related to students’ understanding of stand-alone illustrations (i.e., out of context) and their understanding of illustrations embedded in the test items, as well as their usability of the embedded illustrations.

Overall, the interpretations of many of the illustrations that were embedded within the test items were aligned with the intended meaning of the illustrations. Students found most of the illustrations to be clear, and there were no major usability issues observed. Data did suggest that for the D/HH population, the various communication systems and supports (e.g., interpreters) may pose a challenge for standardization of these features for high stakes assessment purposes. We will share these emerging themes and how they may have some potential impact for guidelines when creating Illustrations Glossaries for high stakes test purposes.

The findings from this study will add to the limited body of research on D/HH students taking large-scale assessments, specifically targeting students who use communication systems other than ASL. The results will inform future test development by providing guidance on the appropriate development and use of illustrations within mathematics test items for D/HH students.

Authors

• Teresa C. King, ETS

• Cara Cahalan Laitusis, ETS