Developing and Piloting an Observation Tool for Multilingual Learners in Preschool and Kindergarten Classrooms

• In Event: AERA Roundtable Session Thursday 9:45 am - Gold 2
  In Roundtable Session: Fostering High Quality Early Learning Instruction for Dual Language Learners through Classroom Observation Research

Abstract

This paper details the development, piloting, and initial findings of a classroom observation tool designed for preschool through kindergarten settings serving multilingual learners (MLs).

Objectives. The first objective was to create a classroom observation tool capable of systematically documenting the frequency and quality of teacher and student behaviors that foster early language and literacy development, including developmentally appropriate academic language and content knowledge, in preschool through kindergarten. A second objective was to establish high inter-rater reliability for the observation tool.

Framework. The tool's development is grounded in extensive research highlighting instructional practices beneficial to MLs (Gunn et al., 2020; Villegas & Lucas, 2002). This framework emphasizes the importance of instruction that cultivates oral language proficiency, academic vocabulary, complex linguistic structures, and foundational early literacy skills. Furthermore, it incorporates the pedagogical principle that culturally and linguistically sustaining practices significantly enhance bilingual language development and academic content learning (Yoon, 2023).

Methods. The project began with a thorough literature review of classroom observation tools for MLs in U.S. early grades (e.g., Chiang et al., 2017). Notably, we identified the other two tools featured in this symposium as particularly relevant and influential in the development of our tool: the CASEBA for use in bilingual early learning classrooms, and the COLES-DLL for use across early learning classrooms serving DLLs. Additionally, we drew insights from a detailed "depth of implementation" tool utilized by a collaborating nonprofit organization that provides professional development for educators serving MLs.

Next, we designed the new tool to assess teacher and student behaviors promoting academic vocabulary, complex language use, oral language development, and early literacy skills. The tool also evaluates the linguistic and cultural representativeness of the classroom environment. Each item is scored distinctly for English and students’ home language(s).

Data sources. A pilot study was conducted in May 2025 across three classrooms. In addition to the pilot study, the presentation will report on findings from 50 classroom observations conducted across 25 classrooms in California during the 2025-26 academic year. Each classroom will undergo two observations: one in the fall and one in the winter/spring.

Results. Preliminary results from the May 2025 pilot demonstrated an inter-rater reliability exceeding 80 percent, indicating strong consistency in scoring. The presentation will report comprehensive inter-rater reliability data and descriptive findings derived from the full 2025-26 study.

Significance: This newly developed classroom observation tool addresses gaps in existing instruments. It captures teacher and student behaviors, providing a more holistic view of classroom dynamics. Its versatility allows for application across diverse instructional settings, including English-medium and bilingual classrooms, spanning early education through kindergarten. Furthermore, the tool’s streamlined design significantly reduces observation time (from approximately 3 hours to 45-60 minutes) and simplifies the scoring protocol. This efficiency, coupled with a shorter observer training period (approximately five hours during the pilot phase), enhances the instrument's practicality and scalability for widespread research and professional development applications.

Authors

• Camille Whitney, Sobrato Early Academic Language (SEAL)

• Martha Irene Martinez, SEAL

• Guadalupe Diaz Lara, California State University - Fullerton