Using Many-Faceted Rasch Measurement (MFRM) to Evaluate a Teacher Observation Instrument

• In Event: Modern Applications in Item Response Theory

Abstract

The Evidence-Centered Design framework was used to create a special education teacher observation system, Recognizing Effective Special Education Teachers (RESET). Extensive reviews of research informed the domain analysis and modeling stages, and led to the operationalization of effective special education teaching as the ability to effectively implement evidence-based practices for students with disabilities. In the assessment implementation stage, a 4 raters evaluated 40 videos and provided evidence to support the assigned scores. An inductive approach was used to create empirically derived, item level descriptors. In the assessment delivery stage, 12 raters evaluated 90 videos using the fully developed rubric. MFRM analyses showed that the item, teacher, lesson and rater facets achieved high psychometric quality.

Authors

• Evelyn Johnson, Boise State University

• Angela Rae Crawford, Boise State University

• Yuzhu Zheng, Boise State University

• Laura Moylan, Boise State University