A Multilevel IRT Model with an Application to DIF Detection for a Teacher Evaluation Instrument

• In Event: Exploring Measurement Constructs in Global Learning

Abstract

Multilevel item response theory (IRT) models have been suggested as a potentially useful methodology for detecting differential item functioning (DIF) of a test or a rating scale. The purpose of this study is to construct a hierarchical generalized linear model (HGLM) based on the IRT framework to analyze a teacher evaluation instrument with ordinal categories and then to extend the IRT models in detecting DIF by considering respondents’ personal attributes. This multilevel model assumes a random intercept representing students’ ability of judging their instructors’ teaching behaviors and fixed effects for the item indicators and other personal attributes. Results of this study indicated that personal attributes failed to biase their ratings of teaching quality and this instrument successfully elicited specific issues of teaching behavior. This study suggests useful modeling extensions of IRT brought by HGLM including level indicators and personal attributes as covariates that would impact item responses for the purpose of detecting DIF.

Author

• Xiaodan Tang, UIC