A Deeper Look at Deeper Learning: Detecting Differential Item Functioning Using Explanatory Item Response Theory Models

• In Event: Excellence in Education Research: Early-Career Scholars and Their Work

Abstract

Deeper learning has become increasingly important, yet the understanding of deeper learning remains limited. The Study of Deeper Learning conducted by the AIR scientists has provided great insights with regard to effective ways to promote deeper learning in schools. However, further studies are needed to better understand some key aspects of deeper learning. Specifically, the ordered logit Rasch model (Yen, 1986) was used to examine students’ key competencies such as creative thinking skill, interpersonal and intrapersonal skills. But Rasch models only focus on item difficulty, and item discrimination was left unexamined. Moreover, it is unclear whether the survey items are able to measure the underlying constructs equally well for different groups of students. Although many methods can be used to detect differential item functioning (DIF), the explanatory item response theory (IRT) model approach has been gaining popularity because it has several advantages. For example, it does not require identifying a set of unbiased anchor items. It not only allows researchers to examine uniform and non-uniform DIF simultaneously, but also allows them to compare multiple groups multiple items at once. Hence, in the present study, explanatory IRT models with person-by-item predictors were employed to reexamine all the student survey item properties and detect DIF. Results indicated that for all eight instruments, items indeed varied in terms of item discrimination and item difficulty. Also, further DIF analyses revealed that a number of items did not perform equally well for all subgroups of students. Differentially-functioning items are able to produce measurement bias and should be either deleted or modeled. The detailed implications for researchers, policy makers and practitioners are discussed.

Author

• Meihua Qian, Clemson University