Testing the Convergent and Discriminant Validity of Test Fairness Review

• In Event: Roundtable Session 14
  In Roundtable Session: Innovative Qualitative Methodologies

Abstract

The purpose of this study is to examine the convergent and discriminant validity of the two methods that are widely used to ensure test fairness in large-scale assessments. One method is called a sensitivity review, examining test items based on the narrative comments and ratings from expert reviewers. The other is detecting biased items based on a statistical method called Differential Item Functioning (DIF). Because these processes require a large amount of time and financial investment, it is important to identify if using a combination of both methods better ensure fairness of assessments than using one method. This study shows how sensitivity review meaningfully contributes to enhancing test fairness in addition to the statistical item analysis using mixed methods.

Authors

• Ah Young Shin, University of Iowa

• Michelle Rogers, University of Iowa