A New Threat to Validity: An Examination of Cultural Discrepancies in Omission Rates on International Assessments

• In Event: Reassessing International Assessments

Abstract

As one result of international standardized assessments, judgments about comparative student achievement are used to inform educational policy in many countries. However, due to some unaddressed sources of potential bias, these comparisons and the decisions which grow from them may lack validity. This study seeks to investigate systematic, geographical patterns of omission rates on mathematics and science items on the TIMSS and PISA. In addition to correlational examinations of the publicly available datasets, the authors will use IRT-based DIF detection methods to locate differentially functioning items. The results of this study will inform the international education community of previously unidentified and serious threats to validity. Additionally, we offer suggestions to increase the equity and fairness of cross country comparisons.

Authors

• Susan Gillmor, The University of Kansas

• Tanya Longabach, University of Kansas Medical Center

• John Poggio, The University of Kansas

• Elena C. Papanastasiou, University of Nicosia