Future Directions of Educational Measurement in International Large-Scale Assessments

Session Type: Symposium

Abstract

The methodology of large-scale assessments needs to be constantly scrutinized and improved according to the state-of-the-art in order to assure a fair assessment, high quality data, and appropriate conclusions from substantive analyses. This symposium addresses central methodological challenges of international large-scale assessments. In four single presentations, the symposium discusses different sources of bias in the data, from imprecisions and differential item functioning in the estimation of achievement scores, to international differences in guessing behavior and unintended response difficulties in questionnaire items. Furthermore, the symposium derives specific conclusions for applied research with international large-scale assessment data, for the interpretation of results, and the future improvement of the assessments.

Sub Unit

• SIG-Large-Scale Assessment

Chair

• Isa Steinmann, TU Dortmund University

Papers

• The Effect of Conditioning Model Choice: A Simulation Study - Laura Zieger, University College London; John Jerrim, University College London

• Differential Item Functioning Analysis in International Large-Scale Assessments: From a Traditional to a Network Perspective - Edwin Cuellar Caicedo, Cito; Timo Bechger, ACT, Inc.; Maarten Marsman, University of Amsterdam; Gunter Maris, ACTNext

• The Effects of Incorrect Answer Substitution in PIRLS - Andrés Christiansen, Katholieke Universiteit Leuven; Rianne Janssen, KU Leuven

• Identifying Inconsistent Respondents to Mixed-Worded Scales in Large-Scale Assessments - Isa Steinmann, TU Dortmund University; Johan Braeken, University of Oslo; Rolf Strietholt, TU Dortmund University

Discussant

• Eugenio Gonzalez, ETS