Issues of Construct Validity in Problem-Solving Assessments: Generalizability and Invariance Across Schools and Countries

• In Event: Roundtable Session 14
  In Roundtable Session: Evaluating Measurement Invariance

Abstract

In large-scale studies such as PISA and TIMSS, problem solving has gained importance, especially when comparing students’ performance in different groups. But until now, little attention has been paid to the generalizability of measurement models across schools and countries. Consequently, the present study aims to analyze sources of variance in order to check for measurement invariance across these groups. By applying multi-group and multilevel item response theory modeling to the PISA 2003 study data sets for Germany and Australia, differential item functioning in the problem-solving assessment has been identified at different levels. The results indicate substantial effects for schools and countries which threaten construct validity. Implications for establishing appropriate measurement models are given.

Author

• Ronny Scherer, University of Oslo