Bias Risks Related to Nonparticipation: Evidence From a Longitudinal Large-Scale Survey in Germany (PISA Plus)

• In Event: 43.050 - WITHDRAWN: New Research Methods for Large-Scale Assessments

Abstract

This study uses evidence from a longitudinal survey (PISA Plus, Germany) to examine the potential bias risks in cross-sectional large-scale assessments (ILSAs). Validity and adequacy of the strict participation rate requirements in ILSAs are evaluated. The study provides evidence for significant selection bias regarding student competencies and background variables when participation is voluntary. Using this evidence, likely magnitudes of bias in cross-sectional studies in varying scenarios are illustrated. Accordingly, if combined participation rates drop below 50%, even a small difference between non-respondents and participants will cause significant bias. It is concluded that participation rate requirements established in most ILSAs are fully valid, reasonable and important as they ensure low risks of biased results.

Authors

• Sabine Meinck, IEA Hamburg

• Joerg Henrik Heine, Technische Universität München

• Julia Mang, Technische Universität München

• Gabriel Nagy, Leibniz Institute for Science and Mathematics Education