From Paper-Pencil to Digital: Administration Mode Effects on Home Survey Nonresponse in International Large-Scale Assessments (Poster 2)

• In Event: Division H: Global Perspectives on Standardized Assessment

Abstract

The home questionnaire, or "Early Learning Survey," is crucial for the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS), collecting vital information about students' home learning contexts from parents. However, these surveys often face higher non-participation rates than student and teacher questionnaires, risking representativeness. Since 2016, countries can administer these surveys via paper or computer. This study examines how administration mode impacts response rates. Digital surveys are faster and more cost-effective but often see decreased participation compared to paper-based surveys, potentially due to technology barriers and distractions. Analyzing data from PIRLS and TIMSS across several cycles, we investigate 78 educational systems using various administration modes. We plot participation trends by administration mode and use a difference-in-differences model to assess the impact on nonresponse. Findings show a significant 23% decline in responses when switching from paper to digital, with no notable impact from mixed modes. This study provides international evidence on the effects of survey mode on response rates, emphasizing the trade-off between efficiency and participation, with implications for ILSA participants.

Authors

• Alec Kennedy, IEA Hamburg

• Yuan-Ling Liaw, IEA Hamburg

• Rolf Strietholt, IEA Hamburg

• Christian Christrup Kjeldsen, IEA

• Rune Müller Kristensen, Aarhus University