Using Propensity Score Matching to Compare Questionnaire Responses Between Two Large-Scale Assessments

• In Event: Psychometrics and Educational Measurement Poster Session 2

Abstract

Questionnaires administered within large-scale assessments (LSAs) provide useful information regarding students’ educational experience. When different LSAs are administered at different time point, it becomes difficult to track students’ responses across years due to comparability issues. This study employed the method of propensity score matching to create similar groups of students for comparing trends of questionnaire responses across two years. Canadian questionnaire data from PCAP 2013 and PISA 2015 were used to establish similar groupings for comparisons, specifically the items related to learning of science and students’ attitude toward science. Results found statistically significant differences between student responses on the PCAP 2013 and PISA 2015 questionnaires. This study suggests a possible method for establishing comparability of different LSAs’ questionnaire responses.

Authors

• Karen Fung, University of Alberta

• Man-Wai Chu, University of Calgary