Unveiling Insufficient Effort Responding in Self-Report Surveys

• In Event: Survey Science in Education: Methods, Analysis, and Emerging Technologies

Abstract

Self-report surveys, prevalent in applied linguistics, face threats from insufficient effort responding (IER), where participants provide inattentive answers, compromising data integrity. This study explores IER in a survey of L2 teachers’ assessment literacy (N=641), using attention checks, self-reported effort, response time, and the lz index. Findings indicate 2.1% to 25.8% of respondents showed IER, inflating means, increasing variability, and lowering reliability and model fit. Older teachers and those motivated by survey relevance or social ties exhibited lower IER. Recommendations include streamlined survey design and multi-method IER detection to improve data quality, advancing reliable research in applied linguistics.

Authors

• Melissa Dan Wang, The Education University of Hong Kong

• Xiaoli Su, The Chinese University of Hong Kong

• Icy Lee, Chinese University of Hong Kong