Integrating Cognitive Interviewing Methods to Evaluate Response Processes Validity Evidence: Curricular Reasoning Self-Assessment Survey Suite

• In Event: Surveys in Action: From Technical Refinement to Real-World Influence

Abstract

Cognitive interviewing is used in pretesting surveys to collect data on participants’ thoughts while completing surveys to evaluate response processes validity evidence and inform instrument refinements. Researchers continue to systematize this methodology. The current study leveraged elements of two established cognitive interviewing methodologies. We illustrate embedding the Cognitive Validity Index within the Three-Step Test-Interview in a development and validation study of the Curricular Reasoning Self-Assessment Survey Suite (CR-SASS). Integration of the two techniques across four rounds of data collection with 12 participants was effective for identifying instrument refinements and evaluating response processes and consequences of testing validity evidences. Findings from one of eight CR-SASS subscales are reported. Implications are discussed for advancing pretesting methodology in survey development and validation.

Authors

• Kristin L.K. Koskey, Drexel University

• Amy M. Roth McDuffie, Washington State University

• Toni A. May, Binghamton University - SUNY

• Tara Heikila, Washington State University - Vancouver

• Yiyun Fan, Binghamton University - SUNY

• Dawn Teuscher, Brigham Young University

• Shannon Dingman, University of Arkansas

• Jonathan D. Bostic, Bowling Green State University

• Travis A. Olson, University of Nevada - Las Vegas