Rapid Development to Design Interventions: Incorporating Qualitative Data in Real Time

• In Event: 31.091 - Fairmont Royal York Roundtable Session Five
  In Roundtable Session: 31.091-8 - Interventions, Validation, Fidelity, and Data Congruence

Abstract

Improvement science and randomized control trials (RCTs) offer traditionally separate yet complementary advantages for the design and evaluation of educational interventions. Using two case studies, the authors argue for integrating improvement science principles into the design and execution of RCTs. First, feedback from college students in a large-scale RCT informed revision of materials in a utility-value intervention, yielding an activity that participants found more relevant. Second, free responses submitted by ninth grade students in the context of a social belonging intervention were analyzed to update the intervention materials, producing greater internalization of key messages and yielding previously unobserved positive effects on academic performance. Results suggest the promise of incorporating principles of improvement science into the experimental evaluation of educational interventions.

Authors

• David Silverman, University of Virginia

• Megan Moran, University of Virginia

• Christopher Leland Williams, University of Virginia

• Quinn Hirschi

• Yoi Tibbetts

• Christopher Hulleman