Are Certain Kinds of Measures More Sensitive to Intervention Effects?

• In Event: Localized Validity Evidence for Specific Assessment Purposes

Abstract

It is important to carefully consider what outcome measures are used in intervention research, as the best designed studies will be unable to detect differential outcomes for intervention participants if the outcome measure is poorly constructed or is only proximally related to the target of the intervention. This study examines the instructional sensitivity of different types of outcome measures used in experimental and quasi-experimental research. Preliminary meta-analysis reviewed publications from 2007 through 2018 in JREE, JEP, and AERJ found that most researchers heavily favor either off the shelf or researcher-developed assessments. Preliminary results suggest that the nuanced nature of researcher-developed measures may overcome any psychometric challenges. However more research is necessary further elucidate the effects of measure type on effect size.

Authors

• Erin Winters, University of California - Davis

• Megan E. Welsh, University of California - Davis

• Scott C. Marley, Arizona State University