Flip It: An Exploratory (Versus Explanatory) Sequential Design Using Delphi Method Evaluating Potential Item Bias

• In Event: Mixed-Methods Design Considerations

Abstract

The Delphi Method (Delphi) has been adapted to inform item refinements in educational and psychological assessment development. An explanatory sequential design using Delphi is a common approach to gain insight from experts for why items might have exhibited differential item functioning (DIF) for a sub-group, indicating potential item bias. Use of Delphi before quantitative field testing to screen for potential sources leading to item bias is lacking in the literature. An exploratory sequential design is illustrated using Delphi in Phase I and DIF analyses in Phase II as an additional approach. A 2x2 integration typology is presented for connecting Phase I and II findings using a worked example from the development of the Problem-Solving Measures Grades 6-8 Computer Adaptive Tests.

Authors

• Kristin L.K. Koskey, Drexel University
  Presenting Author

• Toni A. May (Sondergeld), Drexel University
  Non-Presenting Author

• Yiyun Fan, Binghamton University - SUNY
  Presenting Author

• Dara N. Bright, Drexel University
  Non-Presenting Author

• Gabriel Matney, Bowling Green State University
  Non-Presenting Author

• Gregory E. Stone, University of Toledo
  Non-Presenting Author

• Jonathan D. Bostic, Bowling Green State University
  Non-Presenting Author