Experimenting With Different Standard Setting Methods: Lessons Learned

• In Event: Expanding Horizons in Educational Assessment and Validation

Abstract

The current study describes a series of experimental standard-setting studies using the bookmark method (Mitzel et al., 2001) and the profile method, a variation of the Body of Work method (Kingston & Tiemann 2012). For each method, we conducted several experimental standard-setting studies to collect panelist feedback about the study procedures and to build the feedback from one study to the next. Additionally, we collected panelist ratings to conduct preliminary cut-score analysis across the different panels and between the two standard-setting methods. In this presentation, we will present lessons learned along the way from panelist feedback about the standard-setting procedures. Finally, preliminary results of cut-score analysis and rating reliability will be discussed.

Authors

• Chih-Kai Lin, National Council of State Boards of Nursing

• Hyung Jin Kim, National Council of State Boards of Nursing

• Luping Niu, National Council of State Boards of Nursing