Examining the Recommendations Provided by Highly Qualified and Minimally Qualified Subject-Matter Experts Using Three Standard-Setting Methods

• In Event: 61.086 - Roundtable Session 21
  In Roundtable Session: 61.086-17 - Issues in Validity and Licensure Testing Across the Professions

Abstract

Paul Brandon (2004) conducted an extensive review of the standard setting literature and concluded that selecting highly qualified subject matter experts “can enhance item estimation, but that not all judges need to have high levels of expertise. It is appropriate to select as judges people who have at least a general knowledge of the subject matter addressed in the examination.” (p.66). This study seeks to inform this body of literature by examining the recommended passing standards provided by highly qualified subject matter experts as well as minimally qualified subject matter experts. Using three standard setting methods I examine the passing standards recommended by both highly qualified and minimally qualified subject matter experts.

Author

• Michael Peabody, American Board of Family Medicine