Selecting a Weighting Scheme for a Composite Score: Theory and Application

• In Event: 29.082 - Roundtable Session 9
  In Roundtable Session: 29.082-15 - Methodologies for Helping to Ensure Accurate Screening and Diagnostic Decisions

Abstract

Many assessments consist of more than one type of task or component but result in a single score reflecting a weighted combination of the individual task scores, for example structured observation and multiple choice tasks could be two parts of a single assessment. Component weighting can dramatically shift the meaning and interpretability of a composite score, inherently changing the validity of its use. When combining scores, a weighting scheme should be selected based on 1) the inferences to be made (the claims to be made about test takers) and 2) the type and structure of the elements to be combined. We discuss theoretical considerations and methodological applications in selecting and applying a weighting scheme for mixed-format tests.

Authors

• Amanda Clauser, National Board of Medical Examiners

• Kelly Jane Foelber, James Madison University