Investigating Approximate Measurement Invariance in the Doctor-Patient Humanistic Domain across Gender Groups

• In Event: 14.021 - Evaluation of Measurement Invariance

Abstract

Investigations of test validity for licensing examinations should extend beyond the job analysis to methods providing statistical evidence associated with construct-irrelevant variance. This paper applies the multi-group analytic alignment method proposed by Asparouhov and Muthén (2014) to assess factor means and factor variance among subgroups. The purpose of this study is to explore significant non-invariant parameters among candidate and standardized patient (SP) gender groups on items measuring clinical skills in the doctor-patient Humanistic domain using the alignment method framework. Data represents 2,384 female and 2,590 male students from 31 osteopathic medical schools across the United States tested in the 2013-2014 cycle. Findings from this study showed groups in rank order of highest to lowest factor means were female candidates rated by female SPs, male candidates rated by female SPs, female candidates rated by male SPs, and male candidates rated by male SPs. The alignment method identified invariant and non-invariant parameters that could inform test development or rater training. Continuous evaluation to identify sources of construct irrelevant variance should be a priority of test agencies to provide evidence of validity for claims made regarding test score interpretations.

Authors

• William L. Roberts, National Board of Osteopathic Medical Examiners

• Ping Yang, University of Missouri - Columbia