Comparability of a Paper and a Computer-Based Cognitive Measure in a Low-Stakes Testing Environment

• In Event: Poster Session 9
  In Poster Session: New Developments in Psychometrics, Measurement, and Assessment: Poster Session 1

Abstract

The purpose of this study was to assess whether scores from paper-based and computer-based versions of a cognitive measure were equivalent and could be used interchangeably. This study demonstrated a set of methodological best practices, providing an accurate analysis of the degree of measurement invariance that exists across groups. Scalar, metric, and configural invariance were tested. Differential item functioning (DIF) was tested, and problem items were removed; measurement invariance across test modes was again evaluated. Results indicated that the measure was metric invariant (essentially tau-equivalent) across groups, and the DIF items didn’t impact the degree of measurement invariance found. Therefore, we conclude that the same construct was measured to the same degree, but scores cannot be considered equivalent without rescaling.

Authors

• Barbara Ellen Rowan, Pearson North America

• Joshua T. Goodman, National Commission on Certification of Physician Assistants

• J. Christine Harmes, University of South Florida