A Parable of the Measurer: Disentangling Adjacent Categories in Ordered Multiple-Choice Assessments

• In Event: AERA Roundtable Session Saturday 1:30 pm Four Seasons Ballroom 2-3
  In Roundtable Session: Multidisciplinary Measurement Using Rasch Modeling (Table 4)

Abstract

Rasch modeling can be very helpful building ordered multiple choice assessments. But what happens when some of those levels are similar and we collapse categories then using a Partial Credit Model (PCM). But we still lose valuable information about the collapsed category as they are treated and modeled as the same. This is where the Ordered Partition Mode (OPM) comes into play in helping us disentangle similar categories and model their presence. We used the Rapid Online Assessment of Reading (ROAR) Morphosyntax pilot measure as a case example. Using these approaches (PCM, OPM), we show nuances in the different categories leading to inferences about profiles of learners making types of errors and a way to visualize it.

Authors

• Alexander Mario Blum, Stanford University

• Robin Irey, University of California - San Francisco

• Rebecca Silverman, Stanford University

• Jason D Yeatman, Stanford University