Linking With Constructed Responses: A Hierarchical Model Approach With Advanced Placement Data

• In Event: Roundtable Session 28
  In Roundtable Session: How Reliable Are Raters?

Abstract

Linking with constructed response (CR) items is more complex than linking with multiple choice items, because CR items are scored by raters, which introduces another source of variation into the scores. When an item response theory model is used as a rater model, one must include a condition to link raters, as has been previously discussed (e.g., Tate, 1999). Here it is shown that, when a hierarchical-rater signal-detection model is used, the CR item parameters are not distorted due to changes in the rater parameters (e.g., severity), and thus a rater-linking condition is not needed. The approach is illustrated with an application to data from the Advanced Placement® (AP®) Studio Art portfolios 3-D Design.

Authors

• YoungKoung Kim, The College Board

• Lawrence T. DeCarlo, Teachers College, Columbia University

• Rosemary A. Reshetar, The College Board