Estimating the Effect of Stakes on Standard Setting Impact Data

• In Event: 49.088 - Roundtable Session 17
  In Roundtable Session: 49.088-18 - Standard Setting: Issues and Applications

Abstract

Data from low-stakes administrations may be used to estimate future pass rates when performance standards are set before a test is administered operationally. If students are not motivated, underestimated pass rates could negatively impact recommended standards. In this study, matched samples of students taking high-school achievement tests under low-stakes and high-stakes conditions were used to estimate the effect of stakes on test performance and the “impact data” (i.e., pass rates) shown to standard setting panelists. In all cases, performance was greater under high-stakes conditions, with effect sizes that ranged from 0.39 to 0.46 across five tests. The maximum effect on impact data was 27%, and students with higher ability tended to be more affected by stakes.

Authors

• Jeffrey T. Steedle, Pearson

• Joe Grochowalski, Fordham University