How Much Do Low-Stakes Assessments Underestimate Ability? A Case Study Using CLA+

• In Event: 47.042 - Emergent Methodological Considerations for Higher Education Outcomes

Abstract

Low motivation presents a risk to the validity and reliability of an assessment. If a low-stakes assessment is used to make inferences about students’ abilities, their abilities might be underestimated. This paper describes a recent propensity-matching experiment conducted on a unique population of students participating in voluntary domestic military service. The assessment used in this study went from low stakes to high stakes when the administrators of this program decided to use test scores as part of participants’ subsequent placement in the military. Ability underestimation in the low-stakes condition is estimated, as well as the mediating role of motivation in the relationship between stakes and outcome. A sensitivity analysis illustrates the probable lack of hidden bias in the mediation model.

Authors

• Jon Lehrfeld, Council for Aid to Education

• Doris Zahner, Council for Aid to Education