Evaluating the Validity of Less Standardized Assessments

• In Event: 30.077 - Applications in Validity Research and Theory

Abstract

Standardization has been a cornerstone of assessment interpretations involving comparable scores. Yet calls are increasing for less standardization in assessment, e.g., performance tasks. When less standardization is inherent in these assessments, how are their score interpretations to be validated? This paper presents an approach for analyzing less-standardized assessments in relation to Kane’s “validation argument” (2006, 2013). Key distinctions include where less standardization is a threat to validity and usefulness, and where less standardization is considered essential to validity and usefulness. Close attention is paid to the generalization of claims (Brennan, 2001; Haertel, 2006). Illustrative analysis results are provided for performance assessments in writing, interim assessments, and competency-based assessments. Discussion includes implications for both measurement experts and policymakers.

Authors

• Yi-Chen Chiang, Indiana University - Bloomington

• Brian Gong, Center for Assessment