A Simple Generalizability-Theory Approach to Deriving More Accurate Estimates of Reliability for Common Score Use

• In Event: 60.031 - Reliability: New Approaches

Abstract

Test-retest and alpha reliability estimates routinely overestimate the reliability of assessment measures because they do not take all relevant sources of measurement error into account (i.e., random, specific-factor, and transient error). This in turn can lead to misrepresentations of relationships among educational and psychological constructs. In this proposal, a procedure based on Generalizability Theory is described and applied to estimating reliability of scores to individual difference measures. The procedure quantifies random, specific-factor, and transient error separately and integrates them into formulas that provide more accurate estimates of reliability for such measures. The procedure only requires that the same measure be administered on two occasions with a short time interval between administrations.

Authors

• Walter Peter Vispoel, University of Iowa

• Han Yi Kim, Measured Progress

• Murat Kilinc, University of Iowa

• Carrie A. Morris, The University of Iowa