Operationalizations of the Effective Weights of Indicators in State Accountability Systems

• In Event: AERA Roundtable Session 15
  In Roundtable Session: Applications, Demonstrations, and Evaluations of Statistical Models and Methods

Abstract

School accountability scores are often weighted composite scores based on multiple indicators. The weights used to compute composite scores are often referred to as “nominal weights.” When indicator variances differ, the actual influence of each indicator on the composite score may differ from the nominal weights. Researchers have recommended calculating “effective weights” that attempt to quantify the actual influence of each indicator. We summarize common approaches to compute effective weights and propose the use of the Pratt index as another way to conceptualize and estimate effective weights. We illustrate the use of this approach in the context of a state school accountability system and discuss interpretations and implications of each approach.

Authors

• Kaitlin Mork, University of Colorado - Boulder

• Benjamin R. Shear, University of Colorado - Boulder