A Priori Analysis of Error Rates in Value-Added Models

• In Event: 25.068 - Roundtable Session 8
  In Roundtable Session: 25.068-10 - Toward Equal Opportunity: Seeking Best Practices for Incorporating Value-Added Measures in the Teacher Evaluation Process

Abstract

A Monte Carlo simulation study design is used to investigate whether value-added models (VAMs) estimate teacher effectiveness accurately when all assumptions are met and determine the robustness of VAMs to endogenous peer effects and nonrandom assignment. Three years of simulated achievement data are generated and analyzed with the SAS® EVAAS® Multivariate Response Model (M1) and a basic covariate adjustment model (M2). When all assumptions are met, M1 is more accurate than M2, but both models perform reasonably well, misclassifying fewer than 5% of teachers on average. M1 is more robust to peer effects than M2, however M2 is more robust to nonrandom assignment of students than M1. Implications for practice and research are discussed.

Authors

• Matthew Ryan Lavery, Bowling Green State University

• Debbie L. Hahs-Vaughn, University of Central Florida

• Stephen A. Sivo, University of Central Florida

• Haiyan Bai, University of Central Florida

• Audrey Amrein-Beardsley, Arizona State University