Investigating the Texas Value-Added Assessment System Component of the Texas Teacher Evaluation and Support System

• In Event: The Texas Teacher Evaluation and Support System: Features, Qualities, and Effects on Teachers and Schooling

Abstract

Purpose
After Every Student Succeeds Act (ESSA, 2016), states diminished the teacher accountability plans put into place post Race to the Top (2011). Texas is one such state; however, Texas still promotes its “recommended” Texas Teacher Evaluation and Support System (T-TESS), which heavily relies on a value-added model (VAM) called the Texas Value-Added Assessment System (TxVAAS). The TxVAAS is a derivative of SAS Institute Inc.’s popular Education Value-Added Assessment System (EVAAS).

The TxVAAS/EVAAS is explained to educators throughout Texas, via a web-based source (SAS Institute Inc., n.d.a.) of primary interest in this study. For this study, researchers conducted a content analysis of all asserted across the documents included within this site. Simultaneously, researchers situated that which was advanced as “truth” across documents within the current research to determine whether each claim was supported and to what extent.

Perspectives
Researchers framed their analyses using these key categories defined by the Standards for Educational and Psychological Testing (American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME), 2014), more specifically focusing on: (1) reliability, (2) validity, (3) bias, (4) fairness, and (5) transparency, all of which are defined in the Standards.

Methods
Researchers engaged in qualitative analyses of the textual data included within- and across documents (Miles & Huberman, 1994). Researchers organized all claims by measurement area, collapsed, and then positioned claims within the current literature, including 470 related sources systematically reviewed in a prior study Author(s) (2017).

Results
In terms of reliability, SAS noted that their TxVAAS/EVAAS models yield “highly reliable…correlation [coefficients at]…0.70–0.80” (SAS, n.d.c., p. 2). However, only one untitled, unpublished, and inaccessible report (White, Wright, & Sanders, 2011) was cited in support. Researchers privy to EVAAS data via a court order found reliability coefficients actually ranging around .41 ≤ r ≤ .52 (Author(s), 2017), which is on par with other VAMs.
In terms of validity, only citations in support of the validity of the TxVAAS/EVAAS were cited in support. Interesting here, though, was that a critique of the American Statistical Association’s (ASA) position statement on VAMs was cited (Chetty, Friedman, & Rockoff, 2014) in support of VAM use, but the original ASA (2014) piece was not cited concurrently.

In terms of bias, many unsubstantiated claims were made about how the TxVAAS/EVAAS being free from bias. Current research suggests that all VAMs are sometimes biased, however, and this includes the TxVAAS/EVAAS (Koedel, Mihaly, & Rockoff, 2015).
Please note that other findings, also about transparency and fairness, are included only in the larger paper.

Significance
When analyzing the claims used to promote the TxVAAS/EVAAS, researchers explained the arguably blatantly misguided “truths” presented to Texas teachers, and likely teachers beyond. This is significant should consumers read this piece and become better equipped to critically consume that which is marketed versus that which is “true” in these regards. This is perhaps more significant given this VAM is still being used across districts and states elsewhere (e.g., Tennessee’s TVAAS, Pennsylvania’s PVAAS, Ohio’s EVAAS).

Authors

• Audrey Amrein-Beardsley, Arizona State University

• Clarin Collins, Arizona State University