Examining the Reliability of the National Writing Project’s Analytic Writing Continuum Assessment System

• In Event: Validity Investigations

Abstract

This paper examines the technical rigor of the National Writing Project’s (NWP) Analytic Writing Continuum (AWC), a writing assessment system that provides high quality, reliable data for research and supports teachers’ classroom practice. Data from 498 equating papers that have been scored annually over four years (2008-2011) are analyzed. Results show that the AWC has remarkably high inter-rater reliability (overall 90% agreement across attributes), test-retest reliability, and internal consistency (α=.97). The empirical evidence ascertains that the AWC is a robust instrument that directly measures the growth of writing performance, both holistically and analytically, in an objective, unbiased manner. Implications of these results include using the AWC as a classroom tool, and as a system that supports teacher professional development.

Authors

• Hee Jin Bang, National Writing Project

• Sherry Seale Swain, National Writing Project

• Paul LeMahieu, The Carnegie Foundation for the Advancement of Teaching

• Linda D. Friedrich, National Writing Project