Creating a Writing Assessment System for Research and Practice: The National Writing Project’s Analytic Writing Continuum

• In Event: The National Writing Project: Networked Development and Use of Knowledge to Support Teachers’ Professional Learning

Abstract

The Common Core State Standards heighten attention on the critical role of writing in students’ preparation for college and career. This increased focus raises questions about how teachers develop and strengthen their practice in teaching writing and what role assessment may play in informing instruction. Over the past eight years, the National Writing Project has engaged in developing and using a writing assessment system that provides high quality, reliable data for research and supports teachers’ classroom practice. NWP’s Analytic Writing Continuum (AWC) is designed to directly measure the growth of writing performance in an objective, unbiased manner. It is the result of several modifications to specific traits in the Six+1 Trait Writing Model (Culham, 2003), made to yield an instrument suitable for evaluating NWP’s professional development programs and providing data to teachers to inform writing instruction.

The AWC applies both holistic and analytic scoring procedures. The holistic score provides a summary judgment about the overall quality of writing. The analytic scores focus on the individual disaggregated components of writing: Content, Structure, Stance, Sentence Fluency, Diction, and Conventions. The scale for the holistic score and each attribute ranges from 1 to 6 (Authors, 2011). Analyses examining the technical properties of the AWC system indicate that it is appropriate for evaluating various genres of writing across grade levels. It has also demonstrated remarkably high inter-rater reliability (overall 90% agreement across attributes), test-retest reliability, and internal consistency (Cronbach’s α = .97).

This paper focuses on how teachers and local Writing Project Sites have come to use the AWC. Data are drawn from three sources. First, surveys focused on perceptions of how participating in the scoring affected teachers’ classroom and leadership practices were collected from 136 of the 325 (41.6%) teachers who participated in an NWP scoring event prior to February 2008. Second, 14 teachers were interviewed to explore how they were using the AWC. Finally, we draw case materials from learning projects conducted by experienced teacher consultants. Examples include investigating the use of the AWC in a university writing program; documenting how the AWC can be used in classroom to engage students to talk about their writing; and using the tool in delivering professional development focused on improvement of student writing. Overall, the survey findings suggest that the impact of participating in scoring on teachers’ classroom practice was greatest for assessment (64.0%), teaching (55.4%) and response (57.3%). Teacher interviews and writings bring life to the ways in which being immersed in this research-based assessment system helped them more fully engage their own students in writing and discussing the quality of their writing. The lessons learned from a wide array of projects around the AWC emphasize the importance of bringing together the knowledge and skills of researchers and teachers in interactive spaces (Stein & Coburn, 2010). This paper illustrates how a community with a shared set of values and common language can develop a rigorous assessment system that also guides teaching and learning and fosters professional development.

Authors

• Sherry Seale Swain, National Writing Project

• Paul LeMahieu, The Carnegie Foundation for the Advancement of Teaching

• Hee Jin Bang, National Writing Project

• Linda D. Friedrich, National Writing Project