Automated Writing Trait Analysis to Support Instruction

Session Type: Symposium

Abstract

This session will explore the use of an automated writing trait model to support instruction.

First, we review the use of natural language processing to build automated trait models and examine the potential for such models to provide richer formative information for teachers.

Second, we present an automated writing trait model trained on 1.37 million submissions to ETS' digital writing service, Criterion® and will present evidence of its validity.

Third, we apply the trait model to an intervention study in which students were taught to write argument essays and examine relations between essay score increases and trait scores increases.

Finally, we present results from a qualitative teacher study that explored potential for feedback from trait profiles to inform instructional decisions.

Sub Unit

• Division C - Learning and Instruction / Division C - Section 1a: Literacy

Chair

• Tenaha P. O'Reilly, Educational Testing Service

Papers

• Toward an Integrated Writing Trait Model - Paul D. Deane, Educational Testing Service

• A 12-Trait Model of Variation Among Student Essays - Duanli Yan, Educational Testing Service; Paul D. Deane, Educational Testing Service

• Using a Trait Model to Characterize Growth Effects in a Writing Intervention - Paul D. Deane, Educational Testing Service; Yi Song, Educational Testing Service; Ralph P. Ferretti, University of Delaware

• Validating Writing Traits for Classroom Assessment Purposes - Priya Kannan, WestEd; Geoffrey C. Phelps, Educational Testing Service; Paul D. Deane, Educational Testing Service

Discussant

• Danielle S. McNamara, Arizona State University