Transitioning NAEP Writing to a Digitally Based Assessment

• In Event: 58.074 - Discussion and Demonstration of New Digitally Based Assessments for NAEP

Abstract

In today’s society, writing with paper and pencil is being replaced by writing on computer. In 2015, NAEP data showed that a vast majority of fourth-graders (90%) have computers available in schools, for use by students and their teachers for reading and/or language arts activities. Increasingly, students are expected to compose on computer as they move through high school, go on to college, and enter the workforce.

While the adoption of technology varies from household to household and classroom to classroom, today’s student population was born into a digital world. How can NAEP begin to efficiently and effectively measure the knowledge and skills of these digital natives? The 2011 NAEP Writing Framework calls specifically for the assessment of writing on computer. In 2011, NAEP assessed eighth- and twelfth-graders in a Writing Computer-based Assessment (WCBA). This was followed by a pilot study for fourth-graders in 2012.

Students participating in the 2011 and 2012 WCBA responded to tasks designed to measure one of three communicative purposes common to many typical writing situations. Each student who participated in the WCBA responded to two tasks. The assessment included some writing tasks incorporating multimedia stimuli, such as audio video. Before being presented with the first task, students were shown a tutorial. Students completed their writing tasks on laptop computers provided by NAEP, using software similar to common word-processing programs. The interface included standard tools for editing, formatting, and viewing text, but did not have access to potentially irrelevant or distracting tools such as clip art, font type and color, or the Internet. Additionally, selected student actions, such as frequency of use of spellcheck, was collected, and explored for correlations with performance on tasks.
In 2017, the NAEP writing assessment for grades 8 and 12 will transition from laptops to tablets that will have many of the features and functions from the 2011 and 2012 WCBA. In an effort to adjust to ongoing shifts in digital technologies, NAEP will conduct a comparability study in Spring 2017. This study includes administering writing tasks via laptop in order to help NAEP evaluate any differences in student performance between tablet and laptop administration. Additionally, the collection of student actions taken during the assessment has been updated in preparation for the 2017 administration to capture click streams and thus allow a window into students’ composing in response to tasks, which will offer a rich source of data for potential exploratory reporting.

This portion of the symposium will discuss the approaches taken to be sure the NAEP writing assessment appropriately reflects students’ writing and experiences with technology in and out of school, and takes advantage of the affordances of technology.

Author

• Hilary Persky, ETS