Developing a Rubric to Identify Kindergarten and First Grade Students at Risk for Writing Difficulty

• In Event: Poster Session 10
  In Poster Session: PS 10 Topic - Education, Schooling

Abstract

This presentation describes the development of the Early Elementary Writing Rubric, an assessment designed to measure kindergarten and first grade writing and inform educators’ instruction. Participants included a variety of stakeholders working in schools: 55 classroom teachers (25 kindergarten and 30 first grade), 1 site-based administrator, and 5 resource teachers. There were a total of 634 students (285 kindergarteners and 349 first graders) who provided writing samples. To develop the Early Elementary Writing Rubric we used Crocker and Algina’s (1986) iterative, multi-step process and a series of studies. Focus groups, expert panel review, and cognitive interviews were conducted to establish content validity. Pilot testing occurred to allow for evaluation of the psychometric properties of the assessment. An exploratory factor analysis was conducted to evaluate the internal structure validity. Internal consistency estimates (as measured by Cronbach’s alpha) and inter-rater agreements were calculated to assess reliability. Participants also completed a social validity survey on feasibility and acceptability of the use of the rubric and scoring manual. There were five major findings. (1) First grade students received higher scores than kindergartners on all 10 scoring elements. The rubric distinguished performance and developmental differences of kindergarten vs. first grade children providing evidence of relations to other variables. (2) The results of exploratory factor analyses indicate that kindergarten and first grade narrative, informational, and opinion writing is characterized by a one-factor model. All scoring elements were retained and loadings ranged from 0.49-0.92. (3) Internal consistency estimates ranged from .89-.91. High internal consistency estimates for all genres provide evidence for the internal consistency of this latent construct. (4) Interrater reliability between the researcher and participants varied from poor to good revealing modest agreement in teachers’ ratings of student writing, a result that will require continued refinements to the measure and more extensive training of teachers. (5) The social validity feedback provided was favorable and useful for guiding revisions. Despite the large number of writing samples gathered for this study and the diversity of the participants, future research should use a stratified random sample. Additional studies need to be conducted to further analyze differences between grade levels and students using demographic and cultural subgroups and measurement invariance testing. Studies of how student writing correlates to reading performance and report card grades in English Language Arts also are needed. Further iterative development is needed to fully investigate its ability to accurately identify the present level of student performance and to determine how sensitive this measure is to developmental and instruction gains. Continued collaboration with stakeholders working in schools ensures that input from practitioners occurs and further revisions to the rubric contribute to ease of administration, scoring, and implementation. Accountability for assessment of writing in kindergarten and first grade may increase if teachers have access to a universal measure that can be used with relative ease to identify the present level of student performance. The continued iterative development of the Early Elementary Writing Rubric has potential to make a meaningful contribution to the writing assessment of young children.

Authors

• Meaghan McKenna, University of Kansas
  Presenting Author

• Howard Goldstein, University of South Florida
  Non-Presenting Author

• Robert F. Dedrick, University of South Florida
  Non-Presenting Author