Literacy-Based Assessment of, for, and as Learning

• In Event: From the Common Core State Standards to Rigorous Instructional Practice: Creating Knowledge Around the Development and Use of Formative Assessment Tools

Abstract

The Literacy Design Collaborative (LDC) is an assessment program intended to create assessment task templates (task specifications) and model curriculum units that document students’ literacy skills and abilities across content areas. Literacy assessment templates (“of learning”), across content areas help to model and identify how to integrate literacy instruction (“for learning”) within and across content areas. This focus is particularly relevant given the Common Core State Standards (CCSS) emphasis on writing for information and “evidence based” argumentation and inquiry (“as learning”).

LDC assessments are being built by teachers to satisfy a set of assessment design principles driven by the idea that reading for information and argumentation writing are core skills for college success regardless of an individuals' career choice. Specifically, teachers embed and align writing task templates into the curriculum and then “swap in" relevant subject-based content that is aligned to CCSS. To assess writing within and across the content areas, the following 4 rubric dimensions are used: not met, met expectations, exceeded expectations and advanced.

Methodology

The use of LDC tasks by districts and states is rooted in the assumption that the LDC template tasks are reliable and valid and could be used to inform both formative and/or summative policy decisions. To assess reliability for the classroom based scoring and distributive scoring we will identify a comparison group of classrooms within subject areas and conduct a socially moderated audit of their student scores using expert scorers to examine the consistency of classroom-based scoring under the two scoring conditions. To further examine reliability we will conduct a generalizability study in two content areas to identify possible sources of classification errors due to either the template task or raters with well trained and certified scorers.

Further, the LDC approach to template development in-which teachers can make the choice to "swap in" content raises the question of the equivalency of tasks (i.e. are they equally difficult). To assess comparability we will randomly identify classrooms in two content areas and administer a range of common template tasks to examine the comparability of tasks that use different informational text to assess writing. Comparability will be examined using latent trait analyses to equate tasks. Finally, to examine the impact of LDC tasks on student and teacher learning, we will use a mixed model evaluation design that includes both surveys of teachers and students as well as "think aloud" protocols to evaluate the depth of student knowledge and understandings of informational text.

We will report preliminary findings from a large scale pilot during SY 2011-12 in five states. This study will address the following research questions: (1) What level of reliability can be achieved using both a classroom based (teachers score their own class) and distributive scoring model (teachers score other class work)? (2) How comparable are the writing task templates within subject areas? (3) What is the impact of the LDC template tasks on student learning and on the instructional practices of subject area teachers?

Authors

• Raymond Pacheone, Stanford University

• Stuart R. Kahl, Measured Progress