A Systems Approach to Developing a Learning Progression and Associated Instructional Units and Assessments

• In Event: Systems Thinking Approaches to Learning and Assessment Design

Abstract

Objectives
1. Describe the process used to design a learning progression and to develop instructional units and assessments aligned to it.
2. Report on results of field tests and early results from an efficacy study of the package.

Perspectives
Presenter 3 will describe the Assessing Data Modeling and Statistical Reasoning project (Lehrer, Kim, Ayers, & Wilson, 2014), which developed a learning progression to support the development of middle school students’ reasoning about data and statistics by engaging them in practices of data modeling. Data modeling refers to the invention and revision of models of chance to describe the variability inherent in particular processes.

Methods
The construction of the progression involved taking a systems perspective to analyze the core concepts and practices of data modeling; conjectures about fruitful instructional means for supporting student learning; and the design and development of an assessment system that could be employed for both summative and formative purposes.
The design of the assessment system, consisting of both formative and summative assessments, was guided by the four building blocks for assessment suggested by Wilson (2005). The first of these is a set of construct maps that specify the dimensions of learning (7 in this work) to be assessed. Each construct map specifies ordered relations among milestones of conceptual development. Second, these construct maps guide the design of items. Items are generators of specific learning performances that is, statements of what students located at a particular point in a progress map are expected to say or do, and which are coded into an outcome space, the third building block, which matches the student performances back to the construct map (via scoring exemplars). Finally, we use a Wright Map, based on an item response model, to relate the student responses back to the construct map, allowing the interpretation of student responses as locations on that map, and hence, use of them for instructional planning, student diagnosis, and student evaluation.

Results
We developed a multidimensional learning progression designed to capture the complexity of data modeling and to represent prospective relations among these dimensions (Black, Wilson & Yao, 2011; Wilson, 2009). This guided the development of instructional units, and an item bank co-ordinated with the learning progression. Moreover, because we are interested in influencing classroom instruction, we are also engaged in brokering professional development, viewing components of the assessment system as boundary objects (Star & Griesemer 1989; Bowker & Star, 1999) and our role as brokers (Wenger, 1998) of these objects with teachers. This effort complements the psychometric effort and involved substantial revision in components of the system, with reverberations for the psychometric effort as well.

Significance
Potential fruits of this synthesis include greater impact on teaching and learning, because cognitively based assessment offers prospects for tailoring teaching to learners’ states-of-knowledge, rather than to points of progress in a curricular sequence. The NRC (2001) suggests that achieving the desired synthesis will require “increased and sustained multidisciplinary collaboration around theoretical and practical matters of assessment” (p. 11).

Author

• Mark R. Wilson, University of California - Berkeley