Using Design-Based Implementation Research to Transform Science Education at Scale (Poster 8)

• In Event: Mapping the Landscape of Improvement Research: Representing Diverse Approaches Using a Common Grammar

Abstract

Purpose

The purpose of this poster is to use our study of improving science education across the United States to represent Design-Based Implementation Research (DBIR; Penuel et al., 2011) as a formal approach to improvement research in education.



Perspective

DBIR has roots in learning sciences and policy research (Fishman et al., 2013). It engages educators and leaders in partnerships for systems-level change toward educational equity (Penuel et al., 2022). Co-design is a key practice, focusing on the design of instructional guidance infrastructures like curriculum (Penuel, 2019). Mapping the improvement space is a collaborative process shaped by sociopolitical contexts (Hopkins et al., 2022; Penuel et al., 2013). Guided by theories of learning and implementation, interventions are tested using mixed methods approaches (Russell et al., 2013). Sustaining improvement focuses on leveraging and transforming existing infrastructures.

Modes of Inquiry

In this poster, we showcase our efforts to use DBIR as a Developers Consortium in collaboration with our partners in ten states to improve science education through enhanced instructional materials and linked professional development.


Data Sources

We drew on the following data sources: student exit tickets and surveys, teacher surveys, student assessments, and teacher interviews.


Findings

Building the foundational conditions: A nonprofit organization contracted with a Developers Consortium of four research organizations and a partner district to develop materials based on collaboratively developed, evidence-based design specifications. Structures included regular input from a steering committee of ten state education agency leaders as part of 4-month development and testing cycles.


Mapping the improvement space: Activities included a National Academies workshop, a market research study, and monthly state steering committee meetings.


Identifying a theory of (and ideas for) improvement: The materials were guided by an instructional approach called storylines (Reiser et al., 2021), emphasizing coherent learning sequences from the student’s perspective. Storylines are a form of problem-based learning with strong evidence (Hmelo-Silver, 2004). Change relies on sustained, high-quality professional learning (Gonzalez et al., 2022).


Iterating and measuring: Initial unit versions were field-tested in 300 classrooms. Data on student experience, teacher practices, and teacher professional learning were collected. These data, along with direct feedback and co-revision workshops with teachers, were used to improve units. All units were formatively and externally evaluated using a rubric developed by WestEd and EdReports, and effects on student learning were measured.


Spreading and sustaining improvement: Spread is supported by an independent nonprofit that provides free materials and contracts with publishers and kit providers, helping to fund future revisions. Over 30 certified professional development providers also support spread.


Significance

This study shows that a design and development study can be more than a small-scale effort to test a single innovation but can be conducted at scale with robust partnerships.

Authors

• William R. Penuel, University of Colorado - Boulder

• Carol Pazera, University of Texas at Austin

• Zoe Buck Bracey, BSCS Science Learning

• Daniel C. Edelson, BSCS Science Learning

• Kate Henson, University of Colorado - Boulder

• Brian J. Reiser, Northwestern University

• Tamara Sumner, University of Colorado - Boulder

• Douglas A. Watkins, Denver Public Schools