Exploring Learning Engineering Design Decision Tracking: Emergent Themes From Practitioners' Work (Poster 4)

• In Event: From Research to Application: Learning Engineering as a Process for Learning-Sciences-Informed Educational Design

Abstract

Those involved in design for learning are situated in increasingly complex settings where they must consider contextual factors that influence the design and implementation of the learning experience. This paper examines the challenge of tracking design decisions and sharing them with teams and stakeholders across implementations and iterations. Using a design decision tracking tool as the structure (Authors, 2024), this study presents three cases that bring to light the practical implications of tracking design decisions and how other key aspects of design decisions connect with data to inform cycles of improvement.
Prior research has examined how those involved in learning design make decisions and develop an awareness of their decision making, including how instructional designers reflect on their design judgments (Lachheb & Boling, 2021) and how teachers make technology integration decisions (Xu & Stefaniak, 2024; Koehler & Mishra, 2005). Informed by these, we ask 1) how do practitioners track and take action on design decisions 2) how do these decisions take shape in future cycles of their design work and 3) what sources of influence (i.e., data) lead to changes in these decisions?
Using a qualitative case study methodology, this work investigates how design decisions can be grounded in lessons learned from educational research and connected with data to inform cycles of enacted iteration. For each example in the study, at least one practitioner directly involved in a learning design project was provided with a design decision tracking tool - a Learning Engineering Evidence and Decision (LEED) tracker (Authors, 2024) - which was filled in and maintained across at least one implementation of that project.
Utilizing these LEED trackers from across 19 individual projects, written project feedback forms, and interviews, emergent coding (Miles and Huberman, 1994) is used to examine how decisions were logged, tracked, and connected with other affordances and constraints comprising the design context. Written evidence of design decisions is collected from each tracker and analyzed for emergent themes. Within-case analysis allows for understanding how design decisions unfold within set contexts, and cross-case analysis provides emergent patterns across contextual factors.
Initial coding indicates that practitioners track changes at different grain sizes depending on the context of the work. When practitioners write design decisions in a LEED tracker, they are also prompted to record the justifications for those decisions. While emergent coding is ongoing, initial work suggests justifications fit into 4 major categories (personal experience, literature, data, technology). When these sources of influence are paired with interviews and post-project feedback, a process for why and when practitioners revisit prior decisions in conversations with stakeholders emerges in ways that improve the overall design.
This study offers insights into how design decisions and tracking play out over the course of individual projects and collective design of instruction work. The LEED tracker provides a structure for collecting and analyzing evidence of the design decision process while serving as a tool that practitioners can adapt to their specific context and use to make continuous, more intentional improvements.

Authors

• Lauren Totino, Massachusetts Institute of Technology

• Aaron Kessler, Massachusetts Institute of Technology