Using Verbal Protocol Methodology and Sketch Analysis to Explore Engineering Design Ideation

• In Event: Using Novel Research Methods to Understand Teaching and Learning in the Professions

Abstract

Research has demonstrated that the “think-aloud approach,” a verbal protocol research method in which research participants are asked to verbalize their thoughts as they are engaging in an activity (Ericsson & Simon, 1980, 1993; Hannu & Pallah, 2000; Gero & McNeill, 1998), is useful for identifying complex thought patterns and mental strategies of practitioners and students in professional fields. This is especially true in situations where participants cannot recall each decision in a lengthy process or where they do not consciously know their strategies (e.g., Akin & Lin, 1995; Atman, Adams, Mosborg, Cardella, Turns, & Saleem, 2007; Atman & Bursic, 1998; Atman, Chimka, Bursic, & Nachtmann, 1999; Eckersley, 1988). Design, which is central to many professions, including the engineering disciplines (e.g., ABET Board of Directors, 2011, Committee on the Engineer of 2020, 2004; Daly, Adams, & Bodner, 2012; Duderstadt, 2008; Sheppard, Macatanga, Colby & Sullivan, 2009), is one example of a complex intellectual activity for which it can be challenging to uncover approaches, processes, and strategies. This presentation will focus on describing the think-aloud approach, and how we used it in combination with sketch analysis, to study idea generation strategies in design.

Idea generation in design is most successful, and most likely to lead to innovation, when multiple and diverse concepts are generated (Akin, 1990; Atman, Chimka, Bursic, & Nachtmann, 1999; Brophy, 2001; Cross, 2001; Liu & Bligh, 2003); however, strategies to achieve quantity and diversity goals are often anecdotal. The goal of our work was to empirically uncover natural strategies employed by engineering design practitioners and students as they sought to successfully explore design solution spaces, with the long-term goal of translating these approaches into strategies that others can use (Daly, Yilmaz, Christian, Seifert, & Gonzalez, 2012). Our research team recognized a need to capture decisions and strategies while designers were generating ideas, as attempts to capture ideation strategies retrospectively were not successful (Daly, Yilmaz, Seifert, & Gonzalez, 2010). Thus our focus was on capturing strategies as they occurred. To do this, we developed an approach based on the think-aloud methodology, which allowed us to uncover a sequence of information without altering cognitive processes; Atman and Bursic (1998) demonstrated that participants who verbalized concurrently with a task could provide information that did not change the nature of their thinking. We combined the think-aloud method with a technique to capture the sequence of participants’ sketches as they were thinking aloud (using a LiveScribe pen).

The goal of this presentation is to describe and emphasize the value of a think-aloud approach in order to support broader adoption of these methods when they are appropriate for the research goals, specifically in investigations of thought processes in situations where students and practitioners may not know or be able to verbalize their approaches.

Author

• Shanna Daly, University of Michigan