Developing an Engineering Design Process Assessment Using Mixed Methods: An Illustration With Rasch Measurement Theory and Cognitive Interviews

• In Event: 70.063 - STEM Education in the Postsecondary and University Classroom

Abstract

This study demonstrates the use of mixed methods to guide assessment item development within the context of engineering education. Principles from Construct Modeling (Wilson, 2005) and Evidence-Centered Design (Mislevy, 2007) provide a framework for the use of evidence-based methods to support the validity argument for assessment items. Rasch measurement theory and cognitive interviews are illustrated as methodological techniques to gather additional evidence about student responses to multiple-choice items to inform item development and revisions. The quantitative and qualitative data are considered together using a convergent parallel design for mixed-methods analyses (Creswell & Plano-Clark, 2011). Preliminary results suggest that quantitative and qualitative evidence can be used to gain a more complete understanding of the psychometric quality of assessment items.

Authors

• Stefanie Anne Wind, The University of Alabama - Tuscaloosa

• Meltem Alemdar, Georgia Institute of Technology

• Jessica Gale, Georgia Institute of Technology

• Jeremy Lingle, Georgia Institute of Technology

• Roxanne Moore, Georgia Insitute of Technology