Measuring Motivation in Educational Settings: A Case for Pragmatic Measurement

• In Event: Motivation and Learning: What Is Provocative and/or Novel?

Abstract

Critical to any motivation study is the quality of its measurement. Unfortunately, commonly accepted practices for ensuring measurement quality can at times be at odds with other situational constraints of a research study, especially for applied research conducted in real-world settings. For example, tensions between the technical and practical aspects of a measure can lead to measurement solutions that are (a) scientifically valid but impractical/implausible to use in practice or (b) practical to use but lack scientific validity.

Therefore, the purpose of our presentation is to describe the use of a more flexible method of assessing measure quality in applied settings. We refer to this set of methods as Pragmatic Measurement (see Kosovich, 2017). This approach seeks to minimize the resources required for data collection (e.g., class time and participant time) while maximizing the quality of the data being collected (i.e., validity). For example in one study, school administrators indicated that the researchers could spend no more than three minutes of class time per week conducting measurement (Yeager et al., 2013). Such constraints would be prohibitively restrictive if researchers were unwilling to compromise on either the number of constructs measured or the number of items dedicated to any given construct. Although measurement standards (AERA et al., 2014) acknowledge situational constraints, the role of these factors is frequently under appreciated in common practice.

During our presentation, we first will introduce pragmatic measurement and how it can complement traditional measurement practice. Second, we will describe how we are currently using pragmatic measures to inform work in three common areas of motivation research (cross-sectional group differences, change over time, and interventions). Third, we will discuss the implications of pragmatic measurement for both research and practice and offer our recommended next steps in this area.

Authors

• Chris S. Hulleman, University of Virginia

• Jeff John Kosovich, University of Virginia

• Kenn E. Barron, James Madison University