The Use of Crowdsourcing in the Development of Measurement Instruments

• In Event: AERA Poster Session 14
  In Poster Session: Division D Section 1 Poster Session 1

Abstract

Crowdsourcing has gained favor among many social scientists as a method for collecting data because this method is both time- and resource-efficient. The present study uses a within subject test-retest design to evaluate the psychometric characteristics of crowdsource samples for developing and field testing measurement instruments. As evidenced by similar patterns of psychometric characteristics across time, strong test-retest reliability, and low failure rates of attention check items, the results of this study provide evidence that a sample from Amazon Mechanical Turk might represent a fruitful platform for field testing to support the development of a variety of measures. These findings, in turn, have significant implications for resource efficiency in the fields of educational and organizational measurement.

Authors

• Emily Michelle Wetherell, University of Iowa

• Stephen B. Dunbar, University of Iowa

• Catherine Welch, University of Iowa