A Critical Review of Validity Evidence in Exercise Psychology: An Examination of the Behavioral Regulations in Exercise Questionnaire

• In Event: When Validity Theory Meets Validation Practice: Research Syntheses of Validity Evidence Reported in Seven Areas

Abstract

Introduction/Purpose
The purpose of this paper was to examine what sources of validity evidence have been published in exercise psychology literature. A systematic review was conducted examining all published studies that have used a well-known instrument called the Behavioural Regulations in Exercise Questionnaire (BREQ: Mullan, Markland & Ingledew, 1997).

Methods
Of the 94 peer-reviewed, published studies that were identified as having used the BREQ, 24 studies presented evidence of score validity. Two independent coders used a standardized coding form to code the 24 studies that presented evidence of score validity. The validity information coded was based on Cizek, Rosenberg and Koons (2008) and included: validity perspectives used (e.g., The Standards (AERA et al., 1999); Messick, (1995)), validity conceptions (e.g., characteristic of test score), sources and kinds of validity evidence (e.g., relations to other variables, internal structure) and overall judgment of the instrument.

Results
Results indicated that only 29% of the studies coded cited a prominent validity framework such as Messick’s (1995) or the Standards (AERA, APA, & NCME, 1999). The most common source of validity evidence presented was derived through evidence of structural validity (e.g., confirmatory factor analysis and item interrelationships) and relations to other variables (e.g., discriminant and convergent validity).

Conclusions
Overall, most authors had a favourable opinion of the score validity obtained from the BREQ. Results also indicated that researchers who have used the BREQ have not presented any validity evidence based on content, response processes or consequences. This paper outlines the crucial role of validity theory for exercise and sport psychology research. Finally, future directions for advancing the field of exercise and sport psychology through validity theory and validation practices are provided.

Authors

• Katie E. Gunnell, The University of British Columbia

• Bruno D. Zumbo, The University of British Columbia

• Philip M. Wilson, Brock University

• Diane E. Mack, Brock University

• Peter R.E. Crocker, The University of British Columbia

• Benjamin Schellenberg, The University of British Columbia