Assessing Study Quality Using the Methodological Quality Questionnaire: A Case Study of Novice Rater Training

• In Event: 41.083 - Roundtable Session 15
  In Roundtable Session: 41.083-24 - Addressing Practical Concerns to Conducting a Meta-Analysis

Abstract

The purpose of our case study was to: (a) investigate error and bias when critically appraising study quality with a group of second year doctoral students (n=14) with no systematic review experience and (b) determine their readiness for assessing study quality independently using the Methodological Quality Questionnaire rating scale after participating in a training where they critically appraised original studies drawn from a systematic review. One-half (n=7) of the novice raters had a percentage of agreement with the expert rater of 78% or higher. These scores support training effectiveness, specifically building capacity by increasing raters’ skill and knowledge about the critical appraisal process. From preliminary findings, we developed a guide for improving and evaluating training on methodological quality assessment.

Authors

• Sandra T. Acosta, Texas A&M University

• Tiberio Garza, University of Nevada - Las Vegas

• Hsien-Yuan Hsu, University of Texas Health Science Center at Houston