Critical appraisal in qualitative evidence synthesis: A worked example of a meta-aggregation

• In Event: Evidence-Based Crime Policy I

Abstract

The critical appraisal of qualitative evidence in meta-syntheses is a widely contested issue, yet critical appraisal provides an opportunity to assess the quality of knowledge, meaning, and understanding generated from qualitative research in qualitative evidence synthesis (QES). A diversity of approaches for appraising qualitative evidence exist. However, to date a tool that captures qualitative evidence generated from qualitative studies and quantitative studies has yet to emerge in the area of critical appraisal for QES. Furthermore, nor does a process exist to facilitate rating the overall quality of evidence for aggregated findings from different studies. We report on a methodological process for critically appraising qualitative findings generated from qualitative and quantitative studies, culminating in the development of a rubric that can help standardize assessments of the overall quality of evidence in QES. The method presented is applied at the findings-level and converts assessments of findings into a numerical score, which is then aggregated into an overall score for synthesized findings using a process similar to calculating an h-index. This approach provides a balanced and integrated method by first assessing individual findings at one stage using flexible criteria and then standardizing the overall assessment of evidence once aggregated.

Authors

• Ajima Olaghere, Temple University

• Catherine S. Kimbrell, George Mason University

• David B. Wilson, George Mason University