Building Evaluation Capacity: An Evaluability Assessment of a School-Based Prevention Program

• In Event: Advances in Evaluative Research

Abstract

Research and evaluation have made substantial progress in the identification and development of evidence-based practices in criminal justice. However, concerns remain regarding the varying definitions of evidence, variety of methodological rigor, and a lack of insight into why programs may be (in)effective. Evaluability assessments are a pre-evaluation activity that can address some of these issues and serve as a mechanism to build evaluation capacity among practitioners. Similar to implementation science, evaluability assessments are a critical yet underutilized component of understanding “what works” in criminal justice policy and practice. Evaluability assessments determine whether a program or policy is implemented as designed and if it is sufficiently prepared for an outcome evaluation. This case study assesses the evaluability of the All Stars program, a school-based prevention program. The program seeks to prevent substance use and reduce bullying, violence, and other conduct problems among middle school students. Findings address the extent to which the program theory aligns with the implementation of the program, program enrollment, program strengths, and adjustments that can be made by key stakeholders to build their capacity for an outcome evaluation. Study results add to the limited, yet growing body of research on evaluability assessment and creates the foundation for an informed outcome evaluation while intentionally incorporating practitioners into the research process.

Authors

• Julie Krupa, Michigan State University

• Kismet Okyere, Michigan State University

• Jordan Parker, Michigan State University