Use of Randomized Controlled Trials for Demonstrating Effect: Policy, Research, and Practitioner Perspectives

• In Event: Advances in Evaluation Methods

Abstract

This session will explore policy maker, researcher, and practitioner perspectives of randomized controlled trials (RCT) in the context of diverse settings and program capacities to carry out the demands of a rigorous research design. The session will discuss the strengths and challenges of a RCT and lessons learned from the perspectives of policy makers that commissioned the study, researchers who used a number of strategies to address participating sites’ evaluation readiness and capacity to collect data for the study, and practitioners who committed to data collection from participants while implementing enhancements to their programs. The evaluation of the Mentoring Enhancement Demonstration Program (MEDP) of the U.S. Department of Justice’s Office of Juvenile Justice and Delinquency Prevention (OJJDP) included 30 diverse agencies with respect to geographical location, program characteristics, and evaluation capacity. The research team worked collaboratively with the federal client and participating sites to understand and address evaluation readiness and capacity to minimize threats to internal validity and increase external validity. While the content of the research was on youth mentoring, the lessons learned in implementing the multi-site, multi-method research study has many implications for researchers working in diverse settings.

Authors

• Roger Jarjoura, American Institutes for Research

• Manolya Tanyu, American Institutes for Research