Investigating Program Impact Using Hierarchical Linear Modeling: A Comparison of Intention-to-Treat and Treatment-of-the-Treated Analyses

• In Event: Investigating Program Impacts Using Multilevel Models

Abstract

Based on a randomized controlled trial experimental design, this study investigated the intervention effects of the Scholastic READ 180 program on the reading proficiency of low-achieving incarcerated youth in a large mid-western state. Impact analyses were conducted based on the Intention-to-Treat (ITT) group and the Treatment-of-the-Treated (TOT) subset. Both the ITT and TOT analyses consistently indicated that the experimental group significantly outperformed the comparison group on reading when controlling the influences of salient covariates, with the TOT impact estimate being somewhat larger than the ITT estimate. General issues in conducting the ITT and TOT analyses were discussed. The Bloom adjustment was also applied to account for treatment no-shows, and the result was compared with that of the TOT subset analysis.

Authors

• Jing Zhu, Metis Associates, Inc.

• William Loadman, The Ohio State University

• Raeal Moore, ACT, Inc