The Effect of iMentor Program Dosage on College Aspirations and Positive Relationships With Adults

• In Event: Mentoring in Schools: The Predictors and Products of High-Quality Relationships

Abstract

Objectives
An evaluation of iMentor found that, on average, the iMentor program had significant positive effects on 9th grade students’ college aspirations and relationships with adults. It also found that some students participated in the iMentor program at different rates (Sabo, 2008). The objective of this paper is to analyze if 1) students with high levels of participation differed from those with low levels of participation; 2) the effect of iMentor on college aspirations and relationships with adults depends on different student program participation; and 3) in-person meetings are more important than other forms of communication.
Theoretical Framework
The implementation literature describes that students who receive more dosage of a program may have differential impacts (Reyes et al., 2012). In the case of iMentor, students vary in their overall participation but they also vary in in-person meetings, emailing, texting, and class attendance. The mentoring literature shows that in person meetings are important for an effective relationship (Grossman & Tierney, 1998; Grossman & Furano, 1999), but there is little research about the effect of other modes of communication such as emailing and texting.

Data
We will link three separate, student-level data sets to test these research objectives for 1,100 students in eight schools. These data include administrative from the NYC Department of education, programmatic data from iMentor, and student survey results.
Methods
To find the average effect of iMentor, we employ a lagged-cohort design with school fixed effects to control for all between school differences (Merrill et al., 2015). We use other student-level covariates to control for differences between the treament and comparison cohorts as well as to improve the precision of the estimate.
To uncover the effect of dosage, we must control for a selection effect: students who are have high levels of dosage may have different characteristics than students who have lower levels of implementation. In order to control for selection, we use a two-stage instrumental variable analysis (Pindyck et al., 1998). We then estimate the probability that a student would have high dosage using a logistic regression equation controlling for student background characteristics and school fixed effects (Equation 1). We then run a model testing the differences between high, low and no dosage including each students’ likelihood of participating with high fidelity given their background characteristics (Equation 2).
Results, Conclusions & Discussions
Table 1 shows that students who met all the benchmarks have a similar background, but were much less likely to be chronically absent in the 8th grade. We anticipate that students who are prone to being absent may be different than those who come to school more regularly, thus validating our use of a two-stage instrumental variable analysis.

Scholarly Significance
This study will contribute to the mentoring field to help determine which types activities—in-person, electronic, or class leads to successful programming. It also provides another example of the importance of testing dosage when conducting program evaluations.

Authors

• Lisa Merrill, Research Alliance for New York City Schools

• James Lauckhardt, iMentor