Poster #59 - Establishing Evidence of Validity for the Youth Mentoring Survey

• In Event: Poster Session 06

Abstract

With the increasing popularity of youth mentoring programs, it becomes imperative to provide valid evaluations of these interventions. In two studies, we provide validity evidence for a worldwide used instrument, the mentee-reported Youth Mentoring Survey (YMS, Nakkula & Harris, 2011; 2018), across two versions. The original version of YMS (v1.3, 47 items) provides an in-depth comprehensive measure of mentoring quality and is still widely used, and the revised version (Rev. 2018, 51 items) further improves the scale’s validity. Both versions assess two types of mentoring match characteristics: internal match quality containing 3 subscales (relational quality, instrumental quality, and prescription) measures how mentees feel about the match; match structure measures three foci of interactions - fun, growth, and sharing. Theoretically, match structure should have an impact on internal match quality.

Study 1 established the structural validity and invariant evidence across two countries for the original YMS. Using responses from 276 mentees in the U.S. and Australia, we assessed four confirmatory factor analysis (CFA) models (see figure 1): a low-level six-factor model (coded blue), a middle-level two-factor model (coded green), a combined model in which six subfactors nested within two factors (coded blue and green), and a general factor model (coded orange). We found that the six-factor model had best fit, and the invariant analysis generated adequate results at the configural, weak, and metric levels (Table 1), which suggest the original YMS measures the intended latent constructs of mentoring match quality and can be used reliably across the U.S. and Australia.

Study 2 examined the structural and predictive validity of the revised YMS using data from 3 mentoring programs in the U.S.. The same four CFA models were tested and the six-factor model had the best fit (Table 1). Two OLS regression models examined the association between match structure and internal match quality. When predicting relational quality, fun (b = 2.02 , P < 001), sharing (b = 2.03, P < .001), growth (b = 1.55, P < .05), the two-way interaction between sharing and fun (b = -0.06, P < .01) and between fun and growth (b = -0.06, P < .05), and a three-way interaction between the three structure factors were significant. When predicting instrumental quality, sharing (b = 0.53, P < .001), growth (b = 0.53, P < .001), and the interaction between these two factors were significant.
In a subsample from study 2 (N =332), we explored the moderating effects of mentees’ characteristics on match quality. We found mentee’s grade level moderates the association between relational quality and sharing (b = 0.1207, p < .01), and fun is a stronger predictor of relational quality for boys than girls (b = 0.197,p < .01).

Our findings support that the YMS provides valid and comprehensive evaluations of youth mentoring programs, and that there is a clear association between the focus of mentoring interaction and mentoring relationship quality. The importance of sharing in building good mentoring relationships was highlighted, especially for older mentees.

Authors

• Michael Frisone, University of Pennsylvania
  Presenting Author

• Lingyan Hu, University of Pennsylvania

• My Hong Hoang Do, Harvard University

• Hou Xie, University of Pennsylvnia

• John Harris, Applied Research Consulting

• Michael Nakkula, University of Pennsylvania