Extending and Replicating the Results of a Mentor Recruitment Experiment

• In Event: Extending and Interrogating the Quantitative Evidence That Clinical Mentors Matter

Abstract

Emerging experimental evidence has suggested that completing a clinical placement with a more instructionally effective and experienced mentor positively impacts preservice student teachers’ (PSTs’) readiness to teach (Ronfeldt et al., 2018; Ronfeldt et al., 2020). These results, however, come from only two experiments with narrow sets of outcomes, raising questions about their replicability in different contexts and across measures. Both were conducted with an ideal partner program with a history of state and researcher collaboration, raising concerns about replicability with more typical programs. Additionally, while impacts on perceptions are meaningful, it is unclear whether PSTs who feel more prepared are more instructionally effective (Ronfeldt et al., 2021).
In this paper, we address these concerns by replicating and extending one of those experiments—the Mentors Matter Recruitment initiative—in two ways. In the original initiative, the authors leveraged administrative data to generate lists of the most instructionally effective and experienced teachers who could serve as clinical mentors, which they then provided to recruiters in randomly assigned districts (Ronfeldt et al., 2020). Districts receiving these recommendation lists recruited more instructionally effective and experienced mentors, whose PSTs reported feeling more prepared to teach at the end of their clinical placements.
In one study, we reinvestigate the impact of receiving recommendation lists on mentor quality and PST self-reported preparedness for a new cohort within the same partner program while also including new outcomes of clinical assessment scores during student teaching, employment rates, and observation ratings among employed graduates. In another, we provide three new programs with recommendation lists and compare the mentors they recruit to those who served in prior years. The first study is a direct experimental replication with the goal of assessing estimate stability over similar re-implementation conditions and to new outcome measures; the second is a quasi-experimental scale-up extension with a set of new partners and less oversight, providing estimates of real-world implementation effects beyond the ideal case.
In the replication study, we find virtually identical experimental effects—districts receiving recommendation lists recruited substantially more instructionally effective and experienced mentors whose PSTs felt significantly better prepared to teach. Additionally, we find that PSTs in districts receiving lists earned marginally higher clinical assessment scores and—although not significantly—were more likely to gain employment and received higher observation ratings.
In the extension study, we find that recommendation lists continued to increase the average instructional quality and experience of mentors among a new set of programs under less oversight, suggesting the promise of the initiative across various program contexts. However, the magnitude of these improvements depended on the effectiveness of programs’ business-as-usual recruitment strategies in prior years.
Together, these two replication studies suggest that providing recommendation lists to teacher education programs is a viable, stable, and scalable way to improve the average instructional quality of mentors as well as PSTs’ field placement and early career experiences. Moreover, both studies serve to ensure that the results of the initial experiment are valid, meaningful, and generalizable before encouraging their use informing policy implementation.

Authors

• Matthew Truwit, University of Michigan
  Presenting Author

• Matthew Ronfeldt, University of Michigan
  Presenting Author

• Emanuele Bardelli, Santa Rosa City Schools
  Non-Presenting Author