Engaging Youth and Program Participants as Partners in Evaluation Research: Approaches and Insights from two International Scholarship Programs

Group Submission Type: Formal Panel Session

Proposal

This panel session brings together two growing areas of scholarship–evaluation of international scholarship programs, and engagement of program participants and alumni as partners in research and evaluation studies–to explore examples of innovative work at this intersection. For this panel, international scholarship programs are defined as those programs which fund students from multiple countries to pursue overseas higher education, often with an expectation that program graduates will use the education to benefit others in their home societies.

In the realm of international scholarship program evaluation, studies examine multiple components, including program efficacy, alumni journeys, and spillover effects to how host universities, foundations/funders, or governments have learned and changed as a result of these programs (Campbell, 2021; Martel & Talha-Jebril, 2021). Evaluators face key challenges in monitoring and evaluating scholarships, such as the tendency for evaluations to occur immediately after program completion and focus on program experiences and logistics rather than longer term impacts (Mawer, 2017). Yet new approaches are emerging to examine longitudinal influences of international exchanges, networks for employment and social change, and the role of technology in both creating and supporting partnerships and collecting longitudinal data (D’souza, 2024; Martel & Mason, 2024). In this panel, we also make the distinction between the evaluation of a large scholarship program and specific elements of scholarship initiatives, such as how alumni networks are built through technological platforms.

This panel explores one such emerging approach to evaluating different aspects and components of international scholarship programs: partnering with scholarship program participants and alumni as evaluators (Purdue et al., 2018). The presentations aim to synthesize and contribute to the literature on program participants and alumni as researchers and evaluators by bringing together two case studies of large, multi-country, multi-year evaluation projects. While these two evaluation case studies are distinct–with one focusing on longitudinal program impacts of a large international fellowship program and the other focusing on a particular element of an international scholarship program’s alumni engagement strategy–both outline how the evaluation was designed and or modified over time to include alumni of scholarship programs in the evaluation process and describe key insights and lessons learned through the process of collaborating with program alumni as partners in research and evaluation.

The goal of the panel is to both deepen understanding of methodological approaches to evaluating educational programs and to provide attendees with concrete examples of how two projects carried out this work. In alignment with the conference theme, the two case studies explore how these approaches intersect with the technological changes and how technology both facilitates and presents limitations in efforts to engage program alumni in evaluating international scholarship programs and collaborate virtually across multiple contexts.

For example, the case study of the Institute of International Education’s 10-year evaluation of the International Fellows Program highlights the necessity of a forward-thinking study design that balances strong methodology and milestones with space to accommodate new ideas and not yet known technological innovations. Similarly, the case study of the Neil Butcher and Associate’s 5-year evaluation of the Mastercard Foundation Baobab platform–a social learning platform designed to connect and support program participants and alumni–highlights how alumni leveraged technological tools to build research teams and communicate across multiple country contexts, as well as the key challenges they encountered collecting data and engaging in collaborative writing processes virtually. Both case study presentations also discuss the role of technology in sharing findings with various audiences and platforms.

The first paper provides a framework for the case studies by synthesizing existing literature on approaches to engaging youth and program participants/alumni in participatory research and evaluation processes. Building on this framework, the case studies contribute to a growing understanding of the potential approaches, opportunities, and challenges of participatory research and evaluation in the context of international scholarship programs. In doing so, they inform the efforts of others involved in longitudinal evaluations of scholarship programs and related platforms that leverage technology to sustain program networks and impacts beyond graduation. They will also inform the work of those employing similar methodological approaches in adjacent education spaces.

Following the three presentations, discussant remarks will frame an interactive discussion with the audience around experiences, benefits, and challenges of including participants in research and evaluation activities.

Sub Unit

• Monitoring and Evaluation SIG

Organizer

• Aryn R Baxter, Colorado State University

Chair

• Anne C Campbell, Middlebury Institute of International Studies at Monterey - Monterey, CA

Individual Presentations

• Partnering with program participants and alumni in international scholarship program research and evaluation: A review of literature and key synergies - Aryn R Baxter, Colorado State University; Anne C Campbell, Middlebury Institute of International Studies at Monterey - Monterey, CA

• Case study of IIE’s 10-Year evaluation of IFP - Leah Mason, Institute of International Education

• Youth participants evaluating and conducting research on the Baobab Platform - Angela Ochse, Neil Butcher & Associates; Olive Uwimana, Neil Butcher & Associates

Discussant

• Anne C Campbell, Middlebury Institute of International Studies at Monterey - Monterey, CA