Individual Submission Summary
Share...

Direct link:

Shifting teacher pedagogy: Leveraging a diverse toolbox to assess ongoing program effectiveness

Thu, April 18, 10:00 to 11:30am, Hyatt Regency, Floor: Pacific Concourse (Level -1), Pacific E

Proposal

Educate! tackles youth unemployment by reforming what and how schools teach, so that students in Africa have the skills to start businesses, get jobs and drive growth. Our program in Uganda builds partnerships directly with schools to deliver a practical, skills-based curriculum designed to equip students with the skills they need to succeed.

To maximize the number of students we reach, Educate! has begun to integrate this model into national education systems. That means delivering our skills-based curriculum through teachers. Now, successfully engaging teachers to improve their ability to use active teaching methods is core to our success.

While we aim to create the same student experience, our M&E systems must shift to adapt to this new delivery. We’ve updated our M&E system to provide reliable information about pedagogical knowledge and practices that feed directly into program decisions. For Educate!, a well-functioning learning loop requires short, medium, and long-term evidence
of effectiveness, including:

Qualitative exploration of processes and outcomes: One question on Educate!’s learning agenda is to understand what role Educate!’s work with teachers in Uganda plays in creating positive changes in the classroom. To answer this, we partnered with IDInsight to conduct a qualitative evaluation to answer this question. We found that students, teachers, and administrators perceived that teachers gained valuable new skills and attitudes through their involvement with Educate!. The link between perceived knowledge and attitude changes and behavior changes in the classroom was less clear.

The timing, structure, and focus of the evaluation allowed us to deeply explore which changes were playing out, which were not, and why. The evaluation generated recommendations to consider and pilot as we continuously iterate and improve our program.

More continuous feedback is useful for other programs, such as our teacher training and support program. In Rwanda, Educate! supports entrepreneurship teachers in adopting active learning pedagogies. Educate! created a teacher impact framework that provides a rubric for how teachers’ perceptions and actions should develop from awareness to full adoption. We’ve used two methods to track teachers’ progress along this framework:

Rigorous quantitative evaluations which measure midline, intermediate outcomes: We are partnering with IPA Rwanda and affiliated researchers on a randomized evaluation. Given that the full program would take two years to deliver, results would not arrive until almost three years after we began the program. Our research partners planned two rounds of midline “monitoring” to assess progress on pedagogical outcomes. Results from these monitoring rounds provided information about how teachers perceptions and behaviors were shifting, and where they required more support.

Real-time monitoring data from program staff: Educate! had a hypothesis on where, within our Teacher Impact Framework, most teachers were, but wanted hard, updated data to back that up. Using the Framework, we created a rubric to assess teachers’ level of adoption that could be completed by program staff via a brief conversation with teachers. Like our monitoring data from the RCT, this provided information what pedagogical practices and beliefs teachers had adopted and what areas they still lacked support

Author