Randomized Controlled Trials of Multimedia Education: A Multimedia Science Education Impact Evaluation in Afghanistan

• In Event: Learning Through Broadcast: Enhancing Girls’ Education In Grades 7-10 Through Multimedia Distance Education In Afghanistan

Proposal

In these two presentations, we will (1) first provide an overview of our research design, and (2) second offer some of our preliminary findings from our data analysis.

Our research includes a multi-pronged impact evaluation, examining the accessibility and effects of the mass media and direct service programs described above on secondary school children’s access to education, learning, and social and emotional development. This impact evaluation covers TV broadcast programming, and assesses the type and number of contact hours between facilitators/providers and youth. The primary respondents for this phase are those receiving only distance education via broadcast services (TV and radio).

This study aims to understand the reach and causal impact of 15 weeks of distance education programming (TV and radio) in Afghanistan on 10-18 year-olds’ access to education programming and learning, including social and emotional learning outcomes. To do so, the evaluation created an RCT targeting young people living with various levels of access to media.

Working with our local partner, we first constructed a sample of probable media users in urban and rural areas in a vicinity of selected focal areas across the country. We then collected data via household surveys and learning assessments. T Although some of the sample did not watch the broadcast, it was important to include them in the study in order for the sample to be representative of the broader population that has access to the media. The diversity in our sample allowed us to assess how household characteristics are associated with engagement with the broadcasts as well as the effects of the broadcasts on academic outcomes. This diversity will also help in generalizing to the broader Afghan population.

To test impact, our RCT design randomly selected a subset of the households in the panel to receive intensive encouragement to engage with the educational media. These randomized encouragements were designed to create a large difference in engagement levels, with the treatment group having “intense” engagement and the control group having a lower level of “incidental” engagement. Differences between the treatment and control groups capture the effects of more intensive engagement with the educational media. Statistical methods based on “instrumental variables” allow us to estimate the effect of accessing the educational media per se. This design also allows us to assess household characteristics that affect access to the media with and without the intensive encouragement. Here we present our preliminary outcomes.

Authors

• Dana Burde, New York University

• Heddy Lahmann, New York University

• Abdul Hamid Hatsaandh, New York University

• Mohammad Jawed Nazari, New York University