Can Parenting Programs Give Children Wings?: Findings from an Impact Evaluation of Parenting Program in Roma Communities in Serbia

• In Event: Impacts of ECE programs

Proposal

OVERVIEW
Roma communities in Serbia have been disadvantaged for years, but new investments by the government, foundations, and multilaterals are increasing the need for evidence of existing programs that could be scaled to make a difference in child outcomes. The program for children and families - Strong from the Start - Dam Len Phaka - Give Them Wings (“the program”) is a parenting program administered by NGOs within Roma settlements to build parenting skills, but also to build healthy relationships between parents and their children. Our quasi-experimental impact evaluation produced a midline report that showed a statistically significant correlation between program participation and child school readiness outcomes (socio-emotional development, literacy, and numeracy) for children aged 3-6.

In February 2019, our final report will update audiences on whether or not children continued to make development progress above and beyond children in the comparison group.

This presentation will share our promising midline findings, yet to be released endline findings, as well as recommendations for improved program implementation in the future.

EVALUATION DESIGN OVERVIEW
The program enrolled 30 families with children aged 0 to 7 in each of the 15 program communities in late 2016 (450 families total). 15 comparison sites were then matched based on population, urbanicity, living conditions, presence of Roma liaison in local government as advocate, etc. and appropriate were families identified with the assistance of a local Roma NGO. The baseline analysis confirmed these families were similarly situated across multiple dimensions.
The evaluation used a quasi-experimental design and a differences-in-differences framework to compare changes in parent and child outcomes for those who participated in the program against a comparison group that did not participate. The parent and child outcomes evaluated include changes in parent attitudes, knowledge and behavior around parenting practices, and child development, specifically socio-emotional development. Within each family in the program and comparison group, an adapted version of the IDELA tool was administered to one child (if the child was aged 3.5 – 5.9 years at baseline) and a survey based of the IDELA and UNICEF’s MICS to the parent (most commonly the mother).

FINDINGS FROM CHILD ASSESSMENT
1. Gains made in total child development in the program group at midline were statistically significantly higher than those made by the comparison group. Scores for both the program and comparison group increased over the course of the first year of implementation, however, program children increased at a statistically significant higher rate (31% increase in score) than those of the comparison group (24% increase in score). These analyses were found to be significant when controlling for socio-demographic factors and participation preschool or other early learning programs.
2. Similarly, gains made in the socio-emotional sub-domain, this evaluation’s primary indicator, as well as early numeracy, early literacy, and persistence sub-domains, were statistically significantly higher than those made by the comparison group. The socio-emotional sub-domain was identified at the outset of the evaluation as the primary indicator, the indicator that the program’s felt most confident in positively affecting. And indeed, the gains made by the program group (27% increase) exceed than those made by the comparison group (20% increase) and hold when the scores are controlled for socio-demographic differences and participation in an early learning or preschool program. Similar gains were found across the other IDELA sub-domains of early numeracy, early literacy, and persistence.
3. Program participation positively impacted gains in child development scores. When scores were regressed against attendance, the analyses showed that children who participated in more than ten workshops saw statistically significant gains in school readiness, while children participating in 10 or fewer workshops did no better than those in the comparison group. However, parent attendance did not have a significant effect on child development outcomes

FINDINGS FROM PARENT SURVEY
1. Parents’ attitudes about some parenting practices and knowledge of their impact on children’s development improved over the course of the program. Program parents reported greater satisfaction in their relationship with their child and greater confidence in their parenting than the comparison group, something that was not the case at baseline. A statistically significant difference emerged at midline on program parents’ knowledge about their agency in their child’s development at home and on the importance of reading and writing.
2. After Phase 1, parents in the program reported decreases in the type of negative discipline techniques used (e.g. hitting, spanking, and criticizing), but curiously, they also reported decreases in the number of developmentally supportive activities they engaged in with their children. At baseline, the program group reported using more negative disciplinary techniques than the comparison group, however, at midline, this relationship reversed. Importantly, the program group’s use of each technique dropped between baseline and midline, and to rates below those of the comparison group. For instance, at baseline 19.1% of parents in the program group reported hitting their child for misbehaving (5.6% of comparison group), but only 5.3% report doing so at midline (6.5% comparison group).
3. On average, parents attended fewer than half of the available program workshops. Of the potential 28 workshops, parents, on average, only attended 11. Lack of participation could be for many reasons and could explain why there wasn’t as much change in parent behavior or attitude.

CONCLUDING RECOMMENDATIONS
1. Midline evaluation results for children are promising and point to significant changes in child development.
2. Midline evaluation results for parents point to important reductions in the use of negative discipline techniques.
3. Limited changes in the use of developmentally supportive activities suggest that program designers might consider renewed emphasis on these activities in year two.
4. Efforts should be made to increase attendance for parents and children and ensure that the Roma NGOs delivering the program are adequately trained and monitored.

Authors

• Lucas Heinkel, Results for Development

• Kavita Hatipoglu, Results for Development Institute