What Do They Remember? Probing for Recall in an Employment Survey Using Cognitive Interviews for USAID’s Employment Programs

• In Event: USAID’s revised guidance for measuring employment outcomes of youth employment programs

Proposal

USAID’s Office of Education is developing an employment survey to measure key employment related outcomes, including employment and earnings, among youth who participate in USAID’s WFD programs. These programs aim to prepare at-risk youths for or place youth in wage employment or self-employment. Activities may include skills and entrepreneurship training, career counseling, or job matching. In 2017, USAID implemented 47 WFD programs in 34 countries globally, reaching 725,000 youth.

USAID’s WFD programs worldwide will use the employment survey to measure changes in youth employment status and earnings pre- and post- intervention. The data generated from the survey will enable USAID to compare and assess outcomes across countries and contexts. However, various studies have identified recall as an important issue in collecting accurate employment and earnings data, particularly among youth. It is a systematic bias that researchers have identified in employment and earnings data. In addition, a recent USAID-funded peer review of various youth development measurement tools, for example, noted that it was unclear whether youth could accurately report changes in their income. Other studies, mainly among adults, have found that recall bias is a significant issue when it comes to reporting hours worked and income earned.

USAID conducted cognitive interviews Fall 2018 in the Philippines, Kenya and El Salvador with more than 60 youth to better understand what information youth can accurately provide on their number of hours worked and on their earnings. Differences in recall for both time worked and amount earned were evident by the type of employment youth are engaged in. A variety of cognitive probes were used to better understand the timeframe for which youth can accurately recall this information. In this presentation we will share how we sought to better understand the issue of recall for these types of data and implication of results for the employment survey.

Authors

• Elena Walls, US Agency for International Development (USAID)

• Mousumi Sakar, Consultant