Parallel Studies on Measuring the Association of Soft Skills on Longer-term Labor Market Outcomes

• In Event: Predictive Validity of Soft Skill Measures: Evidence from Around the Globe

Proposal

This presentation summarizes lessons learned from seven parallel studies that were commissioned by Innovations for Poverty Action to better understand the predictive validity of various soft skill measures on longer-term economic empowerment outcomes including employment, earnings, and job performance. Each study used different survey instruments and methods of data collection, but the research questions and statistical approaches were harmonized. The studies are summarized as follows:

1) A team at the World Bank has developed a series of online, open access, task-based skill measures called SkillCraft, which uses games to measure cognitive skills, risk-taking, and soft skills such as emotional recognition and persistence. These, along with self-report scales, were tested with young jobseekers in Ethiopia and South Africa.

2) A team at the University of Toronto created and tested a virtual online platform to measure collaborative skills and four dimensions of group dynamics among low-income Peruvian students. An RCT gauges the impact of an “escape room”-style virtual game focused on developing leadership and teamwork skills.

3) A team based at the University of Oxford and Duke University examined growth mindset and behavioral activation among job candidates from 1,200 branches and 8 firms in South Africa, and evaluated the predictive validity of self report and task-based measures, and correlations of measures with regards to hiring, job performance, retention and earnings.

4) A team from World Learning studied the ability of its self-administered soft skills measurement instrument, the Worklinks Skills and Values Assessments (WLSVA) as well as an observation-based measure, to predict labor market outcomes for 4,000 youth in Algeria following exposure to different combinations of trainings.

5) A team from International Youth Foundation (IYF) studied the predictive validity of three different methods of measuring soft skills: self-reported survey, scenario-based questionnaire, and employer feedback. The team enrolled nearly 300 low-income Chicago-area youth in the study, half of whom were assigned soft skills training. All were followed up to measure employment and earnings outcomes.

6) A team at UC Berkeley has been studying the impact of life skills training in Uganda for several years using randomized controlled trials (RCTs) of Educate! and SEED programs. Long-term (8-year) followup on approximately 6,000 Ugandan young adults provides an opportunity to examine how measures of soft skills (self-reports, task-based, and biomarker measures of emotional regulation) predict labor market outcomes. The study uses machine learning heterogeneous treatment effects estimation to identify causal pathways through which soft skills influence later outcomes.

7) The World Bank’s Africa Gender Innovation Lab, BRAC, and IPA have partnered to examine the predictive validity of self reports and alternative measures (scenario-based measures and tasks) for 14 socio-emotional skills. Using a sample of 4800 men and women, ages 16-27 who are neither full-time students nor employed full time, the team will randomize which skills are included in 27-54 hour training programs, and the impact of this training on later skill measures and economic empowerment.

We discuss findings, fieldwork challenges, and study design issues, which should aid researchers planning future work in this area.

Authors

• Catherine A Honeyman, World Learning

• Linda Fogarty, IYF

• Steven Glazerman, Innovations for Poverty Action