Predictive Validity of Soft Skill Measures: Evidence from Around the Globe

Group Submission Type: Formal Panel Session

Proposal

Education is about much more than just mathematics and literacy. As the CIES 2024 conference theme suggests, education can empower citizens to be a catalyst for change. But what are the skills that create this power and agency? Are traits like assertiveness, expressiveness, and extraversion related to future empowerment? The relationship between parental economic status and individual income cannot fully be explained by an individual’s childhood IQ or years of schooling (Bowles et al. 2001). Can soft skills be used to improve social-mobility for individuals from poorer backgrounds, by imparting soft skills that well-off parents pass to their children for labor market success? Can these skills help women succeed in work environments dominated by men? We propose a panel that will explore in depth the alternative skills, often called soft skills, and personality traits that are hypothesized to predict behavior and outcomes later in life.
We blend psychological theories of personality development and economic theories of skill accumulation to explain the relationship between these so-called “soft skills” and future life outcomes of youth and young adults (Bowles et al. 2001; Cabus et al. 2021]

This proposed panel explores from several different perspectives the predictive validity of different types of measurement instruments using longitudinal studies in different contexts in Africa and the Americas. The proposed panel would assemble scholars and scholarship from multiple disciplines and country contexts who have been convened by the panel organizers over the past three years to tackle a common set of research questions: What personality traits and skills beyond traditional academic skills like literacy and numeracy are important to the development and life outcomes of young people? Which of those skills predict such long term outcomes and how can we measure them?

Literally hundreds of instruments have been developed to measure soft skills [Galloway et al. 2017], and much is known about their basic psychometric properties like reliability and internal consistency (Galloway et al. 2017; Jones et al. 2020). However, very little is known about their relationship to longer term outcomes, because such evidence requires longitudinal data collection.

We consider the most commonly used measurement approach — self-reported checklists – as well as observation- and task-based measures, administered both in person and online. We take an approach of drawing together literature from 30 years of scholarship in multiple disciplines and
The panel is made up of two synthesis papers and a third presentation that goes into depth on one of the studies.

The first synthesis is a sweeping systematic review of papers from several disciplines published between 1990 and 2022. It examines several themes, organized by skill type and by measure type.

The second presentation summarizes lessons learned from seven parallel studies that were commissioned by Innovations for Poverty Action to better understand this topic. The studies were conducted in the United States by International Youth Foundation, Algeria by World Learning, Peru by the University of Toronto, Tanzania by the World Bank Gender Innovation Lab, Uganda by the University of California-Berkeley, and South were Africa by Oxford University and by the World Bank. Each study used different survey instruments and methods of data collection, but the research questions and statistical approaches harmonized.

The third presentation will drill down on one of the studies, conducted by the World Bank Africa Gender Innovation Lab, examining which of 14 skills and skill aggregates matter most for economic empowerment in Tanzania, and how this varies with gender and measurement-type.

The original contribution of this panel is to provide extensive empirical evidence to a topic that has so far been insufficiently explored, precisely because the data challenges are so steep. Following youth over time and measuring their soft skills at one or more points in time using multiple skill measures requires a much greater effort than traditional studies of concurrent validity, internal consistency, and test-retest reliability. This is because it takes time to observe both changes in measured soft skills but also in long term outcomes such as economic empowerment. In addition to requiring longitudinal data, this type of study often requires exogenous variation in the skill being studied, which is best identified through an intervention that is assigned randomly to a treatment and a control group. We assemble in one place all the known evidence on this topic, including past research from different disciplines, but ongoing research that the panel organizers have stimulated through a convening. We navigate the challenges posed by a lack of common language for describing these skills and situating them in conceptual frameworks ( Harvard Taxonomy Project). By presenting this evidence in one place we create the opportunity to build consensus around the skills and skill measures that best predict long term outcomes for youth.

Sub Unit

• General Pool

Chair

• Steven Glazerman, Innovations for Poverty Action

Individual Presentations

• Systematic Review of Methodological Studies of the Predictive Validity of Soft Skill Measures - Steven Glazerman, Innovations for Poverty Action; Sarah Kabay, Innovations For Poverty Action; Savanna Henderson, Innovations for Poverty Action; Smita Das, Innovations For Poverty Action

• Parallel Studies on Measuring the Association of Soft Skills on Longer-term Labor Market Outcomes - Catherine A Honeyman, World Learning; Linda Fogarty, IYF; Steven Glazerman, Innovations for Poverty Action

• Socio-Emotional Skills in Tanzania - Smita Das, Innovations For Poverty Action