Evaluation of industry-driven technical and vocational training courses in the country of Georgia

• In Event: Educational Research and Diversity

Proposal

At independence, the country of Georgia inherited a dilapidated Soviet technical and vocational education and training (TVET) system. Despite an intensive effort to reform the system since 2013, by the mid-2010s there remained a strong mismatch between TVET training and skills demanded by industry. TVET was poorly perceived and viewed as an inferior option relative to higher education, while the lack of industry-relevant technical skills in the workforce was a constraint to economic growth.

The Industry-Led Skills and Workforce Development project was designed to increase the availability of Georgian technical workers with skills aligned with industry demand. The centerpiece of the project was an $11.7 million initiative that funded Georgian TVET providers on a competitive basis to establish new or improved training courses that reflected industry demand for skills. The 10 institutions selected to receive grants established 23 new diploma courses, improved 15 existing diploma courses, and established 13 new short certificate courses. Most of the diploma courses were at the highest levels within the TVET system and were between 6 and 24 months in duration. We conducted an outcomes study to measure the labor market outcomes of trainees in project-supported courses and used several analytical approaches to benchmark those outcomes in the absence of a well-defined counterfactual. A complementary qualitative study explored the potential mechanisms driving the results observed in the outcomes study.

We found that more than three-quarters of trainees in project-supported courses found employment within a year of the course ending. However, only one-third found a full-time job that was relevant to their course, reflecting both supply and demand constraints. Specifically, among the trainees who did not obtain a course-relevant job, about one-half never searched for one due to a lack of interest or availability for work, or perceptions wages would be too low (supply-side constraints). A lack of available job opportunities might have also been a constraint for some trainees, although the courses were intended to be in areas where labor demand was high (demand-side constraints). This was especially true for those with limited relevant work experience: trainees who had more than two years of prior work experience obtained full-time, course-relevant jobs at about double the rate of those with less experience. This suggests that employers in course-relevant fields may have had substantially less demand for course graduates who did not also have meaningful prior work experience.

The evaluation did not include a rigorous impact analysis with a well-defined counterfactual for the outcomes of trainees in project-supported courses. However, results from the study’s three descriptive benchmarking exercises consistently suggest that the courses were likely to have improved trainees’ employment rates and wages. Compared to a national benchmark of all TVET courses in Georgia, graduates of project-supported courses maintained higher rates of employment during the pandemic. Specifically, for graduates surveyed during the pandemic (one year after graduation, on average), those in project-supported courses were 15 percentage points more likely to be in paid employment and 12 percentage points more likely to be in course-relevant paid employment than a nationally representative sample of TVET graduates. In part, this could be because many project-supported courses focused on sectors and roles that tended to have more stable employment during the pandemic.

Further, relative to the national benchmark, graduates of project-supported courses earned wages that were 12 to 13 percent higher for male and female graduates, respectively. For nine existing courses enhanced by project grants, trainees during the compact appear to have earned wages that were 16 percent higher than those of earlier cohorts. Finally, trainees who were employed before enrolling in a project-supported course increased their earnings by 10 percent after the end of the course, and trainees who returned to the same job experienced a 21 percent wage boost. Overall, these findings consistently suggest that the project-supported courses meaningfully boosted trainees’ employment and wages relative to existing courses in Georgia.

The findings have several implications for policy and practice. First, TVET providers need to strengthen systems for identifying and enrolling trainees who are ready to pursue careers in their chosen fields. The existing literature on TVET effectiveness focuses heavily on constraints related to employer demand for trainees, but creating a supply of graduating trainees interested in pursuing a relevant career is also critical to the success of these programs. Relevant interventions could include encouraging providers to more fully screen and support trainees, providing career guidance during secondary school to support students in making more informed choices about vocational education, and establishing a well-functioning labor market information system, so that trainees would have more realistic expectations about job opportunities and wages. Second. there is a need for TVET providers to engage with a broader range of employers, rather than just a few large providers in specific sectors. A few of the project-supported courses benefited from very strong relationships with large employers, but these relationships proved more difficult to create in sectors where employment tends to be scattered across smaller firms. Developing broader private-sector links with smaller firms could have helped to provide better pathways to employment—especially for trainees who came out of the course with no experience. Third, there is a strong argument that investing in well-designed and sustainable TVET programs can improve trainees’ labor market outcomes and produce a positive economic return. Establishing new TVET programs and policy reforms is difficult work, and there is no guarantee that new or enhanced course offerings will establish a sustainable business model, identify trainees appropriately, involve employers adequately, and succeed in matching graduates to well-paid positions in their fields. But the evidence from Georgia suggests that TVET does have the potential to position trainees to improve their performance in the labor market and meet the skills needs of employers in their fields.

Authors

• Evan Borkum, Mathematica

• Ira Nichols-Barrer, Mathematica