Improving Measurement in Career and Technical Education to Support Rigorous Research

• In Event: Measures, Data, and Counterfactuals: Approaches to Improving Research in Career and Technical Education

Abstract

Nationwide, more than half (58 percent) of high school students and two-thirds (66 percent) of those in community colleges choose to concentrate their studies in career and technical education (CTE), yet little is known about the efficacy of this programming. Most studies of CTE have not used research designs that support causal inference, relying instead on information from nationally representative datasets to estimate relationships between students’ CTE course taking and subsequent education and employment outcomes. These studies have tended to associate students’ in-school experiences and post-program results with levels of credit accumulation, but do not account for other measures of experience that educators and policy makers argue are likely to contribute to any potential impact. As a result, the literature has failed to fully capture the effect that CTE programming has on those individuals who concentrate in and complete a comprehensive, high-quality program of study. There is a need to strengthen the rigor of CTE research to estimate the impact of CTE exposure on students. The current study catalogs the measures of CTE exposure used in prior studies, documents measures and outcomes in more recent causal studies, and offers a framework for how CTE exposure and outcomes should be measured in the future to contribute to this field.
To examine the measures used in previous studies CTE research, we compiled a list of 75 papers assessing the relationship between CTE participation in high school and graduates’ subsequent postsecondary education and employment outcomes. This list was then narrowed down to 30 papers to only include those that were specifically focused on CTE and that employed empirical quantitative research. Each paper was then coded and categorized based on its methodology, data/sample, measure(s) of CTE exposure, key covariates, and outcome measures.
Analysis of the coded data most notably revealed that the modal mechanism for measuring CTE exposure was some form of high school CTE course-taking with the number of courses taken as the most commonly used metric (16/30 studies). In addition, many papers took advantage of nationally representative longitudinal datasets that contained student transcript data (14/30). The most commonly measured outcomes were labor-market related (17/30); almost always, this meant some measure of earnings and/or employment status. Many papers (12/30) included high school graduation as an outcome measure, with a similar number including other high school achievement metrics (12/30), such as attendance, GPA, dropout status, and/or academic performance as measured by standardized test scores. Lastly, a slightly smaller portion (10/30) included some form of post-secondary success measure, such as enrollment or completion of an associate’s or baccalaureate degree.
Moving beyond measuring CTE experiences in high school primarily by course taking behavior is critical for capturing potential differences in programming and elements that may relate to subsequent student academic and workforce outcomes. As such, the paper concludes with a thorough outline of alternative approaches to measuring CTE exposure and student outcomes.

Authors

• Shaun Michael Dougherty, Vanderbilt University

• Samuel J. Kamin, University of Connecticut

• Steve Klein, Education Northwest