Long-Run Impact of an Applied STEM Program: Evidence From Missouri

• In Event: Pathways Into STEM Fields: Studies of Project Lead the Way

Abstract

Objective
To improve STEM education, one of the national initiatives during the early 2010’s focused on integrated applied STEM curricula. Project Lead the Way (PLTW) is one such program and currently adopted by over 12,000 high schools nationally. In Missouri, the program expanded from 13 schools in 2005 to 174 schools by 2020 – 35% of all public high schools.
We investigate the impact of PLTW program offering (ITT) and the participation impact on participants (TOT). The outcomes include college enrollment, STEM major declaration, overall degree attainment, and STEM degree attainment.

Theory of Action
Applied STEM programs attempted to increase student participation in more meaningful/ engaging STEM learning and generate greater interest in STEM by connecting the traditional STEM disciplines to the real-world/applied problems. PLTW provided active, project-based learning in Engineering, Computer Science, and Biomedical Science (“Pathways”). Foundational courses introduce major ideas of the field with a goal of developing an enthusiasm for further study. To expand students’ understanding of the field, advanced courses offer deeper and specialized content through capstone projects where students took their own ideas from design through development of a product.

Data
We use three cohorts of first-time 9th-grade students who began Missouri public high school in AY2010, AY2013, and AY2014 (N≅ 68,000 per cohort). Racial compositions are 75.8% White, 16.5% Black, 4.2% Hispanic, and 3.6% others. The FRL rate is 42.4%. The analytic population consists of students in two types of schools: schools that began offering PLTW for the 2013 and/or 2014 cohorts, but not for the 2010 cohort (N=92 schools and 43,578 students) and schools that never offered PLTW for the three cohorts (N=273 schools and 49,788 students).

Research Design
We apply a Difference-in-Differences analysis (DiD) to estimate the ITT impact. This is estimated by deciles of “prognostic scores”—predicted probability of completing a STEM degree given pre-treatment predictors (math and science test scores and GPAs). Students in non-PLTW schools serve as a comparison group, providing an estimate of the outcome difference between the 2010 and later cohorts in the absence of PLTW.
To estimate the TOT impact, the propensity-score adjusted DiD is used. Propensity scores—the probability of program participation given pre-treatment covariates—are used to construct the “would be treated” and “would be untreated” populations from the 2010 cohort who are made similar to, respectively, the 2013/2014 program participants and non-participants in the same PLTW school. The would-be-untreated and observed-untreated students and students in non-PLTW schools are two comparison groups for the DiD analysis. In additional DiD analyses, these two comparison groups are weighted/matched to resemble the program participants.

Results
We found positive ITT effects on STEM major declaration and STEM degree completion for the top prognostic-score deciles. Program participants also had higher STEM enrollment and degree completion.

Significance
Our evidence suggests that an applied STEM program like PLTW has a potential to increase post-secondary STEM outcomes among students with higher STEM readiness. The study’s limitations include bias due to unobserved confounders. This will be addressed through sensitivity analyses.

Authors

• Darrin Joseph DeChane, Saint Louis University

• Dillon Fuchsman, University of Missouri - Kansas City

• Takako Nomi, Saint Louis University

• Michael Podgursky, University of Missouri