Beginning Teachers’ Self-Reported Challenges:​ Evidence from Induction Program Narratives​

• In Event: The Teacher Pipeline Under Pressure: New Evidence on Interest, Certification, Hiring, and Retention

Abstract

Beginning teacher attrition costs school systems more than two billion dollars annually and falls disproportionately on high-need schools. Yet the administrative records that track who leaves contain almost no information about what those teachers experienced in their classrooms. This paper bridges that gap by applying large language model classification to 86,727 written narratives submitted by beginning teachers in a statewide induction program between 2011 and 2024.

The program structures professional development across four sequential modules: Classroom Environment, Planning, Instruction, and Assessment. At the start of each module, every participating beginning teacher selects a growth focus and writes a self-assessment of their current practice. Using a combination of transformer-based topic modeling and a three-step concept-guided chain-of-thought classification pipeline, we classify each narrative against a validated framework of 27 binary challenge indicators and link the resulting measures to administrative records for 14,301 general and special education beginning teachers.

We document three findings. First, the challenge landscape is concentrated but heterogeneous: promoting student engagement (63.9%) and fostering behavioral standards (62.5%) are the most frequently reported Classroom Environment challenges, while planning, instruction, and assessment challenges are more evenly distributed. Fewer than 3% of Classroom Environment module narratives contain no identifiable challenge, rising to roughly 6% in later modules. The average beginning teacher accumulates 6.1 challenge flags across all four modules.

Second, observable teacher characteristics predict challenge profiles but explain very little of their variation. Assignment role is the strongest predictor, with special education teachers substantially more likely to flag individualized intervention challenges (up to +29.8 percentage points in the Assessment module) and less likely to flag whole-class instructional design challenges (−16.7 pp in the Instruction module). Gender is a consistent secondary predictor, and race and ethnicity predict smaller but substantively coherent differences concentrated in classroom environment and instructional strategies. Even the strongest predictors leave more than 90% of challenge variation unexplained, and school fixed effects account for at most 4.4% of any single indicator's variance. Challenge profiles are overwhelmingly a within-school, within-group phenomenon.

Third, this residual within-group variation carries information about career trajectories. In a preferred specification entering all indicators simultaneously with school and entry-year fixed effects, Classroom Environment challenges with promoting student engagement (−3.3 pp district, −3.9 pp school) and fostering behavioral standards (−3.1 pp district, −3.0 pp school) predict lower three-year retention and survive Benjamini-Hochberg correction. Planning challenges with content-area literacy and numeracy instruction predict higher retention (+3.1 pp district, +3.6 pp school), also surviving correction. Assessment module associations are nominally significant but do not survive multiple-testing adjustment. Retention associations concentrate at within-state mobility rather than system exit.

Author

• Mingyan Ma, Boston University
  Presenting Author