Examining the Latent Structure of Self-Reported Constructs to Enhance STEM Program Evaluation

• In Event: AER Roundtable Session Friday 1:45 pm - Gold 4
  In Roundtable Session: Innovations in Item Response Theory for Modeling Student Performance and Growth

Abstract

This study investigates the psychometric quality of self-report survey instruments commonly used in STEM program evaluations, focusing on science and engineering: interest, efficacy, and value constructs. Drawing from over a decade of STEM education evaluations, we compiled multiple datasets to conduct a Rasch-based analysis using the Partial Credit Model. We assessed item and person fit, rating scale functioning, and dimensionality across constructs. Results revealed stronger psychometric properties for engineering constructs compared to science, with higher unidimensionality and WLE reliability. Wright maps indicated good item-person targeting, though scale category issues emerged in some items. These findings support refining self-report tools to better capture students’ STEM-related beliefs and improve the sensitivity of program evaluations.

Authors

• Theode Niyirinda, University of Alabama

• Joni M. Lakin, University of Alabama

• Daniel O. Oyeniran, University of Alabama