Informing NAEP Survey Development: Noncognitive Predictors of College Enrollment From the High School Longitudinal Study of 2009

• In Event: Does NAEP Predict Student Postsecondary Preparedness?

Abstract

This exploratory paper examines the predictive validity evidence of noncognitive factors NAEP could include within its student surveys. It is generally acknowledged that college preparedness necessitates noncognitive skills as well as academic preparation in core subject areas. However, there is no consensus on which noncognitive factors measured in high school, are considered to be most predictive of later college success (Farrington et al, 2012; National Research Council, 2012).

Specifically, it is unclear to what extent noncognitive factors would predict college success over and above NAEP scores. Decisions about what noncognitive constructs to measure within NAEP surveys is generally based on the survey constructs’ relationship with NAEP achievement (NAGB, 2013). Predictive validity evidence showing the relationship between noncognitve factors and college success could provide another criterion to consider when deciding what to include in NAEP student surveys.

Analyzing HSLS:09 data, we examine whether the following noncognitive factors are related to postsecondary attendance three years after high school graduation: mathematics/science self-efficacy, mathematics/science identity, mathematics/science interest, mathematics/science utility goals and values, school belonging, and academic behavior (Table 3.1).

As can be seen in Figure 3.1, our analysis employed a structural equation modeling approach where the relationship between each noncognitive factor and four-year enrollment is tested before and after controlling for 9th grade algebra performance. Ogut et al. (2015) found a substantial correlation between HSLS:09 algebra scores and NAEP mathematics achievement. As such, in this analysis the algebra scores serve as a proxy for NAEP achievement. Controlling for the algebra scores thereby provides an estimate of this relationship after controlling for NAEP achievement. Given the large sample size in HSLS:09, we test for practical significance using a standardized coefficient of 0.1.

Tables 3.2 and 3.3 show preliminary results. All ten models have satisfactory fit (RMSEA <0.05, Comparative Fit Index <0.96). Table 3.3 provides the path coefficient results. Eight of the ten factors showed significant positive relationships with the postsecondary outcome before controlling for 9th grade Algebra, with only mathematics and science utility having nonsignificant relationships with the postsecondary outcome.

Of particular interest for this analysis is the relationship for each of the constructs with four-year enrollment, after controlling for 9th grade algebra scores. The results show that academic behavior, school belonging, and science identity are positively related (β>0.1) to college enrollment status after controlling for algebra scores.

This research provides key evidence that three noncognitive predictors are positively associated with college enrollment after controlling for achievement. Based on this HSLS:09, these predictors could be considered for NAEP surveys as indicators of college preparedness. Academic behavior and school belonging, seem to be especially important for predicting college enrollment three years after high school. Science identity is also related to later postsecondary enrollment, but mathematics identity is not— further research should examine whether this is because science identity is more predictive of college outcomes or due to the fact that algebra achievement (and not science achievement) was used as a control variable.

Authors

• Martin Hooper, American Institutes for Research

• Alison Filbey, American Institutes for Research

• Jennifer Mazzella, American Institutes for Research

• Juliet Holmes, American Institutes for Research

• Grace Ji Cheng, American Institutes for Research