Investigating the association between instructional OTL factors and mathematics attitudes using NAEP 2022

• In Event: AERA Roundtable Session Friday 11:40 am Four Seasons Ballroom 2-3
  In Roundtable Session: Exploring Student SES and OTL From the Perspectives of Large-Scale Education Studies (Table 2)

Abstract

Opportunities to learn (OTL) can be defined in many ways and levels and has been shown to be significantly associated with achievement (Schmidt et al., 2021). One of the many definitions for OTL is students’ exposure to content. One study observed a relationship between differentiated instruction (i.e., a type of OTL) and affective dispositions toward learning mathematics, dispositions such as enjoyment and confidence (Gervasoni et al., 2021). In this study, we sought to address the following questions: (1) What types of instructional OTL factors can be extracted from the teacher questionnaire? and (2) Which of these factors can explain a larger amount of variance in students’ enjoyment of math and confidence in math knowledge?

Using student and teacher questionnaires and school administrative data from the 2022 NAEP Mathematics assessment for grade 8 exploratory factor analyses (EFA) and confirmatory factor analyses (CFA) were conducted to generate instructional OTL factor scores. Criteria for factor extraction included selecting eigenvalues based on parallel analysis, factor loadings greater than .30, selecting factors that had three or more items, and selecting factors that had Cronbach’s alphas greater than .80. Fit index thresholds for the CFA followed the AERA, APA, and NCME (2014) guidelines to assess whether the models met acceptable fit criteria. CFA models with acceptable fit criteria were used to develop factor scores. The generated factor scores would then be included in regression models. The OTL factors, covariates (i.e., gender, economically disadvantaged, and race at the school level), confidence in the mathematics knowledge index, and enjoyment of the mathematics index were used to assess the amount of explained variance among the models. We assessed the amount of explained variance in the stepwise regressions.

The EFA and CFA resulted in the extraction of four factors that met the fit criteria: differentiated instruction, teaching practice for argumentation, teacher’s self-efficacy during remote instruction, and teacher’s collaboration during remote instruction (see Table 1). For predicting confidence, the OTL factors and covariates were similar in sign and magnitude between each block of the models (see Table 2). With the addition of each OTL variable in subsequent models, the variance explained in confidence was relatively equal across each block (adjusted R2 = 0.002, or 0.2%). For predicting enjoyment, the OTL factors and covariates were also similar in sign and magnitude between each block of the models (see Table 3). With the addition of each OTL variable in subsequent models, the variance explained in enjoyment was relatively equal across each block (adjusted R2 = 0.022, or 2.2%).

The stepwise additions of variables that help characterize instructional OTL (i.e., differentiated instruction, teaching practices for argumentation, teachers’ self-efficacy during remote instruction, and teachers’ level of collaboration during remote instruction) appeared to make a minimal difference in predicting a student’s enjoyment of and confidence in mathematics. The evidence provided by this study suggests that using these proxies for instructional OTL requires additional justification or assessment of their relationship with performance, which would be the next feasible step.

Authors

• Antranik T Kirakosian, Washington State University

• Robert Finnegan, Educational Testing Service

• Marisol Kevelson, Educational Testing Service

• Selene Sunmin Lee, McGraw Hill