An Intersectional Investigation of Measurement Invariance on the Force Concept Inventory

• In Event: Advancing Methods for Evaluating Construct Validity in STEM Education Research

Abstract

In this investigation, we expand the validation argument of the Force Concept Inventory (FCI; Hestenes et al., 1992) through an intersectional evaluation of measurement invariance. To ensure the accuracy of the comparisons across groups requires that an instrument be measurement invariant. Eaton and Willoughby (2018) established the FCI as covering five factors (the EW5 model) in students’ conceptual understanding of forces and kinematics. They found the five factors were invariant across gender (Eaton, 2021). We extend the validation argument for the FCI’s five factors (Table 1), across the intersections of race, ethnicity, and gender.

Theoretical framework:
Intersectionality (Collins, 2002; Crenshaw, 1991) posits that individuals’ social identities (e.g., race, gender, and ethnicity) interact in complex ways with each other and the societal power structures to create privilege and oppression, (McCall, 2005). The advantages and disadvantages accrued through each aspect of an individual’s social identities are often not additive in nature.

Methods:
Based on Rocabado et al. (2019), we examined three stages of measurement invariance across ten race/ethnicity/gender groups. These three stages are necessary for making comparisons across groups. In keeping with Eaton’s (2021) factor structure of the FCI, all factors held equal weight for all stages. For all three stages, we examined the following fit indices and cutoffs: CFI > 0.9, TLI > 0.9, and RMSEA < 0.06 (Putnick & Bornstein, 2016).

Data sources:
We collected the data from the LASSO platform’s anonymized research database (Van Dusen, 2018) for 15,666 students from 333 courses at 48 institutions. Based on sample size, we were able to study the intersection of five racial/ethnicity groups - Asian, Black, Hispanic, White, and White Hispanic, and two genders - men and women.

Results:
We used a change in CFI across the stages of 0.02 (Putnick & Bornstein, 2016) as a cutoff for invariance in each stage (see Table 2). The FCI’s EW5 model met configural, metric and scalar invariance on the pretest and on the posttest. Therefore, the EW5 factor model of the FCI supports comparisons across factors and groups.

Scientific significance of the work:
Instrument developers seldom examine instrument validity across gender, race, and ethnicity (Martinková et al., 2017; Traxler et al., 2018). The lack of these validation arguments for diverse populations may undermine comparisons across groups on assessments, a central measure in many investigations of diversity, equity, and inclusion.This work can guide future intersectional measurement invariance analyses to ensure the assessments provide meaningful and unbiased comparisons across groups. The FCI is the most commonly used research-based assessment in physics. Establishing its measurement invariance allows ongoing work to use the factor structure on that instrument to investigate gender and racial inequities in physics and inform policies and practices that address those inequities.

Authors

• Alicen Morley, Iowa State University
  Presenting Author

• Jayson Nissen, Nissen Education Research and Design
  Non-Presenting Author

• Ben Van Dusen, Iowa State University
  Non-Presenting Author