Bifactor-ESEM Validation of Basic Psychological Need Satisfaction and Frustration in Classroom Assessment

• In Event: New Opportunities in Motivation and Emotion Research Through Advanced Statistics

Abstract

Purpose: The purpose of this study was to create a basic psychological need satisfaction and frustration (BPNSF) scale for the domain of classroom assessment (CA) and collect evidence of its validity.

Theoretical framework: Students characterize assessment in higher education as anxiety-provoking, frustrating, annoying, and one of the largest and most persistent contributors to their stress (Linden et al., 2023). Despite this, researchers have not leveraged psychological theories to address the deleterious impact of this unavoidable aspect of education. According to self-determination theory (SDT), satisfaction of autonomy, competence, and relatedness acts as a “nutrient” in supporting student well-being; whereas, frustration of these three BPN acts as a “vulnerability factor for ill-being” (Vansteenkiste et al., 2023, p. 90). This means that the tendency for classroom assessment to routinely compromise student well-being may indicate an absence of BPN satisfaction and an abundance of BPN frustration. Research in this domain is stalled by the lack of a robust domain-specific measurement tool.

Method: We designed the 24-item BPNSF-CA survey by modifying Chen and colleagues’ (2015) scale (Appendix A). We collected validation evidence through a self-report study with n=400 college students using Prolific (ethnicity 50% White, 24% Asian, 10% Black, 10% mixed; gender 46% men, 50% women, 4% non-binary/not reported; Mage = 27.12, SD=9.08). We measured stress, anxiety, fairness, and well-being using four single items to represent the nomological network of student assessment well-being (Daniels & Wells, 2024). Analyses were run in MPLUS 8.3 (Muthén & Muthén, 2017) with maximum likelihood estimator. We followed Alamer’s (2022) decision tree to contrast CFA, ESEM, bifactor CFA, and bifactor-ESEM. We expected a bifactor-ESEM to provide the best fit of the data including three specific satisfaction factors, three specific frustration factors, and one general factor.

Results: Fit indices for all models are in Table 1. We chose to retain the bifactor-ESEM over ESEM (Table 2) because it was characterized by (1) a well-defined and highly reliable G-factor (𝝺 range=⎸.47⎹ to ⎸.72⎹, M𝝺=⎸.59⎹, ⍵=.95); (2) six S-factors with good reliability (⍵=.79 to .88); (3) small to moderate cross-loadings (M𝝺 = ⎸.11⎹) with a decrease in the range compared to ESEM (𝝺 range=⎸0⎹ to ⎸.36⎹). In the validation model, the G-factor was significantly associated in the expected directions with all four indicators of student assessment well-being (Table 3) and unique S-factor associations emerged.

Scholarly Significance: This research makes two important conceptual contributions. First, the validity evidence shows that BPNSF are important considerations for student assessment well-being. As a part of schooling that consistently undermines well-being, this creates space for research to focus on improving assessment through BPN. Second, identification of strong G- and S-factors suggests that researchers need to think on both global and specific levels. Empirically, this means that if researchers continue to model BPNSF without a G-factor, it is likely that the associations between specific satisfaction and frustration factors will be overestimated (Sánchez-Oliva et al., 2017) perpetuating an undifferentiated approach to using BPN to support well-being.

Authors

• Lia Daniels, University of Alberta

• Kendra Lin Wells, University of Alberta