Bayesian Bias-Aware Network Psychometrics

• In Event: AERA Poster Session Saturday 3:45 pm
  In Invited Poster Session: Division D Graduate Student In-Progress Research Poster Session

Abstract

Fairness in educational and psychological measurement is typically studied at the item level, focusing on whether item parameters differ across groups. Much less attention has been paid to how items relate to one another. If two items are locally dependent for one group but not another, the structure of residual dependence itself may be biased. This article outlines a bias-aware network psychometrics framework that treats residual dependence as a primary object of interest. Starting from a graded response item response theory model for ordinal data, standardized residuals are computed, and residual networks are estimated via weighted topological overlap (wTO). Group differences in edge weights are assessed using a permutation-calibrated statistic with Benjamini–Hochberg false discovery rate (FDR) control. A preliminary simulation with 12 items and four response categories, two groups of 1,500 examinees, and four local dependence edges in the focal group shows promising performance: all four edges are detected at q < .05 with no spurious discoveries. These results suggest that residual networks can complement traditional differential item functioning (DIF) analyses by highlighting where relationships among items differ across groups. Ongoing work extends the simulation design and embeds the network layer in a fully Bayesian model to propagate measurement uncertainty into the network. Keywords: item response theory, local dependence, network psychometrics, fairness, residual networks, permutation tests, false discovery rate

Author

• Fatih Ozkan, Baylor University