New Insights From Applying Differential Item Functioning and Bifactor Models to a Scientific Reasoning Test

• In Event: Poster Session 15
  In Poster Session: Division D Poster Session

Abstract

Advanced analysis methods are presented and compared to previously used methods in the field of scientific reasoning (SR) assessment. Tree and lasso based DIF models are applied to evaluate the domain generality assumption of a SR test. Bifactor model based calculations explore the connection between general reasoning and SR. We tested 507 university students majoring in physics, biology, or medicine. Mean comparisons showed an advantage of physics students but DIF analyses revealed biases benefitting biology students. According to regressions the unique variance of SR is at least 64%. Bifactor model based calculations put this number at less than 1%. Overall, the presented analyses provide important insights and are a much needed addition to the evaluation of SR tests.

Authors

• Ansgar Opitz, Ludwig-Maximilians-Universität München

• Moritz Heene, Ludwig-Maximilians-Universität München

• Frank Fischer, Ludwig Maximilian University of Munich