Measuring the Impact of Multidimensional Science Instructional Materials

• In Event: Saturday Roundtable Session 1:15 pm
  In Roundtable Session: Innovative Methodologies in Item Response Theory (Table 20)

Abstract

NGSS-aligned assessments that measure multidimensional science learning tend to be time and resource intensive. This presents a challenge when developing instruments to measure the impact of large-scale interventions and related trade-offs of using different assessment formats. Rasch and two-level random intercept models were employed to investigate if multidimensional tasks and multiple-choice (MC) items could detect an effect of the use of instructional materials focused on developing students’ model-based reasoning. A significant effect was detected with the multidimensional tasks. While the MC items did not show an effect, they were helpful in reducing testing burden and estimating more accurate person measures on the assessment. Our findings suggest utilizing both multidimensional tasks and MC items to measure impacts feasibly and effectively.

Authors

• Cari F. Herrmann-Abell, BSCS Science Learning

• Jeffrey Snowden, BSCS Science Learning

• Molly A.M. Stuhlsatz, BSCS Science Learning

• Brian M. Donovan, University of Colorado - Boulder

• Cynthia M. Passmore, University of California - Davis

• Patricia Olson, BSCS Science Learning

• Christopher D. Wilson, Biological Sciences Curriculum Study