Enhancing Assessment Rubrics: Unraveling Interdisciplinary Science Understanding Through Topic Modeling Techniques

• In Event: Sunday Roundtable Session 11:25 am
  In Roundtable Session: Assessing Science Understanding and Critical Thinking Competencies (Table 26)

Abstract

We present a novel approach to quantitatively enhance rubrics of constructed response items in science assessment. We analyzed 454 high school and early college students’ written answers from the Interdisciplinary Science Assessment of Carbon Cycling along with their corresponding rubrics, utilizing the supervised latent Dirichlet Allocation (sLDA) model. The results demonstrated a significant alignment, ranging from 51.6% to 78.6%, between rubric keywords and the top 20 words identified from students’ responses. Additionally, the results revealed strong relationships of scores between human raters and sLDA results (R2=.434 to .809). These findings provide valuable insights into leveraging topic modeling techniques for enhancing rubrics, paving the way for more accurate and efficient assessment practices in science education.

Authors

• Minju Hong, Chung-Ang University

• Hyesun You, University of Iowa

• Mandy Duffey, University of Iowa