A Computational Approach for Assessing Quantitative Aspects of Multiple-Text Integration Using Text Mining Techniques

• In Event: AERA Roundtable Session Thursday 9:45 am - Gold 4
  In Roundtable Session: Innovations and Evidence-Based Perspectives in Literacy Instruction, Learning, and Research

Abstract

This study investigated the feasibility and validity of a computational approach to assess students’ integration of information from multiple text sources. Using entropy-based bias scores derived from cosine similarity metrics, we examined whether computational measures of source use (i.e., quantifying how evenly students incorporate content from multiple texts) align with human ratings of integrative essays and predict reading comprehension performance. Results showed that entropy bias correlated with both human-judged integration quality and comprehension outcomes, supporting its validity as a scalable, interpretable index of integration. This approach offers a promising alternative to traditional, labor-intensive assessments, enabling automated evaluation of students’ integrative reading and writing across large datasets.

Authors

• SeongYeup Kim, Pennsylvania State University

• Rayne A. Sperling, Pennsylvania State University