A General Ordinal Diagnostic Model for Item- and Category-Level Structural Inferences Using a Pólya-Gamma Data Augmentation Strategy (Poster 11)

• In Event: Promising Scholarship in Education Research: Dissertation Fellows and Their Research

Abstract

International assessments such as the Trends in International Mathematics and Science Study (TIMSS) have been implemented on a global scale to measure student achievement within critical content domains. Cognitive diagnostic models (CDMs) are valuable psychometric tools for inferring diagnostic information about student skill achievement provided large-scale assessment data. Specifically, CDMs use item response data to classify examinees into diagnostic classes, each characterized by a distinct pattern of possession or mastery on the set of discrete latent attributes conceptualized to govern responses to a set of assessment items. The inclusion of polytomous-scored item formats, such as polytomous constructed-response items, in large scale assessment settings allows for the specification of multiple internal tasks within-item, each with a (potentially) distinct response process. To accommodate more flexible item formats, we introduce a novel CDM framework which partitions item effect and structural parameters into item-level and category-level parameters. We show that the proposed model framework can identify differences in response processes (when present) and provide accurate recovery of person, class, and item parameters. In addition, we show trends in recovery as sample size and structural attributes change across study conditions. Following which, we fit the model to the 2019 TIMSS 8th grade United States sample data measuring mathematics achievement in multiple content domains (algebra, geometry, etc.). We report fit statistics for model structures, person and class diagnostic information, and item and structural characteristics. Lastly, we discuss the estimated group-level outcomes and their impact on informing future directions in educational policy.

Author

• Auburn Jimenez, University of Illinois at Urbana-Champaign
  Presenting Author