What a Cluster! Examining Local Item Dependence in Multidimensional Science Assessment.

Session Type: Coordinated Paper Session

Abstract

In assessing multidimensional science standards using items related to phenomena, questions about item clusters are likely to arise. Will we meet the assumption of local item independence (LID) which is fundamental in Item Response Theory? The four papers presented in this session each address cluster-based assessment of multidimensional science standards: a large-scale testing vendor’s findings from the largest operational application of a bifactor model across state summative assessment programs; a university center’s research questioning the traditional method of model comparison to examine testlet effects if the intended purpose is individual measurement rather than parameter estimation; a large scale testing vendor’s analysis of two summative assessments where evidence of LID was not found; and, a research organization’s findings from two unique state’s summative assessments where comparisons between stand-alone items and cluster-based tasks could be made, and no evidence of LID was found. The session discussant worked on the NRC science framework, several state assessments, and the updated framework for NAEP science, and will engage the presenters and audience in lively academic dialogue around item dependency in cluster-based science assessments.

Sub Unit

• Individual Papers, Innovation Demonstrations, Coordinated Sessions, or Organized Discussions

Session Organizer

• Sarah Quesen, WestEd

Individual Presentations

• Calibration Model for Three-Dimensional Science Assessments - Allison Boykin, Cambium Learning Group; Suhwa Han, Cambium Assessment, Inc.; Frank Rijmen, Cambium Assessment

• Examining Local Item Dependence Effects in Testlets: Practical and Statistical Significance - Perman Gochyyev, UNIVERSITY OF CALIFORNIA, BERK; Mark Wilson, Berkeley School of Education, UC BERKELEY

• Exploring LID in Two NGSS Large Scale State Testing Programs - Timothy O'Neil, Pearson

• Evaluating Item Clusters in Multidimensional Science Assessment - Sarah Quesen, WestEd; Marianne Perie, WESTED; Daniel Murphy, WESTED

Discussant

• James W Pellegrino