Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Search Tips
Annual Meeting Housing and Travel
Personal Schedule
Sign In
X (Twitter)
The presentation will focus on the initial efforts of the research project entitled Diagnostic Inventories of Cognition in Education (DICE) to design and refine assessment questions. The project is funded by the Institute of Education Sciences and will develop a freely-available, web-based formative assessment system that provides teachers with valid, timely, and actionable feedback about middle school students’ understanding of probability and chance (“probabilistic reasoning”). The DICE system will comprise two critical components: (1) a web-based assessment system that adaptively administers diagnostic items and uses innovative psychometric methodology to provide feedback that is immediate, reliable, and multidimensional, and (2) feedback reports with interpretative guides to support the classroom use of the results. To develop these components, the interdisciplinary research team will engage in multiple related studies to collect feedback and validity evidence about the assessment system and feedback reports, including cognitive labs, eye-tracking studies, teacher focus groups, large-scale administrations, and small scale experimental intervention studies.
This presentation will provide an overview of the overarching goal of the project to coordinate learning theories and innovative psychometrics to design a concept inventory that is diagnostic. A concept inventory is a test created specifically to identify examinees who exhibit robust misconceptions when reasoning. We define a misconception as a conception that is incongruous with expert or scientific understanding. By diagnostic, we mean the assessment results will provide model-based probabilistic categorizations of students according to the presence and absence of misconceptions by using a blend of item response theory and diagnostic classification modeling.
A key component of a concept inventory is that items are designed where incorrect options are systematically created to correspond to answers produced by reasoning with a specific misconception. After we present a brief overview of the project, we will focus on the designs of questions we created to elicit misconception-based reasoning. We will discuss the opportunities and challenges in designing questions to target multidimensional constructs and present results from student think-alouds to demonstrate key points we have learned and specific hurdles we have faced.
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141–158.
Laine Bradshaw, University of Georgia - Athens
Lisa Famularo, Research Matters, LLC
Hollylynne Stohl Lee, North Carolina State University
Jessica Masters, Research Matters