Design Features Supporting Teachers' Use of a Dashboard for Diagnostic Assessment Results

• In Event: 31.045 - Supporting Teacher Action Based on Diagnostic Assessment Results

Abstract

Effective score reports are “clear, concise, and visually attractive; also should include easy-to-read text that supports and improves the interpretation of charts and tables; … statistical jargon should be avoided; data should be grouped in meaningful ways” (Goodman & Hambleton, 2004, p. 64). These recommendations have historically been applied to static (i.e., paper or PDF) score reports. With the availability of digital reporting systems, we have the opportunity to provide teachers with dynamic dashboards that allow for interaction with student diagnostic data and better of support instructional decision-making. Nonetheless, such formative use of student data remains a challenging task for teachers. Supporting them effectively requires us to move beyond one-size-fits-all solutions in favor of flexible dashboard designs that accommodate differences how teachers view, navigate, and leverage data for instructional purposes.
This presentation describes a rigorous process for designing a dashboard that promotes interpretability and usability of diagnostic science assessment results at an individual and classroom level. This iterative design process focused on application of universal design for learning (UDL) principles by seeking ways to provide multiple means for engagement, representation, and action & expression (CAST, 2018), as well as scaffold them on more difficult processes, such as interpreting learning maps. Researchers enlisted support from a cadre of thirteen educators across five states to iteratively design and evaluate a prototype dashboard, initially designed through a separate set of needs analysis and requirements gathering focus groups. The cadre participated in four virtual design team meetings roughly one month apart. During each design team meeting, the current prototype was demoed (participants also had access to the prototypes in the days before each meeting), with participants voicing likes, dislikes, concerns, and ideas in a semi-structured, discussion-oriented fashion. Redesigns were based on data including thematic analysis of open-ended feedback and quantitative evidence of navigability of the final prototype. The final dashboard design, which will undergo pilot testing in classroom settings with actual student data, provides teachers with a flexible environment for interpreting data and includes graphic displays of skill mastery in a learning map format as well as tabular information and links to additional resources.

Authors

• Robert P. Dolan, Diverse Learners Consulting

• Cara Wojcik, CAST

• Emma Starr, CAST

• Kim Ducharme, CAST, Inc.

• Jose Blackorby, CAST