Updating the Teacher User Interface of the Multiple-Choice Online Causal Comprehension Assessment (MOCCA)

• In Event: Updates on Multiple-Choice Online Causal Comprehension Assessment (MOCCA), MOCCA-CAT, and MOCCA-College: A Diagnostic Reading Comprehension Assessment for Struggling Comprehenders

Abstract

The purpose of this paper is to describe efforts to revise the teacher-facing user interface for MOCCA, an online reading comprehension assessment for students in grades 3-6. Early research on MOCCA focused, first, on the content of the assessment, and second, on the assessment experience for students, resulting in a fully functional, responsive system that works well on desktop computers, laptops and tablets. However, the system was not originally designed for optimal use by teachers. As a result, the UI underwent significant revision to improve the intuitiveness and utility of the system and its instructional recommendations for teachers, as well as to provide streamlined tools for uploading and downloading data. Specifically, we developed a new user dashboard that summarized key information and refined the more detailed rostering and reporting functions. We piloted the revisions with a panel of current and former MOCCA administrative and teacher users serving Grade 3 to 6 and college students. Following common user experience research approaches, users met individually with the research team and completed think alouds while carrying out a series of tasks in the MOCCA system, including rostering, administering the assessment, and viewing results. Panelist feedback was extremely positive and enthusiastic towards the updated interface. Nonetheless, a small-scale randomized control trial evaluation showed few differences between the old and new systems in terms of usability, usefulness, and feasibility. In fact, the only significant difference was that teachers rated the old system as more feasible to use (g = 1.41). Despite this unexpected difference, users of both the old and new systems indicated a preference for MOCCA over other similar reading comprehension assessments. Implications regarding what makes MOCCA and similar assessment systems useable and feasible from teachers’ perspectives will be discussed.

Authors

• Gina Biancarosa, University of Oregon
  Presenting Author

• Sarah E. Carlson, Georgia State University
  Non-Presenting Author

• Benjamin Seipel, California State University - Chico
  Non-Presenting Author

• Mark L. Davison, University of Minnesota
  Non-Presenting Author