The Future of Multiple-Choice Online Causal Comprehension Assessment (MOCCA) and Diagnostic Assessments: Adult Readers, Reading Times, and a Computer Adaptive Version

• In Event: 53.077 - Year 2 Results for the Multiple-Choice Online Causal Comprehension Assessment: Project MOCCA

Abstract

Objective and Theoretical Framework. The importance of reading, coupled with the prevalence of poor reading skills at the postsecondary level, has led many institutions to provide reading support for their students through developmental coursework, reading centers, and tutoring (Tinto, 2012). However, without reliable and valid methods of identifying struggling readers, determining which aspects of reading are challenging for students, and assessing changes in reading comprehension skill, it is impossible for institutions to properly offer services and assess the effectiveness of these services in developing reading skills. Moreover, there is a renewed push for standardization of placement assessments for the purpose of identifying students in need of remediation (Boylan, 2002, 2009), but such practices may be ineffective if these assessments only provide generic information that students struggle without differentiating why students struggle.

Reading is a complex process comprised of many components, and students may struggle with reading for different reasons (e.g., Brasseur-Hock, 2011). Therefore, knowing the reasons students struggle could provide valuable information for targeted services and correct course placement, remediation, or support. In fact, one reason developmental education programs on college campuses are unsuccessful is a lack of understanding for why students struggle with reading (Hughes & Scott-Clayton, 2011). Also, the need for targeted instruction is all the more urgent given the mounting external pressures for aligned literacy instruction for postsecondary education, and increased postsecondary graduation/credentialing rate (e.g., Biden, 2011). Thus, there is a need for new assessments that differentiate the comprehension skills of postsecondary students in regard to comprehension processes, a need for meaningful scores that students understand, and a need for scores that are informative for various postsecondary personnel including classroom instructors who teach those developmental and remedial reading courses.

Methods. One future aim of the MOCCA team is to address the needs of struggling readers in other age groups by refining and validating MOCCA to become a measure of reading comprehension that differentiates reading comprehension skills of postsecondary students. This presentation includes pilot data from a MOCCA administration to college students (N=12) enrolled in non-credit bearing English classes. The presentation also includes think-aloud data from these same students.

Results and Significance. Preliminary results from think-aloud protocols indicate that intermediate-aged students and post-secondary students engage in similar cognitive processes while thinking aloud. Preliminary MOCCA data indicates that the original test items (Flesch-Kincaid Grade 5) used with this adult sample were generally too easy for postsecondary students, but some students did have a propensity to specific errors types – either paraphrasing or lateral connecting. In conjunction, these prelimary findings indicate that developing new MOCCA items with higher readabilities may serve to identify and diagnose (i.e., by determining their error propensity) struggling comprehenders enrolled in postsecondary education. More specifically, a distractor-driven assessment such as MOCCA is only effective when examinees actually make errors.

Additional information will also be presented on other future extensions of MOCCA as well including the use of reading times to calculate comprehension rates and the development of a computer adaptive (CAT) version of MOCCA.

Authors

• Ben Seipel, University of Wisconsin - River Falls

• Sarah Elizabeth Carlson, University of Oregon

• Mark L. Davison, University of Minnesota

• Gina Biancarosa, University of Oregon

• HyeonJin Yoon, University of Nebraska-Lincoln