Exploring the Relationship Between Skipping Questions on School Assessment Exams and Future Educational Outcomes

• In Event: The Michigan Consortium for Education Research: A Research and Data Policy Partnership

Abstract

Objective: This paper proposes an objective and non-expensive proxy for students’ soft skills directly derived from test taking behavior, and studies the predictive power of this measure for future educational outcomes. The measure proposed here captures soft skills such as persistence with difficult questions, focus or attention to instructions, intrinsic motivation with learning and/or disengagement with school or examinations.

Perspective and Theoretical Framework: During the last years, economists, educators and policy-makers have become increasingly interested in the relationship between soft skills such as temperament, personality, persistence and motivation, and student performance. (Almlund, Duckworth, Heckman, and Kautz, 2011; Baron and Cobb-Clark, 2010; Heckman, Stixrud and Urzua, 2006; Cunha, Heckman and Schenach, 2010). Understanding the nature of these skills and identifying educational interventions that could boost them, however, has been restricted by the scarcity of objective and inexpensive soft skills’ measures.

Methods: In this paper we examine the incidence of skipping questions on a low-stakes standardized exam in middle school that has no penalties for guessing, and gives students as much time as they need to answer. After controlling for cognitive skills as measured by previous test scores and other student characteristics, we examine how predictive this measure is of future education outcomes including high school performance and attainment, and college enrollment and choice.

Data: We use administrative data from the Department of Education of the state of Michigan. These data include student-level records for all students enrolled in Michigan public schools, including scores on state standardized tests, as well as student characteristics. Data have also been matched to records from the National Student Clearinghouse which records college going of students in the US.

Results: We find that, conditional on previous test scores, the incidence of skipping questions in the 7th and 8th grade consistently predicts education outcomes in high school such as grade repetition, high school drop-out, on-time graduation, as well as higher education choices, such as enrollment in a 4-year college. These results are robust to the definition of skipping incidence or to the measurement of cognitive ability as captured by test scores.

Significance: This evidence suggests that the incidence of skipping questions on low-stakes standardized tests serves as a useful proxy for soft skills that are not related to intelligence or academic knowledge. The incidence of question skipping is an objective and non-expensive measure of non-academic skills that overcomes the weaknesses of many previously used measures such as self-report data, or other subjective evaluations. This paper presents a new measurement tool with the potential to be used in other states or systems that maintain comprehensive education administrative databases and whose standardized tests have no penalties for guessing, and no binding time constraints. Moreover, it could also help to identify interventions that boost student’s soft skills, as well as the mechanisms through which some interventions have effects on student achievement and behavior.

Authors

• Monica Hernandez, University of Michigan - Ann Arbor

• Jonathan Hershaff, University of Michigan - Ann Arbor