Measures of Numeric Relational Reasoning Across the Grades: Evidence of Response Processes

• In Event: Igniting Discussions About Measures for K–12 Mathematics Education Contexts

Abstract

Numeric relational reasoning, the ability to analyze relationships between numbers or expressions (Baroody, Purpura, Eiland, Reid, & Paliwal, 2016; Farrington-Flint, Canobi, Wood, & Faulkner, 2007; Jacobs, Franke, Carpenter, Levi, & Battey, 2007), supports number sense and algebraic reasoning and is highly predictive of mathematics achievement (Aunio & Niemivirta, 2010; Nunes et al., 2007; Nunes, Bryant, Barros, & Sylva, 2012). The three components used when reasoning relationally (i.e., relations, composition & decomposition, properties of operations) begin to develop in early childhood and continue to develop well into middle school. In this paper, we will present validity evidence collected in two studies for two assessments focused on numeric relational reasoning, the Algebra Readiness Progress Monitoring (ARPM) system (Author_B, 2015) for Grades 6-8 and the Measures of Mathematical Reasoning Skills (MMaRS) system for Grades K-2. For this presentation, we will highlight the use of evidence based on response processes in each assessment’s Interpretation-Use Arguments (Kane, 2013). Evidence based on response processes is theoretical or empirical data that links the construct with the observed responding behaviors of the examinee (AERA, APA, & NCME, 2014).

Study 1: To collect validity evidence for the ARPM system, we conducted a content review with subject matter experts and a pilot test with 39-59 students per grade level per subtest (Quantity Discrimination, Number Properties, and Proportional Reasoning). We used Rasch modeling to estimate item difficulty parameters and examined response latency. The items were developed using a systematically developed template to assess numeric relational reasoning. We found a moderate level of evidence to support the claims that (1) the items designed using the template elicited similar reasoning strategies, and (2) that the items could be solved using numeric relational reasoning.

Study 2: To collect validity evidence based on response processes for the MMaRS system, we conducted a total of 64 cognitive interviews and 64 think-aloud interviews across the three progressions within the MMaRS system (i.e., Relations, Composition & Decomposition, and Properties of Operations). We used the response processes demonstrated by students in these interviews to refine the construct and the item models used for the assessment. We found that certain item features supported students’ use of numeric relational reasoning, while other item features dissuaded students from using numeric relational reasoning. This evidence will be used to strengthen the claim that the items in the MMaRS system assess the intended construct.

While numeric relational reasoning is a critical construct for mathematics, few assessments include numeric relational reasoning items or focus on this construct (Author_B, 2016). Therefore, these assessments and studies contribute to the field by providing validity evidence for two new assessments in K-2 and 6-8 that assess numeric relational reasoning. This paper also contributes to the field by highlighting multiple ways to collect and use validity evidence based on response processes to strengthen claims on an Interpretation-Use Argument.

Authors

• Lindsey Perry, Southern Methodist University

• Leanne R. Ketterlin-Geller, Southern Methodist University

• Cassandra Hatfield, Southern Methodist University