Development and Validation of a Learning Progression for Mathematical Thinking in Science

• In Event: Tracing Learning Progressions/Trajectories: Novel Theoretical and Methodological Perspectives at the Intersection of Mathematics and Science

Abstract

Mathematical Thinking in Science (MTS) is an essential goal of science education (NGSS Lead States, 2013). MTS refers to abstracting relevant measurable variables from ‘messy’ phenomena, identifying mathematical relationships among the variables, and using scientific mechanisms to explain the mathematical relationships. In this presentation, we report on the development and validation of an LP for MTS in high school physical and life sciences. We developed a hypothetical LP for MTS (Authors, 2019) which is based on four achievement levels: Level 1 Intuitive observations—Explaining phenomena in terms of macroscopic observations and/or personal feelings; Level 2 Attributes—Identifying qualitative attributes without quantification, Level 3 Measurability—Identifying measurable variables and simple quantitative relationships, and Level 4 Relational Complexity—Distinguishing among different types of variables (e.g., intensive vs. extensive variables) and identifying complex quantitative relationships among variables (e.g., exponential population growth).
Based on validity literature (AERA, APS, & NCME, 2014; Kane, 2013), we developed a framework to guide the systematic validation of this LP. The five-stage framework in figure 1, is established on an interpretive argument and a validity argument. The interpretative argument contains a chain of inferences (e.g., students’ MTS proficiency inferred from the raw scores), and the validity argument provides an evaluation of these inferences and examines whether eight assumptions are supported by empirical evidence.
Assumption 1 and 2 encompass the development of effective items for diagnosing students’ LP levels. To evaluate these assumptions, we developed a set of items and used them in interviews with 44 high-school students. Based on the data, we identified item features that make them effective for diagnosing students’ levels. These features and the subsequent item revisions are the evidence for Assumption 2. We then assembled the revised items into different test forms and administered them to 5353 students from different racial/ethnic, socio-economic, and English language status. For each item, we developed a scoring rubric to score the responses against the LP levels. We then repeatedly scored responses and revised the scoring rubrics. These revisions are the evidence for Assumption 3—the LP levels capture salient patterns of students’ thinking as it relates to MTS. Next, we performed Item Response Theory (IRT)-based analyses to the scores. The analysis results are presented in Wright Maps, which present student proficiency and item difficulty on the same scale. The Wright Maps provided evidence to support Assumptions 4 and 5: The LP levels are differentiated from each other; the order of the LP levels in the Wright Maps are consistent with the order of the LP. To support assumption 6, we examine the correlation between students’ science perfomence and their MTS proficiency. For assumption 7, we examine whether there is adverse impact in terms of race/ethnicity and gender. Assumption 8 is beyond the scope of this presentation. Our research illustrates how to develop and validate a LP of mathematical thinking in science. We show that the development of this construct is a complex iterative process, that reflects commitment to considerations of equity, fairness and model fit.

Authors

• Hui Jin, Educational Testing Service

• Hyo Jeong Shin, Educational Testing Service

• Dante Cisterna, Educational Testing Service