Comparing the Practical and Theoretical Implications of Different Methods of Empirical Recovery of Learning Trajectories

• In Event: Tracing Learning Progressions/Trajectories: Novel Theoretical and Methodological Perspectives at the Intersection of Mathematics and Science

Abstract

Empirical recovery, a means of validating measures of LT/LPs, offers insight into whether data from student learning conforms to the theory of the LT (Graf & van Rijn, 2016). Methods of empirical recovery in science (e.g., Authors, 2012; Wilson, 2012) and mathematics (e.g., Carney & Smith, 2017; Graf & van Rijn, 2016; Lehrer, Kim, Ayers, & Wilson, 2014) have been proposed, however there is no specific guidance on which methods should be used, and limited research has discussed the theoretical implications of recovery methods. The choice of model reflects theoretical commitments about how students move through the LT/LP, the purpose of empirical recovery, and the topic and grain size of interest. For instance, some claim that LP in science tend to be of larger grain size than most LT in mathematics (Ellis, Weber, & Lockwood, 2014). Consequently, LP with larger grain-size tend to discover multiple pathways among learners, while smaller grain-size LT delineate a more precise sequence of development. In order to investigate these issues related to empirical recovery model choice, a substantial source of data on a variety of LT/LPs is needed to examine the different patterns of empirical recovery as they are applied to a variety of topics, both applied and conceptual.
Math-Mapper (MM) comprises a diagnostic assessment system that addresses nine big ideas in middle school mathematics, ranging from applied topics in measurement, geometry and statistics, to more basic ideas in number and algebra. Over a period of four years, assessment data has been gathered from over 75,000 students. This database provides an ideal opportunity to examine different methods of recovery because it covers the entire mathematics curriculum which encompasses different content and types of levels.
Our presentation identifies five methods of empirical recovery (see Table 1) that have been proposed in science and mathematics education, delineate the theoretical implications of the different methods, and highlight diverse patterns in cases of the recovery methods for LTs from the applied and basic topics. For example, the empirical recovery results for two content areas in mathematics (Key Ratio and Representing and Linear Functions) highlighted how the models were impacted by differences in grain size and possible sequencing of the LTs. The Wright map diagrams (see Figure 2) show that a Ratio LT following a strict linear ordering of levels more closely than the Linear Function LT, while the learned Bayesian Network diagrams (Figure 3) of the LTs illustrates possible alternative non-linear patterns of levels for the Linear Function LT. This suggests that the Wright map method may be more applicable for LT/LPs or content that has a well-established learning path, while the Bayesian Network method may better compliment LT/LPs and content that could have multiple paths to understanding or intertwined relationships. The choice of empirical recovery method is important because it has consequences for how an LT/LP is evaluated and modified, which should reflect the theoretical commitments of the content area. The study highlights the uses and challenges of different statistical methods of empirical recovery for different content areas.

Authors

• Emily Toutkoushian, The American Board of Anesthesiology

• Jere Confrey, North Carolina State University

• Meetal Shah, LearnPlatform