Poster #53 - Initial Variability in Arithmetic Strategies Predicts Future Accuracy in a Teaching Experiment

• In Event: Poster Session 14

Abstract

Strategy variability emerges from rich conceptual knowledge about numbers (Ebby et al., 2020), predicts later arithmetic learning (Carr & Taasoobshirazi, 2017), and is advantageous when encountering novel problems (Alibali, 1999). Typically, variability is computed by counting the number of unique observed strategies (Siegler, 1996).
Yet, this intuitive metric fails to capture the empirically established notion that individuals know and use multiple strategies and can choose among alternatives at any given moment (Siegler, 2007). To better reflect the nuance of moment-to-moment strategy selection, we propose an entropy-based metric to represent arithmetic strategy variability per problem-solving attempt, which allows for existence of multiple competing strategies.
RQ1: How do the model-derived variability metrics from students’ early instructional sessions predict arithmetic accuracy in later instructional sessions?
RQ2: How do the model-derived variability metrics compare with traditional metrics?
Methods
The sample consists of 40 students (19 girls) from an urban school district who participated in a teaching experiment designed to strengthen arithmetic problem-solving competences through at least 11 instructional sessions. Sessions were video-recorded and six categories of information were coded (Table 1) for each attempt to solve a story problem. Inter-rater reliability (0.86 overall agreement) between four coders was established on 13% of the total number of instructional sessions.
Accuracy: 1 = Correct and 0 = Incorrect, per problem-solving attempt.
Strategy Sophistication is a 9-level ordinal metric that ranks problem-solving strategies (Authors, 2021) based on the Learning Trajectories (Clements & Sarama, 2021).
Strategy Variability: number of unique strategies observed (Siegler, 1996) during an instructional session and problem-type.
Proposed Entropy-based Metric. Entropy (Shannon, 1954) is a measure of “information” or “uncertainty” in a set of k probabilities using the formula: -∑_k▒〖p_k×log⁡(p_k)〗. Multi-level sequential logit models (Authors, 2022) were used for strategy sophistication and were also used to generate the likelihood of the 9 strategies being employed per attempt for each student (p1,…,p9). Thus, high entropy reflected variable or uncertain set of probabilities and vice-versa.
Analytic Approach. “Initial” variability was computed using the entropy-based metric for each student during sessions 1-7. “Future” accuracy was computed for each student during sessions 8-16. Generalized Additive Models (Wood, 2017) were used to compute associations between initial variability and future accuracy, accounting for clustering by student and GRA.
Results
RQ1: Higher initial entropy-based variability predicts greater future accuracy, even when adjusted for problem-type. RQ2: In contrast, the number of initial unique strategies is associated with lower future accuracy. See Table 2.
Sensitivity analyses (not shown) reveal no non-linear relationships and no time-varying relationships between initial variability and future accuracy.
Significance
The traditional metric of strategy variability, taken at the students’ initial 8 sessions, has a counter-intuitive negative association with future accuracy. In contrast, the proposed entropy-based variability metric suggests greater initial variability predicts higher future accuracy. The conflicting direction of association implies the two metrics measure different cognitive processes for arithmetic problem-solving strategies.

Authors

• Menglong Cong, University of Denver

• Traci Shizu Kutaka, University of Virginia - Charlottesville
  Presenting Author

• Pavel Chernyavskiy, University of Virginia - Charlottesville

• Kayla McCreadie