Bridging Design, Research, and Instruction: Iterative Curriculum Improvement in Statistics Education

• In Event: AER Roundtable Session Friday 1:45 pm - Gold 4
  In Roundtable Session: Reimagining Introductory Statistics: A Research-Practice Partnership Model for Impactful Statistics Education

Abstract

Theoretical Framework
This study combines the “Better Book” approach (Stigler et al., 2020) - a data-driven improvement science framework - with an effort to center students’ experiences in order to enhance an online introductory statistics and data science textbook. By closely examining students’ responses to targeted survey questions embedded within each textbook chapter, we identified barriers and challenges in the curriculum that impacted their motivation, allowing us to redesign those sections using qualitative feedback from both students and instructors. We then examined whether these revisions improved students’ motivational experiences in a subsequent cohort taught by the same instructor.

Methods and Data Sources
Using items embedded in a 12-chapter interactive online textbook, students reported on their motivational experiences (expectancy, intrinsic and utility value, and perceptions of cost), 11 times (once per chapter starting in chapter 2) over a 10 week period. This data identified hot spots in the textbook where students experienced dips in motivation, with dips in expectancy and increases in cost (see Figure 1). Those chapters were then redesigned based on qualitative feedback from both students and instructors. In the redesigned version of the textbook, we analyzed the impact of those changes on the same motivational measures. Our data set thus includes longitudinal data from both the prior version and redesigned version of the textbook.

Results
To examine whether motivational experiences differed between the original (version 4.0) and redesigned (version 5.0) textbook, we conducted 2 (version: original vs. redesigned) × 11 (chapter: 2–12) mixed repeated-measures ANOVAs separately for cost and expectancy (see Figure 2). Students using the redesigned textbook reported significantly lower cost, F(1, 395) = 16.44, p < .001, partial η² = .040, with cost also varying by chapter. Expectancy showed no overall version effect but differed by chapter and displayed a significant version × chapter interaction, F(7.0, 2863.7) = 7.47, p < .001, partial η² = .018, with the redesigned version showing more stable or improved expectancy patterns.

Scientific Significance
In the prior version of the textbook (version 4.0), we identified elevated perceptions of workload cost and fluctuating expectancy beliefs throughout the course, with noticeable drops around chapter 7. To address this, we redesigned the textbook (version 5.0) to provide a more visual, modular structure, splitting longer chapters and creating a more consistent learning experience. Analyses of students’ motivational experiences in the redesigned version revealed meaningful improvements: perceptions of cost decreased overall, and expectancy beliefs followed a more favorable trajectory.
Overall, this research contributes to the broader field of education by demonstrating how a student-centered, data-driven design in the context of introductory statistics can improve students’ experiences. By examining the motivational trajectories of students using different textbook versions, this study highlights how redesigning educational systems and structures (rather than placing the burden on instructors or students themselves) can foster more supportive environments. These findings have important implications for statistics and data science education, suggesting that thoughtful curricular design can help all students thrive, regardless of background.

Authors

• Claudia C. Sutter, CourseKata

• Matthew Jackson

• Adam B. Blake, University of California - Los Angeles

• James W. Stigler, University of California - Los Angeles

• Karen B. Givvin, University of California - Los Angeles