Assessing Bootstrap: Algebra Students on Scaffolded and Unscaffolded Word Problems

• In Event: Power in the Plurality of Pedagogies and Partnerships in "Computer Science for All"

Abstract

OBJECTIVE
Bootstrap:Algebra (“BS:A”) integrates computing into standard math classes, using a functional language called Racket that reinforces---rather than undermines---the semantics of mathematics, and a highly-accessible programming tool (recent work has improved this tool for visually impaired students). Such integration is appealing for several logistical and intellectual reasons, but given the known challenges of transferring skills between disciplines (Bransford and Schwartz, 1999; Perkins, 2009), teachers and administrators are right to demand proof. Our curriculum includes a pedagogical scaffold called the Design Recipe that explicitly bridges the gap between computing and algebra, giving students a structure that applies in both contexts with the goal of improving performance through mutual reinforcement.

THEORETICAL FRAMEWORK
The topics of multiple representations and structured problem solving appear in state and national standards (NCTM, 2016). BS:A explicitly teaches both via the Design Recipe. Given a word problem, students are taught to (1) articulate the domain and range of the problem, (2) write at least two examples of the input/output relationship described in the problem, and (3) write the symbolic form of the function that models the word problem The curricular materials explain how to leverage each step when attempting the next one, thus modeling ways in which representations depend on one another. By studying the degree to which students were able to apply the Design Recipe to standard pencil-and-paper word problems, we obtained a measure for how effective this innovation is in improving math outcomes.

METHODS AND DATA
Students were given pre/post-tests involving standard, pencil-and-paper word problems based on various standardized tests. All of the problems on the pre-test and half of the problems on the post-test explicitly asked for the scaffolded representations present in the Design Recipe. This allowed us to check whether the same students perform differently with or without the scaffold. We received a total of 468 matched pre- and post-tests from 22 teachers across multiple states, representing several school types and district level demographics. To analyze the differences between pre- and post-tests, we used a mix of two-tailed t-tests and IRT.

FINDINGS
Looking at raw assessment scores using a paired t-test, the gains between pre- and post-test scores are both positive highly significant (p < 0.001, effect size of 0.888 by Cohen’s d). When analyzing incorrect answers, we found that students made fewer serious mistakes in the post-tests: even when wrong, they were wrong for less glaring reasons. We also found fewer questions left blank in the post-test, suggesting either greater confidence or an ability to start more problems in the same length of time after the intervention.

SIGNIFICANCE
Evidence of positive transfer between computing and math is significant for administrators and teachers looking to harmoniously embed computing content in mathematics. For other computing interventions that seek to integrate with mainstream subjects, this points to the need to deeply align concepts in ways that take the learning goals of the host discipline seriously.

Authors

• Emmanuel T. Schanzer, Harvard University

• Kathi Fisler, Brown University

• Shriram Krishnamurthi, Brown University