The Assessments Underlying ASSISTments

• In Event: Scaffolding Self-Regulation, Co-Regulation, and Socially Shared Regulation of Future Learning: Affordances of Learning Analytics Dashboards

Abstract

ASSISTments is an online learning platform that hundreds of teachers use to assign classwork and homework to more than 50,000 student users around the world. With a primary focus in mathematics, the platform has over 25,000 problems mapped to the Common Core State Standards (NGACBP & CCSSO, 2010) and delivered through certified problem sets as a free service of UNIVERSITY. The platform provides students with instructional ASSISTance while serving as a powerful assessMENT tool for teachers (Authors, Date).
In a recent efficacy trial, SRI International found that use of ASSISTments for homework over the course of an academic year resulted in large, reliable learning gains on standardized assessments (Rochelle, et al., 2016). One hypothesis suggested that gains were caused in part by changes in classroom homework review protocol brought about by the ASSISTments reporting infrastructure. Surveys revealed that teachers knowingly shifted their homework reviews toward targeted problems, using the same amount of time to cover fewer, more troublesome problems in greater depth (Rochelle, et al., 2016). This transformation seemingly allowed teachers to act on student performance data by addressing common misconceptions and promoting more powerful learning opportunities. Preliminary results of an analysis of 22 randomized controlled trials (Authors, Date) testing learning interventions within homework assignments embedded within the larger efficacy trial suggest that teachers’ report usage improved homework completion within certain skill contexts. An effectiveness trial is currently underway to replicate and scale the results of the efficacy trial, with additional measures in place to better understand how teacher report usage influences instructional adaption and differentiation. Data from the first year of this trial will be presented in greater depth during the session.
One of the most commonly used teacher reports, the Item Report highlights how student data can provide teachers with powerful insight for formative assessment. Figure 1 pinpoints the report’s aggregated, class-level data that assists teachers in determining which problems to target during review. For instance, only 56% of students answered the fourth problem accurately, with 83% of wrong answers sharing the response “12:28.” Figure 2 shows how teachers can also visualize individual students’ progress within the same report. For each problem a student solves, the teacher is able to quickly visualize her answer, its accuracy, and whether she required multiple attempts, system feedback, or the system-provided answer. Figure 3 displays how teachers can quickly drill down into a student’s performance to access any work that she has uploaded in support of her solution. Past research supports the hypothesis that students learn significantly more when teachers implement this type of data-infused homework review (Authors, Date). Other report formats within ASSISTments aggregate student performance data within mastery-based learning assignments and track gains during pre-requisite skill testing.
The ASSISTments reporting infrastructure allows teachers to leverage student data to focus their instruction time and inform their teaching practices, thereby improving classroom outcomes. Future work within ASSISTments will continue to evaluate how the use of these reports may influence student learning.

Authors

• Neil T. Heffernan, Worcester Polytechnic Institute

• Cristina Heffernan, Worcester Polytechnic Institute

• Korinn S Ostrow, Worcester Polytechnic Institute