Can Behavioral Tools Improve Online Student Outcomes? Experimental Evidence From a Massive Open Online Course

• In Event: 63.018 - Promising Scholarship in Education: Dissertation Fellows and Their Research

Abstract

Online education is growing rapidly. More than 80% of traditional colleges and universities offer online courses and millions of students have enrolled in new forms of online offerings, such as Massive Open Online Courses (MOOCs). While the potential benefits of online education are significant, students enrolled in online courses often struggle. For example, fewer than 20% of students at the largest online bachelors degree institution in the United States complete the degrees they begin and completion rates in MOOCs are often lower than 10%. If behavioral problems impact course completion and online student performance, identifying and addressing these problems could have large benefits. In this study, I leverage insights from behavioral economics to design scalable computer-based tools and evaluate their impact in a Massive Open Online Course.
To identify and address behavioral issues that could impact online student outcomes, I design three software tools intended to address time-inconsistent preferences, limited memory, and willpower depletion. These tools include (1) a commitment device which allows students to pre-commit to daily time limits on distracting Internet activities, (2) a reminder tool which is displayed on student screens after each half hour of accrued distracted browsing, and (3) a focusing tool which prompts students to block distracting websites upon going to the course site. I pilot these tools and a control version of the treatment software in a large university’s online certificate program and then conduct a randomized controlled trial in a statistics MOOC hosted by Stanford University.
The results of this study indicate that the commitment device treatment significantly improved student outcomes. In my preferred estimates, I find that the commitment device treatment increased course completion by 40%, improved total course performance by 0.29 standard deviations, and increased the amount of time students spent on the course website by 24% relative to the control. Contrastingly, I find that the reminder and focus study treatment did not have a significant impact on course outcomes. These results suggest that time-inconsistent preferences play a significant role in MOOC performance, and that scalable interventions designed to address time-inconsistent preferences have the potential to have a significant impact on online student performance.

Author

• Richard Patterson, Cornell University