Educational Data Mining Proof Cases

• In Event: 77.028 - Can Learning Analytics Improve Learning?

Abstract

From the start, research in EDM and learning analytics has been concerned with finding ways to enhance education. New constructs influencing learning and engagement have been discovered (cf. Baker, Corbett, & Koedinger, 2004; Rowe et al., 2009), and models that can accurately infer ill-defined constructs such as robust learning (Baker, Gowda, & Corbett, 2011), affect (D’Mello et al., 2008), and collaboration (Perera et al., 2009) have been developed. The direct impact on K-12 education has been more limited thus far, in part because it takes several years to discover a phenomenon, model it in a valid and generalizable way, integrate the models into learning systems, design and test interventions that leverage the models, and to then deploy those interventions at scale – this will be shown in both of the examples given below. These stages mirror the entire educational design process, and rival the development of integrated learning software in terms of complexity.
In recent years, educational data mining (EDM) and learning analytics (LA) methods have matured into approaches that can be used to analyze and enhance learning at both the K-12 and post-secondary levels. This presentation will review a pair of "proof cases," examples of the potential of EDM methods to improve education, which demonstrate the value of EDM in a concrete fashion. In the first proof case, I will discuss work to automatically assess student mastery and the structure of student knowledge in a domain, towards providing students with the right learning materials at the right time during online learning. The second proof case covers work to automatically assess student engagement and affect, citing examples that indicate that engagement and affect during online learning are associated with substantial differences in long-term outcomes (including college readiness and attendance), and that automated models can be integrated into online learning software to provide information to teachers in real-time. These examples show that EDM/LA can be used at scale to improve the degree to which online learning is personalized to learners’ individual differences.
These cases demonstrate not only the the potential of the methods used, but also the extensive work necessary to take an EDM finding from initial data to eventual large-scale impact. It takes considerable effort to discover a phenomenon, model it in a valid and generalizable way, integrate the models into learning systems, design and test interventions that leverage the models, and to then deploy those interventions at scale. In this way, modern research-based educational design is alike in some respects to the pharmaceutical industry, where hundreds of compounds must be tested, within a set of increasingly complex procedures, to find one that is commercially viable. Even with these challenges in mind, the prospects for broad impact of learning analytics and EDM are promising.

Author

• Ryan Shaun Baker, University of Pennsylvania