Understanding Inequalities in Academic Engagement Through Behavioral Analytics

• In Event: Equity-Oriented Data Science: Leveraging Computation and Novel Data Sets for Higher Education Research and Practice

Abstract

With the prevalence of digital technologies such as learning management systems (LMS), it becomes possible to capture micro-processes of students’ everyday college experience that were previously difficult to observe at scale (Whitmer et al., 2019). In this study, I leverage institution-wide LMS log data at a Hispanic-Serving Institution and large-scale behavioral analytics to systematically document inequalities in college students’ day-to-day academic engagement, and further use predictive modeling to unveil how these inequalities signal achievement gaps. The findings will extend the understanding of inequalities in higher education to the behavioral level and provide more actionable insights to practitioners that facilitate targeted support and positive policy changes for equitable student success.

For the first part, I focus on whether there are systematic differences in learning behavior across students from different socio-demographic groups. Built upon learning analytics research, I compile a list of established behavioral measures that capture critical cognitive and non-cognitive skills such as self-regulated learning. To generate these measures, computational approaches including automated content analysis and process mining are applied to the behavioral logs and textual content generated by students in LMS. Examples of such measures include analytical thinking in discussion posts and the frequency of accessing learning materials when finishing an assignment. To examine socio-demographic differences, I use MANOVA to statistically compare the averages of behavioral measures across groups defined by gender, race, first-generation status and low-income status. To control for different course contexts, these measures are standardized within courses.

I also scrutinize the predictive power of these behavioral measures for both course-level performance and longer-term outcomes (persistence). An array of cutting-edge machine learning algorithms will be used for the prediction, including both static models (e.g., random forests), which use aggregated behavioral signals as predictors, and sequential models (e.g., recurrent neural networks), which account for the dynamic nature of behavioral traces. Common metrics such as accuracy and F1-score are used to evaluate the prediction performance. On top of its scientific value, this analysis helps identify the modeling architecture that most accurately predicts academic risks at an early stage and can facilitate efficient resources allocation on campus.

Author

• Renzhe Yu, University of California - Irvine