Learning Analytics as a Tool for Understanding Clinical Reasoning

• In Event: 51.024 - Educational Data Mining: Affordances and Constraints in the Analysis of Behavior and Performance in STEM

Abstract

Purpose
In recent years, there has been an increasing interest in the application of learning analytics to the field of education research due to an increase in the quantity of data collected in learning environments, advances in computational capabilities, and increasingly widespread availability of tools for data mining (Siemens & Baker, 2012). Although educational data mining techniques add value to our research, we need to be cognisant that such analyses be guided by models that represent underlying theoretical constructs grounded in learning sciences research so that the data can be interpreted in an accurate manner (Winne & Baker, 2013).
A goal of this symposium is to discuss the affordances and limitations of data mining techniques. The applications of these techniques are examined in the context of modeling and analyzing clinical reasoning with adaptive instructional systems designed to support novice physicians (e.g., BioWorld, MedU). In particular, the insights gained in models of expertise and self-regulated learning in the medical domain pertaining to performing different tasks, such as diagnosing patient diseases (see Lajoie & Poitras, 2014; Poitras, Lajoie, Doleck, & Jarrell, 2016).

Methods
Our investigation into learner modeling in these different systems has shown the benefits of grounding data-driven models with knowledge engineering approaches to extract information from human experts and improve data interpretability. For example, we have developed novice-expert overlay and help-seeking models to individualize the delivery of feedback and hints. Learner behavioural detection models trained on the basis of mining data enable these systems to deliver feedback that highlights areas of similarities and differences with the expert solution path, allowing novices to reflect on their own problem-solving (Lajoie & Poitras, 2014).
The most significant outcome of mining the diagnostic processes of novice physicians has been to determine common misconceptions and errors based on the linguistic features that characterize case summaries written by novices and experts (Doleck, Basnet, Poitras, & Lajoie, 2015; Poitras, Doleck, & Lajoie, 2017; Poitras et al., 2016), case annotations written by novices (Poitras, Naismith, Doleck, & Lajoie, 2016), and lab-tests ordered by novice learners (Poitras et al.,2016). By establishing diagnostic performance metrics that characterize learners’ problem-solving along several dimensions (efficiency, efficacy, affective, and hybrid/sequential metrics; Doleck, Jarrell, Poitras, Chaouachi, & Lajoie, 2016; Jang, Lajoie, Wagner, Xu, Poitras, & Naismith, 2016), it is possible to discover novel insights into reasoning processes (Doleck, Poitras, & Lajoie, 2017; Poitras, Doleck, & Lajoie, 2018) and discourse patterns (Lajoie et al., 2015) that challenge assumptions underlying the design of learning environments.

Results & Significance
Based on the review, we derive several recommendations for advancing data analytics in the field of education. Namely, the importance of interdisciplinary collaborations that leverage shared infrastructure and open data repositories, accessibility to software and hardware, and graduate student training in data mining tools and techniques in learning sciences programs.

Authors

• Susanne P. Lajoie, McGill University

• Eric G. Poitras, University of Utah