Analysis of an Instructor Dashboard to Support Learning and Facilitation in an Online Problem-Based Learning Environment

• In Event: 41.025 - Learning, Engagement, and Assessment in Technology-Rich Learning Environments

Abstract

Objectives: Problem-based learning (PBL) is an instructional method in which students learn through solving problems (Hmelo-Silver & Barrows, 2006). A key feature of PBL is the role of the teacher as a facilitator of small group learning. One way to scale PBL is to move it online. This creates, facilitation challenges for scaffolding multiple groups (Martinez-Maldonado, et al., 2012). The data created by online learning may help deal with these challenges. Learning analytics (LA) is an approach to collect, analyze and report student-produced online data and present it as data visualizations, e.g., informative and dynamic graphics (Gašević, Dawson, & Siemens, 2015). The use of LA in online PBL may help expand instructional capacity and support instructional decision-making. This study aims to understand how theoretically-guided LA visualizations were used by facilitators in an asynchronous PBL environment Helping Others with Argumentation and Reasoning Dashboard (HOWARD) (see Figure 1).
Design: HOWARD provides two levels of support: a student interface to foster collaborative problem solving, shown in Figure 1 and an instructor dashboard that presents information about students’ activity be it individual participation or overall group progress, along with pertinent PLB activity as shown in Figure 2 (Kazemitabar et.al., 2016).
Methods and Analysis: Participant demographics are presented in Table 1. Participants contributed to a two weeks PBL unit about Breaking Bad News, where they viewed videos to see how other physicians broke bad news to patients across two different cultural settings. Instructors were interviewed after completing the course to investigate how they understand the LA tools, what kind of information they found helpful, challenges they faced as well as their future expectation of such tools. Interviews were transcribed and coded based on each sentence and categorized into 11 emerging themes. Log data and students interviews were used to triangulate our findings.
Results: Instructors valued the visible output of student data via the whiteboard and threaded discourse that helped them understand student participation, however they paid little attention to student’s activities regarding resource usage or reading other student posts. Instructors felt that the visualizations communicated more on overall student activity participation and less on individual student learning progression. For example, instructors had a good understanding of participants’ relationships in the social network analysis (SNA) chart but encountered difficulties in predicting interaction patterns from the links. Instructors could not use the SNA to identify appropriate times to scaffold students having difficulties. The nature of asynchronous online environments created time intervals between posts and the time delays to produce data sometimes led to fragmented understanding of student conversations.
Significance and Future Direction: The data helped us build and informed understanding of how instructors use visualizations to make instructional decisions. This data will lead to better design decisions about dashboards that will allow facilitators to support multiple PBL groups. Future directions will consider how to best optimize the dashboard to reduce cognitive load while giving facilitators the information that they need to support many groups.

Authors

• Yuxin Chen, Indiana University

• Peter Hogaboam, Indiana University

• Maedeh Assadat Kazemitabar, McGill University

• Stephen Bodnar

• Juan Pablo Sarmiento, New York University

• Ruth K Sherman, New York University

• Cindy E. Hmelo-Silver, Indiana University

• Susanne P. Lajoie, McGill University

• Ricki Goldman, New York University