Artificial Intelligence–Based Transcription as a Tool for Enriching Feedback and Debriefing Practices in Teacher Education Simulations

• In Event: 45.050 - Virtual Reality Simulations as Approximations of Practice: New Ways to Support Teacher Learning

Abstract

Objective and Framework
Facilitated reflections play a crucial role in simulation-based learning to help learners make sense of their experience (Fanning & Gaba, 2007). However, teacher educators may struggle to leverage the complex multimedia data generated for feedback and debriefing. Teacher educators (TEs) typically use two strategies for providing feedback on simulations: (1) viewing full performances from all candidates and providing real-time (Scheeler, Congdon, & Stansbery, 2010) or post-hoc feedback (Benedict, 2016), or (2) reviewing none of the data, and asking candidates to review their own data and bring excerpts to a debrief (Dotger, 2013).

These typical practices represent an “all or nothing” approach, with the middle ground missing because the length and complexity of simulation data make rapid review difficult. This motivated a design-based research project to explore ways to process and present performance data in new forms to allow new kinds of interactions during feedback and debriefing sessions.

Methods and Data Sources
We conducted a multi-cycle design-based research project, working with two TEs from a new teacher preparation program. In Cycle 1, we conducted a user experience interview where we demonstrated the functionality of TeacherMoments--a web-based platform for conducting clinical simulations. In TeacherMoments, candidates view a series of video vignettes from situations teachers might experience in their work, then provide improvised audio responses to particular “verbal triggers” in the simulations (Owho-Ovuakporie, 2017).

For the user experience interview, we showed the TEs a preliminary data visualization dashboard, and asked what kinds of data and representations they would find useful. In Cycle 2, the TEs conducted a short simulation exercise with teacher candidates using the preliminary dashboard and provided feedback. For Cycle 3, we designed a new dashboard (Figure 1) by adding two major new features: AI-based transcription of audio responses and new options for viewing responses by reflection prompt instead of by candidate. Using this new dashboard, the TEs planned and facilitated a feedback and debriefing session with the teacher candidates. Data sources for analysis include observation notes of testing activities, transcribed interviews with the two TEs, and all participant simulation data recorded in TeacherMoments.

Figure 1: New Data Dashboard of TeacherMoments

Results
With the new dashboard, the TEs could easily process different parts of the multimedia resources to understand teacher candidates’ experiences concretely. As a result, the TEs could facilitate and create new learning experiences that would otherwise be impossible without this data representation. During the planning of the debriefing session, they skimmed the transcripts of responses and identified possible discussion topics and additional learning goals that weren't previously defined. Additionally, they used concrete incidents during the group debriefing session and pinpointed specific segments to engage other candidates for the discussion, even when the candidate might have forgotten what her response was.

Significance
This work demonstrates how technology can enrich TEs instructional choices in simulations by rapidly processing performance data. This affordance also enabled different modalities for presenting simplified data to understand how data representations could affect TEs’ instructional decision-making for debriefing and feedback.

Authors

• Justin Fire Reich, Massachusetts Institute of Technology

• Yoon Jeon Kim, Massachusetts Institute of Technology