The Effects of Coaching on Candidate Performance in Digitally Mediated Simulations

• In Event: Approximating Teaching: Using Simulations as a Platform for Understanding and Improving Instructional Quality

Abstract

Objective & Framework
Simulations have the potential to create powerful learning opportunities, but technology alone will not support teacher development. As with any pedagogical tool, the supports that surround simulations may facilitate or impede the effectiveness of the sessions. The goal of this study was to analyze the effects of coaching on candidates’ teaching in the simulator. We focus on a single teaching practice, the quality of feedback TCs provide during a text-based discussion.

Our study was informed by scholarship in practice-based teacher education that suggests candidates need multiple opportunities to practice teaching skills and benefit from support and guidance from expert teacher educators (Ghousseini & Herbst, 2016; Lampert et al., 2013). The professional development literature is coalescing around the idea that coaching is the most effective way to improve teacher practice quickly, but there is comparatively little work on the effects of coaching in the pre-service space (Joyce & Showers, 2002; Kraft, Blazar, & Hogan, 2016).

Methods & Data Sources
Participants included all candidates at a large public university (n=95), where simulations sessions are integrated throughout the two-year program. Candidates are largely white and female, which reflects the demographics of TCs in the state.

Candidates were randomly assigned in course section blocks to either: 1) self-reflect on simulation performance without coaching, or 2) receive coaching between simulations by a trained coach who followed a structured coaching protocol. Groups were balanced on all 16 covariates.

Simulation sessions were coded on measures assessing the quality, quantity, and type of feedback candidates provided. Raters were blind to treatment and simulation session timing (e.g., before or after coaching intervention). Multiple raters scored 20% of videos with high levels of inter-rater agreement (>85%). We examine treatment effects on the feedback candidates provided after the coaching treatment.

Results
Table 1 shows the effects of coaching on the quality and quantity of TC feedback. Coached TCs improved on a measure of overall quality by more than a standard deviation (SD), and improved the proportion of successful responses to students by .95 SDs (p<0.01). Coached TCs also provided significantly more text-based feedback (0.78 SDs, p<0.01) and high-quality, descriptive feedback (p<0.05) and significantly less low-quality, perfunctory feedback than those in the self-reflection group (p<.05). Perfunctory feedback included comments such as “good work,” and “nice job.” Descriptive feedback included comments like “When we are working with texts, it’s also important to note places in the text that support your inference.”

Significance
Simulations provide a space to approximate teaching practices, like providing feedback during text-based discussions, and they also afford a platform for assessing the effects of different kinds of supports on the improvement of teaching. Our data suggest multiple practice opportunities alone are unlikely to promote high-quality feedback. Candidates who received coaching demonstrated higher-quality feedback than those who only self-reflected between simulation sessions. This suggests that if simulations are isolated activities without scaffolding from skilled coaches or teacher educators, they are unlikely to realize their potential as a pedagogy of teacher preparation (Grossman, Hammerness, and McDonald, 2009).

Authors

• Julie Jackson Cohen, University of Virginia

• Emily Wiseman, University of Virginia