Developing classroom simulators for teacher education students in Norway: Insights from a meta-analytic study

• In Event: Virtual Worlds, Real Learning: XR and Simulations in Teacher Training

Proposal

Concern with the practical aspects of teacher education is not new – in 1904, John Dewey argued that it should be included alongside theory in teacher education (in Grossman et al., 2009). Since then, making the link between theory and practice has been attempted through a range of approaches, such as micro-teaching (Koross, 2016) reflective practice (Thompson & Pascal, 2012) and case-based approaches (e.g. Şen Akbulut & Hill, 2020). Alongside this trend is the increasingly predominant framing of teaching as a competence-based activity, termed the ‘New Science of Education’ by Furlong and Whitty (2017). In this view, teaching consists of specific skills that can be deconstructed, analysed, and (as a form of teacher development) practised. Grossman et al. use the term approximations of practice to describe these opportunities, where teachers can “rehearse and enact discrete components of complex practice in settings of reduced complexity” (2009, p.283). Typically, this has been achieved through activities such as role-playing with peers, but more recently attention has turned to the use of classroom simulators as a way of providing these opportunities.
Recent reviews of classroom simulators have, with few exceptions, found positive outcomes for classroom simulation experiences in a wide range of research studies. Huang (2023) reported finding 43 studies with positive outcomes out of a total of 46, while Ersozlu et al. (2021) found uniformly positive results in their review of research on the TeachLive classroom simulator (although this review included other contexts such as law enforcement). Other reviews focused on specific aspects of classroom simulator experiences, such as the technology used, for example Atal et al.’s (2023) review of 360-degree videos in teacher education which revealed differing viewpoints on the value of headsets. Similarly, Dieker et al. (2023) describes varying teacher opinions regarding the value of headsets and different types of display.
The common denominator in all the cited reviews is that they do not conduct any quantitative analysis of the research beyond descriptive statistics, apart from Huang et al.’s review (2023), which does report the individual effect sizes of ten studies that calculated them, with a range from .31 to.51. The most common quantitative approach when reviewing literature is meta-analysis, where effect sizes across studies are combined to provide an overall measure. In this context however, I supplement the combined effect size with meta-regression and subgroup analysis. Both approaches allow us to analyse the effects of individual moderators on effect size, with subgroup analysis being used for categorical variables and meta-regression for metric variables (Spineli & Pandis, 2020). Classroom simulators are being developed and used by a range of individual institutions, often at high cost (noted in studies such as Dalgarno et al., 2016; Dittrich et al., 2022; Rosati-Peterson et al., 2021; Shernoff et al., 2020), meaning that evidence of how best to design and use such tools is greatly needed. These led to the design of a meta-analytic study with three primary aims:
- Measure the combined effect size of classroom simulators in teacher education compared to alternative approaches
- Identify any moderators that have significant effects on the outcomes of VR training in teacher education
- Provide suggestions for designing and implementing an effective classroom simulation experience for teachers.
An initial literature search provided 62 studies that were evaluated according to preset criteria, for example the inclusion of sufficient quantitative data to enable effect size calculations. If a study did not meet one of the criteria, it was removed from consideration. Further articles were found using forward and backward referencing from the eligible literature, providing three additional studies. Additionally, three authors were contacted to request further data for effect size calculation. As no responses were received, these studies were excluded. Finally, the search was repeated in January 2024, yielding one additional eligible study. This process gave a final set of 39 studies for meta-analysis.
Primary analysis was conducted using a random-effects model and the combined effect size was reported using Hedge’s g coefficient. The random-effects model assumes heterogeneity between studies (Hansen et al., 2022), and Hedge’s g is used due to the variation in sample sizes (Woon et al., 2021). Virtual reality approaches gave an effect size of .229 when compared to other approaches (95%CI -.660-1.117, z=.504, p=.614).
A second stage of coding recorded pre-specified moderators, as well as additional variables that were revealed in the initial review of articles. This process of identifying other potential moderators in the data and returning to the initial coding is common in meta-analysis (Stanley & Doucouliagos, 2012).
Subgroup analysis was conducted to investigate any differences in effect size estimates in different categories. Following guidelines suggested by Richardson et al. (2019) for interpreting subgroup analyses, I use a p-value of 0.1 for statistical significance and report the distribution of moderators between subgroups for any significant results. Meta-regression was also conducted to estimate the amount of variance in effect size caused by the moderators described above. Results of these heterogeneity analyses suggested that less resource-intensive designs might achieve similar results, and that more consideration should be given to the design of the simulator experience (e.g. the inclusion of reflection and feedback).
These findings were used to inform the design of classroom simulators for use with teacher education students at the Inland Norway University of Applied Sciences. A design-based research approach will be used to qualitatively study the simulators before making revisions and conducting a larger quantitative evaluation.

Author

• Magnus Coney