Media Comparisons: Necessary but Currently Not Sufficient

• In Event: Media Comparisons: To Do or Not to Do, That is the Question

Abstract

Media comparison research has been conducted for decades as a way of assessing the impacts of new instructional technologies. With each new technology that hits the educational market, studies are developed to assess the impact of these instructional technologies as compared to more traditional forms of instruction. For example, in the age of artificial intelligence, several recent meta-analyses (e.g., Deng et al., 2025; Wang and Fan, 2025) have been published examining the use of ChatGPT compared to more conventional instructional approaches. Other recent tools that have been heavily investigated via media comparison studies include immersive virtual reality (e.g., see Lawson et al., 2024) and augmented reality (e.g., see Buchner & Kerres, 2023). Although media comparison studies have long been the subject of much debate (e.g., Clark, 1983, 1994; Kozma, 1994; Warnick & Burbles, 2007), they have remained a staple within fields such as educational psychology and educational technology. As such, rather than write them off completely and advocate that media comparison studies should never be conducted, we argue that it is more productive to identify ways for these studies to be more informative and to produce stronger practical recommendations.

Further, conducting media comparison studies remains crucial for determining whether new technologies should replace traditional instructional approaches. As has often been found, new technologies do not always lead to better learning (see Clark, 1983; Lawson et al., under review; Lawson & Mayer, 2024) and in some cases even seem to hurt learning (e.g., Parong & Mayer, 2018, 2021). These possible negative impacts of technology would be missed if we stopped conducting media comparison research. Therefore, we must compare anything novel to research-validated, conventional instructional approaches to ensure our students are not being negatively impacted by pedagogical changes.

However, it is vital to improve the design and implementation of media comparison studies if we are to conduct them. We argue that media comparison research must hold all instructional methods not unique to the technology constant. These methods may include practice opportunities, active learning experiences, lecture, and feedback, to name a few. When instructional methods are held constant, identifying unique impacts of a technology becomes possible. If instructional methods not unique to the technology differ between conditions, though, there will be a confounding of the methods with the media (as famously discussed by Clark, 1983, 1994). For example, if students in a virtual reality condition receive opportunities to practice pipetting but students in the conventional condition do not, it is unclear if the use of virtual reality or the addition of practice opportunities was responsible for any learning gains. What needs to occur for cause-and-effect claims to be made is for researchers to outline exactly what occurred within each condition or instructional package so that results can be attributed to any differences between conditions.

Authors

• Amédee Marchand Martella, University of Georgia

• Alyssa Lawson, Colby College