From Whether to When: Five Conditions for Meaningful Media Comparison Research

• In Event: Media Comparisons: To Do or Not to Do, That is the Question

Abstract

There is broad consensus that studies conflating medium and instructional method lack explanatory power and fail to contribute meaningfully to either theory or practice. The critique of such designs is justified and well documented (Clark, 1983; Kozma, 1994; Reigeluth & Honebein, 2023). For a long time, I shared this critique without reservation—especially while working on a review of media comparisons in augmented reality (AR) research (Buchner & Kerres, 2023). At that point, I would have clearly argued for abandoning the approach altogether.

Today, my view has become more differentiated, shaped by two developments: First, I have increasingly focused on educational goals that go beyond cognition—such as interest, motivation, and attitude. Second, I work at the intersection of educational research and teacher education, where researchers frequently engage with practitioners. And practitioners—whether in schools or higher education—often ask a pragmatic question: Which media is better? While it is tempting to dismiss this as the wrong question, it deserves better answers—ones that are both theoretically grounded and practically relevant.

This contribution therefore aims to move beyond dichotomous debates and offer a nuanced, actionable framework for researchers and instructional designers alike. I argue that media comparisons can be meaningful—but only under specific conditions. These conditions are interdependent and should be viewed as part of a broader design matrix. I outline five core principles:
1. Theoretical grounding. Comparisons should be embedded in theory. For instance, in IVR research, CAMIL (Makransky & Petersen, 2021) can be used to examine how different media generates presence and subsequently affects learning, even when content and task type are identical.
2. Instructional equivalence. Learning activities across conditions must be equivalent in depth and complexity. Frameworks like ICAP (Chi & Wylie, 2014) can help ensure that learners engage in comparable cognitive processes, whether in immersive, video-based, or text-based formats. Without such alignment, claims about media effects remain confounded and inconclusive.
3. Focus on affective outcomes. If two conditions are instructionally equivalent but only differ on medium, and one condition results in higher motivation or interest development, this has practical relevance—especially for teachers aiming to foster not only effectiveness but also appeal (Honebein & Reigeluth, 2021).
4. Task–technology fit. The medium must align with the instructional purpose (Bower, 2008). In teacher education, for instance, opportunities to observe classrooms are limited. A 360° video or IVR simulation may offer more authenticity than traditional video, potentially supporting empathy and reflective thinking.
5. Longitudinal and ecologically valid design. One-shot studies offer little insight into sustained learning or transfer (e.g., Zumbach et al., 2022). Meaningful media comparisons require extended implementation, aligned learning goals, and outcome measures beyond short-term recall.

These five principles reinforce one another and form the basis for a multidimensional matrix to guide robust media comparison research. While this position acknowledges limitations—e.g., the risk of underutilizing advanced media in simplified study designs—it also emphasizes that, when theoretically justified and instructionally grounded, media comparisons can meaningfully contribute to both educational theory and practice.

Author

• Josef Buchner, St. Gallen University of Teacher Education