Two Bites of the Cherry: Incidental Inflation of Significant Findings in Media Research and How New Structures in Manuscript Preparation and Review can Prevent It

• In Event: Open Science Practices in Communication Research and Reviewing: Overdue or Overrated?

Abstract

Abstract: While high-profile instances of fraud and data manipulation have captured the attention of social psychology and the broader academic sphere, the field of communication research has remained relatively free of such dramatic scandals. It is undeniable, however, that much more minor violations of the assumptions of the classic null hypothesis statistical testing approach occur on a regular basis and are even encouraged by normative practices in the field. This presentation will share new data hinting at the prevalence of relatively innocuous, but problematic practices such as the “chrysalis effect” (emergence of proportionally stronger support for hypotheses in publications stemming from graduate theses than in the original thesis work) and detail specific anecdotal examples where the editorial review and revision process have also inadvertently encouraged inflation of significant findings to the detriment of open science. Finally, an agenda for eradicating such incidental encouragement of inflation of significant findings will be proposed.

Qualifications: Dr. James D. Ivory is an associate professor in the Department of Communication at Virginia Tech who serves on the editorial boards of several communication journals. Dr. Ivory is lead author of an in-press comment addressing a specific instance of questionable research practices in research on video game effects (Ivory et al., in press). As part of his program of research on the social role of video games, virtual environments, and simulations, Dr. Ivory has conducted research on cognitive factors influencing how people perceive the effects of violent video games (Ivory & Kalyanaraman, 2009), the possible relationship of general media use with criminal behavior (Ferguson, Ivory, & Beaver, 2013), and social influences on misperceptions about evidence linking media use to prominent mass shootings (Ferguson & Ivory, 2012). This program of research examining both the effects of violent games and other media and the way people perceive and misperceive those effects is grounded in a broader program of research on general effects of technological factors in media (e.g., Elson, Breuer, Ivory, & Quandt, 2014; Ivory & Kalyanaraman, 2007; Ivory & Magee, 2009;) and gender portrayals and sex-role stereotyping in video games and other media (e.g., Holz Ivory, Fox, Waddell, & Ivory, 2014; Holz Ivory, Gibson, & Ivory, 2009; Ivory, 2006; Williams, Martins, Consalvo, & Ivory, 2009).

Author

• James D. Ivory, Virginia Tech