Questionable Research Practices in Experimental Communication Research: A Systematic Analysis From 1980 to 2013 (TOP PAPER)

• In Event: Methodological Concerns in Mass Communication Research (TOP PAPER ENTRY)

Abstract

Questionable research practices (QRPs) pose a major threat to any scientific discipline. However, despite the importance of this topic, no systematic analysis of QRPs in experimental mass communication research has yet been undertaken. This paper reports a content analysis of more than three decades of published experimental research in four flagship communication journals: the Journal of Communication, Communication Research, Journalism & Mass Communication Quarterly, and Media Psychology. Current QRPs in experimental communication research include small and insufficiently justified sample sizes, a lack of reported effect sizes, an indiscriminate removal of cases and items, an increasing inflation of p-values directly below p < .05, and a rising share of verified (as opposed to falsified) hypotheses. Implications for authors, reviewers, and editors are discussed.

Authors

• Jorg Matthes, U of Vienna

• Franziska Marquart, U of Vienna

• Brigitte Naderer, U of Vienna

• Florian Arendt, LMU Munich

• Desiree Schmuck, U of Vienna

• Karoline Adam, U of Vienna