Poster #43 - Methodological Integrity Assessment in the Mobile Paradigm Literature: A Lesson for Understanding Opportunistic Use of Researcher Degrees of Freedom in Psychology

• In Event: Poster Session 20

Abstract

The mobile paradigm has played a fundamental role in the study of learning and memory capacities of young infants. Studies implementing the mobile paradigm report that infants as young as 3 months of age can detect and learn the contingency between their kicking and the movement of a mobile tied to their leg with a ribbon (Rovee & Rovee, 1969; Rovee-Collier et al., 1978). Moreover, infants can remember the visual characteristics of their environment after days and even weeks if they are reminded with the training mobile (Rovee & Fagen 1976; Rovee-Collier et al., 1980). One key characteristic of the mobile paradigm literature is that across decades, researchers have faithfully followed a particular methodological protocol with its own unique definitions of learning and memory. In addition, statistically significant findings relying on small sample sizes have appeared consistently within and across studies. Despite considerable research with the mobile paradigm, the extent to which these methodological choices affected the results is unknown. In the present work, we evaluated the mobile paradigm literature, across 77 publications and 505 statistical tests, for four frequently encountered research biases in psychology: opportunistic use of researcher degrees of freedom, p-hacking, unintentional errors, and opportunistic use of the stopping rule. Our analysis showed no signs of methodological bias regarding p-hacking, unintentional errors, or opportunistic use of stopping rule. With respect to these biases, it can be concluded that there is no evidence for scientific misconduct and that the mobile paradigm literature is a product of decades of work conducted with scientific rigor. Nevertheless, the assessment of opportunistic use of researcher degrees of freedom revealed a false-positive rate of up to 65% when we applied a methodological protocol specific to the mobile paradigm to a simulated population that behaved randomly. In other words, methodological choices along with unique operational definitions of learning and memory accounted for more than half of the findings in the mobile paradigm literature. Based on these results, it can be suggested that memory findings in the mobile paradigm literature present a case of scientific endemism, in which the phenomenon under investigation exists only within the ecosystem created by a specific methodological protocol. The customized operationalizations (e.g., operational definitions of learning and memory) and methodological parameters (e.g., small sample size, types of statistical tests, using an eligibility criteria) account for the observed results, rather than for the true state of development in infancy.

Authors

• Umay Sen, Uppsala universitet
  Presenting Author

• Gustaf Gredebäck, Uppsala universitet