Characterizing the Practices of Reporting Sex Differences in Infancy Research

• In Event: Poster Session 07
  In Poster Session: PS 07 Topic - Methods, History, Theory

Abstract

In many fields of psychological research, but particularly in infancy research, it is common practice to check for sex differences during data analysis. There are likely many reasons why this practice is common--for example, federally-funded grants require participants’ sex to be reported, so this demographic variable is often available for analysis. In addition, existing societal beliefs and norms about sex and gender make it a salient construct. Finally, due to the difficult and time-intensive nature of collecting data, infant researchers may be especially interested in identifying patterns in datasets once collected. Due to the habitual nature of exploratory testing--on often relatively small datasets--the significant sex differences that are reported may include a substantial amount of false positives or reflect overestimates of effects even where true effects exist (the “winner’s curse”, Button et al., 2013; Figure 1). The goals of the present study are to conduct a wide-ranging survey of the infant literature to determine the potential scope of this issue and to estimate the degree to which the literature might be populated with false positives or overestimates of underlying effects.
The aim of the literature survey was to examine articles on infancy published in well-known developmental outlets over the last 20 years. Each article was coded for several factors, including the age of infants tested, sample size, whether a sex difference was hypothesized, whether testing for a sex difference was reported, and whether a significant sex difference was found. Currently, 97 articles have been cataloged from one prominent journal that publishes infant research, with a median sample size of 48 infants (range: 8-6377). Even with this limited sample, the preliminary results are striking: 29 papers (29.9%) that reported checking for a sex difference did not provide an a priori hypothesis for doing so. Twelve papers (12.3% of the total sample) reported finding a sex difference without having formed a hypothesis regarding sex differences, and none of these papers clearly identified these analyses as planned or unplanned. Coding is ongoing and we anticipate having at least 200 additional papers from other prominent journals coded by the time of the conference.
The results of this study will have important implications for practices in infant research. By virtue of current reporting practices, the infant literature may provide a distorted view of the size and extent of differences due to sex. More broadly, when these findings are disseminated by popular press, they perpetuate existing stereotypes about sex differences. As part of our presentation, we hope to spark discussion with proposals for best practices for investigating and reporting sex differences (as well as other common moderator variables, to which similar issues apply) in infancy research, in particular clearly labeling such analyses as planned or unplanned, reporting moderation analyses conducted regardless of outcome, and sharing data openly to facilitate meta-analyses in domains where sex differences are hypothesized. In many instances, it may be preferable to simply not test for sex differences unless there is an existing reason to do so.

Authors

• Christina Schonberg, University of Wisconsin - Madison
  Presenting Author

• Martin Zettersten, Princeton University
  Non-Presenting Author