Heterogeneity in Pediatric EEG Data Processing and Analysis: A state of the Field and Implications for Preregistration

• In Event: Towards a More Open, Reproducible, and Representative Field of Developmental EEG Research

Abstract

Reproducibility and replicability in neuroscience studies have historically been low. Neuroimaging research suffers from high levels of false positive results (Poldrack et al., 2017), in part because neuroscientific analyses often have a high degree of flexibility in analysis methods. The past decade has seen rapidly increasing interest in addressing concerns over the replicability of manuscripts as well as reducing p-hacking and publication bias through preregistration. Preregistration of analysis plans and hypothesis are potential solutions to this problem (Poldrack et al., 2017). “Preregistration” is defined as the recording of hypotheses and planned analyses before the collection of data. A “registered report” is a publication submitted with hypotheses and methods, but prior to analysis of results; journals that accept such reports for publication agree to publish the final paper regardless of the specific findings (Chambers et al., 2014). In this talk, I will first overview the current state of field in pediatric resting EEG research. To investigate this, a survey was created and reviewed by five experts in pediatric resting EEG, and disseminated to EEG researchers by email, social media, and word of mouth. To participate, survey respondents needed to be at least a doctoral trainee with expertise in developmental EEG. A total of 77 participants consented to the survey and completed at least some of the EEG questions (22 doctoral students, 20 postdocs, 6 research or staff scientists, 1 adjunct faculty member, 12 assistant professors, 11 associate professors, and 5 full professors). The survey asked researchers to preregister how they would analyze two datasets–one low-density and one high-density. Of the 77 responses, no two respondents preregistered the same processing or analytic plan (see Figures 1 and 2 for examples of data to be discussed). Major differences existed in almost every stage of the analytic stream, including epoch parameters, artifact removal, power band boundaries, power transformations, outlier identification, and corrections for multiple comparisons. While not unexpected, this was particularly notable given that many of the survey respondents were either trained by or had previously published with many of the other survey respondents. Given the vast differences in approaches to data cleaning and analysis, I will discuss important methodological gaps that need to be addressed in pediatric resting EEG. This discussion will be centered around implications for replicability and reliability of future studies. I will close with concrete recommendations surrounding preregistering pediatric, resting EEG studies, including recommendations for how to preregister a replicable study as well as details on what should be included in the preregistration.

Author

• Sonya V. Troller-Renfree, Teachers College, Columbia University
  Presenting Author