An Application of Multiverse Analysis to Address the Inconsistent Results Caused by Data Processing Choices

• In Event: AERA Poster Session 4
  In Poster Session: Poster Session (San Diego)

Abstract

One of the challenges scientists face is choosing the right methodological approach for their analysis. Even after the study design is determined and all information is collected, there are still various ways to process the data. Such variability may create the issue of inconsistent results that are nearly impossible to replicate. To illustrate this, we reinvestigated the relationship between mother-father-child interactions and a child's receptive vocabulary at age five. We performed a Multiverse Analysis by testing different combinations of some common methodological choices producing thirty-six model variations. Comparison of the results across all models highlighted the advantages of the Multiverse approach over a single analysis by revealing that even a small difference in data processing may lead to inconsistent conclusion.

Author

• Aleksandra Kazakova, The City University of New York