Achieving Scientific Reliability in Climate Science via Intercomparisons: Institutional and Infrastructural Coupling

• In Event: Reproducibility and Other Problems: Practical and Institutional Responses to Contemporary Crises in Science - I

Abstract

Commentary on the “crisis of reproducibility” within academic research tends to focus on individual research studies or the work of individual scholars. In a number of research areas, however, groups of researchers come together to define coordinated initiatives based around common methods, data, and/or research questions. In such initiatives, the direct reproducibility of individual contributions are subsumed by the ability to compare and collate large numbers of results from many contributors.

This study investigates Model Intercomparison Projects (MIPs) as one example of a coordinated approach to establishing scientific reliability. MIPs originated within climate science as a method to evaluate and compare disparate climate models, but MIPs or MIP-like projects are now spreading to many scientific fields. Within climate science, MIPs have advanced knowledge of: a) the climate phenomena being modeled, and b) the building of climate models themselves. MIPs thus build scientific confidence in the climate modeling enterprise writ large, bypassing questions of the reliability or reproducibility of any single model. This paper will discuss how MIPs organize people, models, and data through institution and infrastructure coupling (IIC). IIC involves establishing mechanisms and technologies for collecting, distributing, and comparing data and models (infrastructural work), alongside corresponding governance structures, rules of participation, and collaboration mechanisms that enable partners around the world to work together effectively (institutional work). Coupling these efforts involves developing formal and informal ways to standardize data and metadata, create common vocabularies, provide uniform tools and methods for evaluating resulting data, and build community around shared research topics.

Author

• Matthew Mayernik, National Center for Atmospheric Research