Credibility via Coupling: Institutions and Infrastructures in Climate Model Intercomparisons
Abstract
This study investigates Model Intercomparison Projects (MIPs) as one example of a coordinated approach to establishing scientific credibility. MIPs originated within climate science as a method to evaluate and compare disparate climate models, but MIPs or MIP-like projects are now spreading to many scientific fields. Within climate science, MIPs have advanced knowledge of: a) the climate phenomena being modeled, and b) the building of climate models themselves. MIPs thus build scientific confidence in the climate modeling enterprise writ large, reducing questions of the credibility or reproducibility of any single model. This paper will discuss how MIPs organize people, models, and data through institution and infrastructure coupling (IIC). IIC involves establishing mechanisms and technologies for collecting, distributing, and comparing data and models (infrastructural work), alongside corresponding governance structures, rules of participation, and collaboration mechanisms that enable partners around the world to work together effectively (institutional work). Coupling these efforts involves developing formal and informal ways to standardize data and metadata, create common vocabularies, provide uniform tools and methods for evaluating resulting data, and build community around shared research topics.
References
Ankeny, Rachel A., and Sabina Leonelli. 2020. Model Organisms Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/9781108593014.
Aula, Ville. 2019. “Institutions, Infrastructures, and Data Friction—Reforming Secondary Use of Health Data in Finland.” Big Data & Society 6(2): 205395171987598. https://doi.org/10.1177/2053951719875980.
Balaji, Venkatramani, Karl E. Taylor, Martin Juckes, Bryan N. Lawrence, et al. 2018. “Requirements for a Global Data Infrastructure in Support of CMIP6.” Geoscientific Model Development 11(9): 3659–80. https://doi.org/10.5194/gmd-11-3659-2018.
Barua, Maan. 2021. “Infrastructure and Non-Human Life: A Wider Ontology.” Progress in Human Geography 45(6): 1467–89. https://doi.org/10.1177/0309132521991220.
Borgman, Christine L. 2015. Big Data, Little Data, No Data: Scholarship in the Networked World. Cambridge, MA: MIT Press.
Box, George E. P. 1976. “Science and Statistics.” Journal of the American Statistical Association 71(356): 791–99. https://doi.org/10.1080/01621459.1976.10480949.
Braun, Kathrin, and Cordula Kropp. 2010. “Beyond Speaking Truth? Institutional Responses to Uncertainty in Scientific Governance.” Science, Technology, & Human Values 35(6): 771–82. https://doi.org/10.1177/0162243909357916.
Bush, Rosemary, Andrea Dutton, Michael Evans, Rich Loft, et al. 2020. “Perspectives on Data Reproducibility and Replicability in Paleoclimate and Climate Science.” Harvard Data Science Review 2(4). https://doi.org/10.1162/99608f92.00cd8f85.
Cess, Robert D., Potter, Gerald L., Jean-Pierre Blanchet, George J. Boer, et al. 1989. “Interpretation of Cloud-Climate Feedback as Produced by 14 Atmospheric General Circulation Models.” Science 245(4917): 513–16. https://doi.org/10.1126/science.245.4917.513.
Cinquini, Luca, Daniel Crichton, Chris Mattmann, John Harney, et al. 2014. “The Earth System Grid Federation: An Open Infrastructure for Access to Distributed Geospatial Data.” Future Generation Computer Systems 36 (July): 400–417. https://doi.org/10.1016/j.future.2013.07.002.
Collins, Harry. 1985. Changing Order: Replication and Induction in Scientific Practice. Chicago, IL: University of Chicago Press.
⸻. 2013. Gravity’s Ghost and Big Dog: Scientific Discovery and Social Analysis in the Twenty-First Century. Chicago, IL: University of Chicago Press.
Deser, Clara, Flavio Lehner, Keith B. Rodgers, Toby Ault, et al. 2020. “Insights from Earth System Model Initial-Condition Large Ensembles and Future Prospects.” Nature Climate Change 10(2020): 277–86. https://doi.org/10.1038/s41558-020-0731-2.
Easterbrook, Steve M. 2014. “Open Code for Open Science?” Nature Geoscience 7(2014): 779–81. https://doi.org/10.1038/ngeo2283.
Edwards, Paul N. 2010. A Vast Machine: Computer Models, Climate Data, and the Politics of Global Warming. Cambridge, MA: MIT Press.
⸻. 2019a. “Infrastructuration: On Habits, Norms and Routines as Elements of Infrastructure*.” In Thinking Infrastructures (Research in the Sociology of Organizations), edited by Martin Kornberger, Geoffrey C. Bowker, Julia Elyachar, Andrea Mennicken, et al. 62: 355–66. Bingley, Emerald Publishing Limited. https://doi.org/10.1108/s0733-558x20190000062022.
⸻. 2019b. “Knowledge Infrastructures Under Siege: Climate Data as Memory, Truce, and Target.” In Data Politics: Worlds, Subjects, Rights, edited by Didier Bigo, Engin Isin, Evelyn Ruppert, 21–42. New York: Routledge. https://doi.org/10.4324/9781315167305-2.
⸻, Steven J. Jackson, Geoffrey C. Bowker, & Cory P. Knobel. 2007. Understanding Infrastructure: Dynamics, Tensions, and Design. Ann Arbor, MI: University of Michigan. http://deepblue.lib.umich.edu/handle/2027.42/49353.
Feinberg, Melanie, Will Sutherland, Sarah Beth Nelson, Mohammad Hossein Jarrahi, et al. 2020. “The New Reality of Reproducibility: The Role of Data Work in Scientific Research.” Proceedings of the ACM on Human-Computer Interaction 4 (CSCW1): 1–22. https://doi.org/10.1145/3392840.
Galison, Peter. 1987. How Experiments End. Chicago, IL: University of Chicago Press.
Gates, W. Lawrence, ed. 1979. “Report of the JOC Study Conference on Climate Models: Performance, Intercomparison and Sensitivity Studies, Volume I.” Global Atmospheric Research Programme (GARP) Publications Series No. 22. Geneva: World Meteorological Organization.
⸻. 1992. “AMIP: The Atmospheric Model Intercomparison Project.” Bulletin of the American Meteorological Society 73(12): 1962–70. https://doi.org/10.1175/1520-0477(1992)073<1962:atamip>2.0.co;2.
⸻. 1995. “An Overview of AMIP and Preliminary Results.” In Proceedings of the First International AMIP Scientific Conference. World Climate Research Programme, WCRP-92, WMO/TD-No. 732, 1–8. Geneva: World Meteorological Organization. https://library.wmo.int/index.php?lvl=notice_display&id=11852.
⸻, James S. Boyle, Curt Covey, Clyde G. Dease, Charles M. Doutriaux, et al. 1999. “An Overview of the Results of the Atmospheric Model Intercomparison Project (AMIP I).” Bulletin of the American Meteorological Society 80(1): 29–56. https://doi.org/10.1175/1520-0477(1999)080<0029:aootro>2.0.co;2.
Gettelman, Andrew, Cecile Hannay, Julio T. Bacmeister, Richard B. Neale, et al. 2019. “High Climate Sensitivity in the Community Earth System Model Version 2 (CESM2).” Geophysical Research Letters 46(14): 8329–37. https://doi.org/10.1029/2019gl083978.
Guilyardi, Eric, Venkatramani Balaji, Sarah Callaghan, Cecelia DeLuca, et al. 2011. “The CMIP5 Model and Simulation Documentation: A New Standard for Climate Modelling Metadata.” CLIVAR Exchanges 56: 16(2); 42–46. http://centaur.reading.ac.uk/25733/.
Harman, Donna K., and Ellen M. Voorhees. 2007. “TREC: An Overview.” Annual Review of Information Science and Technology 40(1): 113–55. https://doi.org/10.1002/aris.1440400111.
Harter, Stephen P. 1996. “Variations in Relevance Assessments and the Measurement of Retrieval Effectiveness.” Journal of the American Society for Information Science 47(1): 37–49. https://doi.org/10.1002/(sici)1097-4571(199601)47:1<37::aid-asi4>3.0.co;2-3.
Hoeppe, Götz. 2018. “Mediating Environments and Objects as Knowledge Infrastructure.” Computer Supported Cooperative Work (CSCW) 28(1–2): 25–59. https://doi.org/10.1007/s10606-018-9342-0.
Hoesly, Rachel M., Steven J. Smith, Leyang Feng, Zbigniew Klimont, et al. 2018. “Historical (1750–2014) Anthropogenic Emissions of Reactive Gases and Aerosols from the Community Emissions Data System (CEDS).” Geoscientific Model Development 11(1): 369–408. https://doi.org/10.5194/gmd-11-369-2018.
Hulme, Mike, and Martin Mahony. 2010. “Climate Change: What Do We Know about the IPCC?” Progress in Physical Geography: Earth and Environment 34(5): 705–18. https://doi.org/10.1177/0309133310373719.
Intergovernmental Panel on Climate Change (IPCC). 2021. Reports. https://www.ipcc.ch/reports/.
Jasanoff, Sheila. 2004. “Ordering Knowledge, Ordering Society.” In States of Knowledge: The Co-Production of Science and Social Order, edited by Sheila Jasanoff, 13–45. New York: Routledge.
Jebeile, Julie, and Michel Crucifix. 2020. “Multi-Model Ensembles in Climate Science: Mathematical Structures and Expert Judgements.” Studies in History and Philosophy of Science Part A 83(October): 44–52. https://doi.org/10.1016/j.shpsa.2020.03.001.
Jefferson, Gail. 1985. “On the Interactional Unpackaging of a ‘Gloss.’” Language in Society 14(4): 435–66. https://doi.org/10.1017/s0047404500011465.
Kennefick, Daniel. 2007. Traveling at the Speed of Thought: Einstein and the Quest for Gravitational Waves. Princeton, NJ: Princeton University Press.
Latour, Bruno. 2013. An Inquiry into Modes of Existence: An Anthropology of the Moderns. Trans. Catherine Porter. Cambridge, MA: Harvard University Press.
Lee, Charlotte P., and Kjeld Schmidt. 2018. “A Bridge Too Far? Critical Remarks on the Concept of ‘Infrastructure’ in Computer-Supported Cooperative Work and Information Systems.” In Socio-Informatics: A Practice-based Perspective on the Design and Use of IT Artifacts, edited by Volker Wulf, Volkmar Pipek, David Randall, Markus Rohde, et al., 177-217. Oxford: Oxford University Press. https://doi.org/10.1093/oso/9780198733249.003.0006.
Leonelli, Sabina. 2018. “Rethinking Reproducibility as a Criterion for Research Quality.” In Including a Symposium on Mary Morgan: Curiosity, Imagination, and Surprise, Volume 36B, edited by Luca Fiorito, Scott Scheall, & Carlos Eduardo Suprinyak, 129–46. Emerald Publishing Limited. https://doi.org/10.1108/s0743-41542018000036b009.
⸻. 2019. “Scientific Agency and Social Scaffolding in Contemporary Data-Intensive Biology.” In Beyond the Meme: Development and Structure in Cultural Evolution, edited by Alan C. Love and William Wimsatt, 42–63. Minneapolis, MN: University of Minnesota Press.
Lloyd, Elisabeth A., Greg Lusk, Stuart M. Gluck, and Seth McGinnis. (forthcoming) “Varieties of Data-Centric Science: Regional Climate Modeling and Model Organism Research.” Philosophy of Science.
Loescher, Henry W., Eugene F. Kelly, and Russ Lea. 2017. “National Ecological Observatory Network: Beginnings, programmatic and scientific challenges, and ecological forecasting.” In Terrestrial Ecosystem Research Infrastructures, edited by Abad Chabbi and Henry W. Loescher, 27–51. Boca Raton, FL: CRC Press.
Mayernik, Matthew S. 2016. “Research Data and Metadata Curation as Institutional Issues.” Journal of the Association for Information Science and Technology 67(4): 973–93. https://doi.org/10.1002/asi.23425.
⸻. 2019. “Metadata Accounts: Achieving Data and Evidence in Scientific Research.” Social Studies of Science 49(5): 732–57. https://doi.org/10.1177/0306312719863494.
Meehl, Gerald A., George J. Boer, Curt Covey, Mojib Latif, et al. 1997. “Intercomparison Makes for a Better Climate Model.” Eos, Transactions American Geophysical Union 78(41): 445–51. https://doi.org/10.1029/97eo00276.
⸻, Curt Covey, Thomas Delworth, Mojib Latif, et al. 2007. “THE WCRP CMIP3 Multimodel Dataset: A New Era in Climate Change Research.” Bulletin of the American Meteorological Society 88(9): 1383–94. https://doi.org/10.1175/bams-88-9-1383.
Mervis, Jeffrey. 2019. “Shake-Up Threatens Novel U.S. Ecology Facility.” Science 363(6424): 211–12. https://doi.org/10.1126/science.363.6424.211.
Miller, Clark. 2001. “Hybrid Management: Boundary Organizations, Science Policy, and Environmental Governance in the Climate Regime.” Science, Technology, & Human Values 26(4): 478–500. https://doi.org/10.1177/016224390102600405.
Miller, Seumas. 2019. “Social Institutions.” In Stanford Encyclopedia of Philosophy. Stanford, CA: Stanford University. http://plato.stanford.edu/entries/social-institutions/.
Miyakawa, Tsuyoshi. 2020. “No Raw Data, No Science: Another Possible Source of the Reproducibility Crisis.” Molecular Brain 13, 24 (2020). https://doi.org/10.1186/s13041-020-0552-2.
Morrison, Monica A. 2021. “The Models Are Alright: A Theory of the Socio-Epistemic Landscape of Climate Model Development.” PhD Diss. Indiana University.
Moylan, Elizabeth C., and Maria K. Kowalczuk. 2016. “Why Articles Are Retracted: A Retrospective Cross-Sectional Study of Retraction Notices at BioMed Central.” BMJ Open 6(11): e012047. https://doi.org/10.1136/bmjopen-2016-012047.
[NASEM] National Academies of Sciences, Engineering, and Medicine. 2019. “Reproducibility and Replicability in Science.” Consensus Study Report Washington, DC: National Academies Press. https://doi.org/10.17226/25303.
Nobre, Paulo, Leo S. P. Siqueira, Roberto A. F. de Almeida, Marta Malagutti, et al. 2013. “Climate Simulation and Change in the Brazilian Climate Model.” Journal of Climate 26(17): 6716–32. https://doi.org/10.1175/jcli-d-12-00580.1.
North, Douglass C. 1990. Institutions, Institutional Change and Economic Performance. New York: Cambridge University Press.
[PCMDI] Program for Climate Model Diagnosis & Intercomparison. 2021. ESGF CMIP6 Data Holdings. Livermore, CA: Lawrence Livermore National Laboratory. https://pcmdi.llnl.gov/CMIP6/ArchiveStatistics/esgf_data_holdings/.
Penders, Bart, Holbrook, J. Britt, & Sarah de Rijcke. 2019. “Rinse and Repeat: Understanding the Value of Replication across Different Ways of Knowing.” Publications 7(3): 52. https://doi.org/10.3390/publications7030052.
Randall, David A., Richard A. Wood, Sandrine Bony, Robert Colman, et al. 2007. “Climate Models and Their Evaluation.” In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. New York: Cambridge University Press. https://www.ipcc.ch/report/ar4/wg1/climate-models-and-their-evaluation/.
Ribes, David and Steven J. Jackson. 2013. “Data Bite Man: The Work of Sustaining a Long-Term Study.” In “Raw Data” is an Oxymoron, edited by Lisa Gitelman. 147–166. Cambridge, MA: MIT Press.
Rood, Richard B. 2019. “Validation of Climate Models: An Essential Practice.” In Simulation Foundations, Methods and Applications, edited by Claus Beisbart and Nicole J. Saam, 737–62. Springer International Publishing. https://doi.org/10.1007/978-3-319-70766-2_30.
Schmidt, Gavin A., and Steven Sherwood. 2014. “A Practical Philosophy of Complex Climate Modelling.” European Journal for Philosophy of Science 5(2): 149–69. https://doi.org/10.1007/s13194-014-0102-9.
Shapin, Steven. 1995. “Cordelia’s Love: Credibility and the Social Studies of Science.” Perspectives on Science 3(3): 255–275. http://nrs.harvard.edu/urn-3:HUL.InstRepos:3293019.
Somerville, Richard C. 2011. “The Co-evolution of Climate Models and the Intergovernmental Panel on Climate Change.” In The Development of Atmospheric General Circulation Models: Complexity, Synthesis, and Computation, edited by Leo Donner, Wayne Schubert, and Richard C. Somerville, 225–252. New York: Cambridge University Press.
Star, Susan Leigh, and Karen Ruhleder. 1996. “Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces.” Information Systems Research 7(1): 111–34. https://doi.org/10.1287/isre.7.1.111.
Stockhause, Martina, Heinke Höck, Frank Toussaint, and Martin Lautenschlager. 2012. “Quality Assessment Concept of the World Data Center for Climate and its Application to CMIP5 Data.” Geoscientific Model Development 5(4): 1023–32. https://doi.org/10.5194/gmd-5-1023-2012.
Swapna, P., M. K. Roxy, K. Aparna, K. Kulkarni, et al. 2015. “The IITM Earth System Model: Transformation of a Seasonal Prediction Model to a Long-Term Climate Model.” Bulletin of the American Meteorological Society 96(8): 1351–67. https://doi.org/10.1175/bams-d-13-00276.1.
Taylor, Karl. 2013. CMIP5 Standard Output. Livermore, CA: Program for Climate Model Diagnosis and Intercomparison (PCMDI). https://pcmdi.llnl.gov/mips/cmip5/docs/standard_output.pdf?id=4.
⸻, Charles Doutriaux, and Jean-Yves Peterschmitt. 2006. Climate Model Output Rewriter (CMOR). Livermore, CA: Program for Climate Model Diagnosis & Intercomparison (PCMDI). https://pcmdi.github.io/cmor-site/media/pdf/cmor_users_guide.pdf.
Touzé‐Peiffer, Ludovic, Anouk Barberousse, and Hervé Le Treut. 2020. “The Coupled Model Intercomparison Project: History, Uses, and Structural Effects on Climate Research.” WIREs Climate Change 11(4). https://doi.org/10.1002/wcc.648.
Veiga, Sandro F., Paulo Nobre, Emanuel Giarolla, Vinicius Capistrano, et al. 2019. “The Brazilian Earth System Model Ocean—Atmosphere (BESM-OA) Version 2.5: Evaluation of its CMIP5 Historical Simulation.” Geoscientific Model Development 12(4): 1613–42. https://doi.org/10.5194/gmd-12-1613-2019.
[WCRP] World Climate Research Program. 2014. Application for CMIP6-Endorsed MIPs. World Climate Research Programme. https://www.wcrp-climate.org/modelling-wgcm-mip-catalogue/modelling-wgcm-cmip6-endorsed-mips.
⸻. 2019. Report of the 22nd Session of the Working Group on Coupled Modeling: 25th and 29th March 2019, Barcelona, Spain. WCRP Publication 14(2019). World Climate Research Program. https://www.wcrp-climate.org/WCRP-publications/2019/WCRP-Report-No14-2019-WGCM22.pdf.
[WIP] WGCM Infrastructure Panel. 2014. Terms of Reference for the WGCM Infrastructure Panel (WIP). World Climate Research Programme. https://wcrp-cmip.github.io/WGCM_Infrastructure_Panel/Papers/WIP_Terms_of_Reference.pdf.
Wilson, Joseph. 2021. “Two Exploratory Uses for General Circulation Models in Climate Science.” Perspectives on Science 29(4): 493–509. https://doi.org/10.1162/posc_a_00380.
Winsberg, Eric. 2018. Philosophy and Climate Science. New York: Cambridge University Press.
Yan, An, Caihong Huang, Jian‐Sin Lee, and Carole L. Palmer. 2020. “Cross‐Disciplinary Data Practices in Earth System Science: Aligning Services with Reuse and Reproducibility Priorities.” Proceedings of the Association for Information Science and Technology 57(1): e218. https://doi.org/10.1002/pra2.218.
Young, Oran R., Paul Arthur Berkman, and Alexander N. Vylegzhanin. 2020. “Informed Decisionmaking for the Sustainability of Ecopolitical Regions.” In Informed Decisionmaking for Sustainability, edited by Oran R. Young, Paul Arthur Berkman, and Alexander N. Vylegzhanin, 341–53. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-25674-6_15.
Zelinka, Mark D., Timothy A. Myers, Daniel T. McCoy, Stephen Po‐Chedley, et al. 2020. “Causes of Higher Climate Sensitivity in CMIP6 Models.” Geophysical Research Letters 47(1). https://doi.org/10.1029/2019gl085782.
Copyright (c) 2021 Matthew S. Mayernik
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors of all content published in ESTS retain the copyright to their work, and agree to license them under one of the following Creative Commons licenses CC BY-NC-SA 4.0, CC BY 4.0, CC BY-SA 4.0, and refer to the individual article footer for specific licensing data. Please read our open access policy for more information.
