TL;DR
The climate risk science community faces a significant gap between the stated principles of transparency and reusability and actual practice, with only 4% of top studies sharing their data and code. This paper calls for immediate, low-cost actions and larger institutional investments to improve the reliability and utility of climate risk information for decision-making.
What Problem Does This Paper Address?
Climate risk projections are increasingly crucial for decisions ranging from where people live to how governments allocate funding. However, the reliability of these projections is hampered by data limitations and fundamental validation challenges. The authors argue that while transparency and reusability are widely accepted principles for addressing these issues, there's a substantial disconnect between these ideals and current research practices. This gap hinders the scientific community's ability to build upon existing knowledge, scrutinize methods, and ultimately provide sound, evidence-based climate risk management.
Methods
The paper's core "method" is an analysis of the most-cited peer-reviewed climate risk studies published between 2021 and 2022. They assessed the degree of data and code openness for these studies across various leading journals. Beyond this analysis, the paper proposes actionable recommendations for improving transparency and reusability, categorizing them into near-term, low-cost investments and longer-term, transformational investments.
Datasets
The paper does not introduce new datasets. Instead, it analyzes the availability of data and code associated with existing climate risk research studies. The authors reference various types of data that are often used in climate risk assessments, such as address-level flood insurance claims, but do not present a specific dataset for evaluation.
Key Insights and Ideas
- The "Transparency Gap": Despite widespread agreement on the importance of open science, the practice lags significantly. Only a tiny fraction of high-impact climate research makes its data and code fully available.
- Norms Matter: The lack of openness isn't necessarily due to technical barriers but rather weak community norms and a lack of consistent enforcement by journals and institutions.
- Low-Cost Wins: Simple, inexpensive actions like enforcing journal standards, citing open-source resources, and writing clear availability statements can make a big difference.
- Reusability Fuels Innovation: Projects like PyPSA demonstrate how investing in reusability can create thriving ecosystems of open-source initiatives and accelerate scientific progress.
- Beyond Individual Efforts: While individual researchers can do more, transformative change requires institutional commitment, including investments in infrastructure, training, and specialized staff.
- Equity is Crucial: Any push for transparency and reusability must consider potential equity implications, ensuring that marginalized groups are not further disadvantaged.
Key Results
The primary quantitative finding is that only 4% of the most-cited peer-reviewed climate risk studies fully share their data and code. The paper also highlights that many journals have stated standards for openness, but these are not consistently enforced. The authors provide examples of successful open-science initiatives like the Coupled Model Intercomparison Project (CMIP) and the PyPSA software, which have demonstrably benefited from transparency and reusability.
Limitations and Future Work
The authors acknowledge that their perspective is primarily drawn from North American and European contexts and that global applicability requires careful consideration of local needs and priorities. They also emphasize that while transparency and accessibility are essential, they don't fully solve the problem of producing reliable climate risk information, particularly with the rise of complex AI models. Future work should focus on:
- Developing more robust institutional frameworks for transparency and reusability.
- Addressing data access barriers, especially for sensitive or proprietary data, through privacy-preserving methods and controlled access arrangements.
- Fostering a stronger community culture around open science practices.
- Continuing the dialogue on equity and ensuring that open science benefits all stakeholders.
- Exploring how to ensure the interpretability and trustworthiness of complex AI models used in climate risk assessment.
Summary updated