Perspectives on AI Architectures and Co-design for Earth System Predictability

Maruti Kumar Mudunuru, James Ang, Mahantesh Halappanavar, Simon David Hammond, Maya Gokhale, James C. Hoe, Tushar Krishna, Sarat Sreepathi, Matthew Norman, Ivy Peng, Philip W. Jones2024 · DOI

TL;DR

This paper proposes a new paradigm for Earth system modeling by integrating AI across the entire "Model-Experimentation" (ModEx) lifecycle. The authors argue that improving climate and weather predictability requires a radical "co-design" approach, where scientific requirements directly shape the development of next-generation high-performance computing (HPC), cloud, and edge architectures.

What Problem Does This Paper Address?

Traditional Earth system modeling (ESM) relies on massive physics-based simulations that are increasingly struggling to keep pace with the explosion of observational data and the slowing of Moore’s Law. Currently, field observations and computational models often operate in slow, disconnected loops.

The authors identify a "data deluge" problem: sensors are generating petabytes to exabytes of data, making it prohibitively expensive and energy-intensive to move all that information to central data centers. Furthermore, existing hardware is often a "one-size-fits-all" solution that isn't optimized for the emerging hybrid workflows that combine traditional Fortran/C++ simulation codes with modern Python-based AI frameworks. The paper seeks to bridge this gap by outlining how hardware and software should be designed together to handle the complexity of Earth system predictability.

Methods

As a perspective paper resulting from the Department of Energy’s (DOE) AI4ESP workshop series, the authors synthesize expert discussions into four "Future System Concepts":

  1. Centralized Large-scale HPC: Moving beyond current exascale systems (like Frontier) toward "extreme heterogeneity," featuring specialized accelerators for specific tasks like uncertainty quantification (UQ) or graph analysis.
  2. Edge Sensors with Centralized Resources: Using AI at the sensor level to summarize or select relevant features from data streams, drastically reducing the volume of data sent to the cloud.
  3. Federated Processing: Leveraging "fog computing" where data is processed in nearby edge servers, allowing for local model updates and parameter exchanges without a central bottleneck.
  4. Dynamic and Adaptive Federated Processing: A "smart" closed-loop system where AI models at the edge and center communicate to dynamically steer sensors—for example, automatically "zooming in" or increasing sampling frequency when a hurricane begins to form.

Datasets

While this is a roadmap paper rather than a benchmarking study, it highlights the critical role of existing DOE data facilities and the need for new ones:

  • Existing Sources: Data from the Atmospheric Radiation Measurement (ARM) facility and the Environmental Molecular Sciences Laboratory (EMSL).
  • Data Repositories: The ESS-DIVE (Environmental System Science Data Infrastructure for a Virtual Ecosystem) for long-term storage.
  • Future Needs: The authors explicitly call for the creation of new "Benchmark Datasets" specifically designed for Earth system AI to allow for standardized testing of new architectures.

Key Insights and Ideas

  • The AI-Enabled ModEx Loop: The most significant shift is moving from human-led iterations to AI-accelerated loops. AI can generate hypotheses, steer sensors in real-time, and parameterize models faster than traditional methods.
  • Compute is Cheaper than Movement: A recurring theme is that moving exabytes of data is the primary bottleneck. The "aha moment" is the realization that we must move the intelligence to the data (at the edge) rather than the data to the intelligence.
  • Architectural Specialization: General-purpose GPUs are no longer enough. The community needs "architectural specialization"—hardware designed specifically for the kernels used in Earth science, such as sparse data handling and streaming analytics.
  • Cross-Domain Synergy: The paper maps specific AI needs to scientific sub-domains. For instance, Hydrology needs AI for down-scaling data, while Atmospheric Modeling requires co-design for subgrid physics parameterization.

Key Results

The paper provides a strategic roadmap rather than quantitative experimental results. Key outcomes include:

  • Identification of 4 Grand Challenges: (1) Programmability and Usability (bridging the C++/Python gap), (2) Data Movement, (3) Energy Efficiency, and (4) Privacy and Security of distributed sensor networks.
  • A Tiered Goal Structure: Table 1 outlines a timeline for the community:
  • Short-term (<5 years): Benchmark datasets and AI/ML surrogates.
  • Medium-term (5 years): AI@SensorEdge and AI/ML + physics simulators.
  • Long-term (10 years): AI-enabled automation and "Digital Twins" for Earth System Predictability.

Limitations and Future Work

The authors acknowledge several steep hills to climb:

  • Interoperability: Current cloud providers (AWS, Azure, GCP) have different interfaces, making a truly "federated" cloud for science difficult to implement.
  • The Language Barrier: There is a significant technical hurdle in integrating legacy simulation codes (Fortran/C++) with modern AI stacks (Python/PyTorch) without massive performance loss.
  • Energy Consumption: Large-scale sensing networks could become power-hungry; research into neuromorphic-based accelerators is suggested as a potential solution.
  • Security: As we move toward autonomous, AI-steered sensors, the risk of data tampering or "nefarious modification" becomes a national security concern for climate policy decisions.

Summary updated

Abstract

Predictability (AI4ESP) workshop series.From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, lab, modeling, and analysis activities, called ModEx.The BER's 'Model-Experimentation', ModEx, is an iterative approach that enables process models to generate hypotheses.The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process-based) predictions.A total of 17 technical sessions were held in this AI4ESP workshop series.This paper discusses the topic of the 'AI Architectures and Co-design' session and associated outcomes.The AI Architectures and Co-design session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including:(1) DOE high-performance computing (HPC) Systems, (2) Cloud HPC Systems, and (3) Edge computing and Internet of Things (IoT).We also provide forward-looking ideas and perspectives on potential research in this co-design area that can be achieved by synergies with the other 16 session topics.These ideas include topics such as: (1) reimagining co-design, (2) data acquisition to distribution, (3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with earth system modeling and simulation, and (4) AI-enabled sensor integration into earth system measurements and observations.Such perspectives are a distinguishing aspect of this paper.2 SIGNIFICANCE STATEMENT: This study aims to provide perspectives on AI architectures and co-design approaches for Earth-system predictability.Such visionary perspectives are essential because AI-enabled model-data integration has shown promise in improving predictions associated with climate change, perturbations, and extreme events.Our forward-looking ideas guide what is next in co-design to enhance Earth-system models, observations, and theory using state-of-the-art and futuristic computational infrastructure.
BibTeX
@misc{mudunuru2024perspectives,
  author = {Maruti Kumar Mudunuru and James Ang and Mahantesh Halappanavar and Simon David Hammond and Maya Gokhale and James C. Hoe and Tushar Krishna and Sarat Sreepathi and Matthew Norman and Ivy Peng and Philip W. Jones},
  title = {{Perspectives on AI Architectures and Co-design for Earth System Predictability}},
  year = {2024},
  doi = {10.2172/2378014},
}
Figure