Abstract
Geological Carbon Storage (GCS) is a critical technology for climate mitigation, yet optimizing injection operations remains computationally prohibitive due to the complexity of subsurface physics. This paper introduces the Multimodal Latent Dynamic (MLD) model, a deep learning framework that learns compressed representations of reservoir states. By serving as a high-speed surrogate environment for reinforcement learning agents, the MLD model enables a 60% reduction in computational costs while maximizing storage performance and economic value across diverse geological scenarios.
Introduction
To make GCS projects commercially viable, engineers must optimize "well controls"—the time-varying rates of $CO_2$ injection and production—to maximize Net Present Value (NPV) while ensuring safety. Traditional optimization relies on high-fidelity numerical simulators that solve complex physics equations. However, a single simulation can take hours, and optimization algorithms often require thousands of iterations.
Existing surrogate models often suffer from two primary issues: temporal consistency, where errors accumulate over long time horizons, and poor generalization, where models must be retrained for every new reservoir. This research addresses these challenges by combining a novel world model with deep reinforcement learning.
Methodology
The Multimodal Latent Dynamic (MLD) Model
The MLD model avoids predicting high-dimensional reservoir states (like 3D pressure maps) directly. Instead, it compresses this information into a low-dimensional latent space. The architecture consists of three primary components:
- Representation Module: A fused encoder using CNNs for spatial data and MLPs for vector data to generate a latent vector $z$.
- Transition Module: Predicts the evolution of the latent state over time based on current control actions.
- Prediction Module: Translates the latent state into observable flow responses, such as $CO_2$ injection rates.
A core innovation is the Joint-Embedding Consistency (JEC) loss. Unlike standard models that attempt to reconstruct full reservoir images, JEC ensures that the predicted future latent state matches the encoded version of the actual future state. This approach maintains temporal grounding over long horizons without the overhead of full-scale reconstruction.
Deep Reinforcement Learning (MSDRL)
The MLD model acts as a "flight simulator" for a Soft Actor-Critic (SAC) agent. The agent interacts with this latent environment to learn an optimal control policy. Because the MLD model is purely mathematical and bypasses physics-based solvers, the agent can simulate decades of operation in seconds.
Experimental Setup
The researchers utilized the industry-standard ECLIPSE simulator to generate training data across two primary cases:
- Deterministic Case: 500 simulation runs of a specific 3D aquifer model to evaluate raw optimization efficiency.
- Generalizable Case: 2,000 simulation runs where permeability, porosity, and fluid properties were varied. This tested the model's ability to handle "unseen" reservoirs, with 1,600 runs used for training and 400 for testing.
Key Results
- Superior Performance: The proposed MSDRL approach achieved a higher NPV than traditional methods like Differential Evolution (DE) and surrogate-assisted evolutionary algorithms.
- Computational Efficiency: The framework reduced required computational resources by over 60%, saving approximately 30 hours of compute time in test cases. Once trained, the MLD model evaluated flow responses $10^4$ times faster than the full physics simulator.
- Zero-Shot Generalization: In the generalizable case, the agent provided significantly better control schemes for brand-new reservoir scenarios compared to unoptimized baselines, demonstrating that the model learned transferable physical principles rather than just memorizing specific maps.
Conclusion
The MLD framework represents a significant step forward in making GCS optimization practical for real-world engineering. By prioritizing latent consistency over full-scale reconstruction and utilizing multimodal inputs, the model achieves both speed and generalization. Future work will focus on handling more complex, "messy" geological distributions and integrating hard non-linear state constraints, such as safety pressure thresholds, directly into the reinforcement learning framework.
Summary updated