AQ-PINNs: Attention-Enhanced Quantum Physics-Informed Neural Networks for Carbon-Efficient Climate Modeling

Siddhant Dutta, Nouhaila Innan, Sadok Ben Yahia, Muhammad ShafiquearXiv 2024 · DOI · arXiv

Introduction

The rapid advancement of AI has introduced a significant environmental challenge: the energy required to train large-scale models is doubling approximately every 3.4 months. This "AI arms race" creates a paradox in climate science, where the computational tools used to model environmental collapse—such as complex fluid dynamics simulations—contribute to the very carbon emissions they seek to mitigate.

AQ-PINNs (Attention-Enhanced Quantum Physics-Informed Neural Networks) address this challenge by fusing quantum computing with physics-based deep learning. This novel architecture aims to provide high-fidelity climate modeling, specifically solving the Navier-Stokes equations for atmospheric and oceanic flow, while significantly reducing the computational burden and carbon footprint.

Methodology

The AQ-PINN framework integrates three core components to achieve efficiency and accuracy:

  1. Physics-Informed Neural Networks (PINNs): Unlike traditional black-box models, PINNs embed physical laws (Partial Differential Equations) directly into the loss function. This ensures the model adheres to fundamental principles like mass conservation.
  2. Quantum Multi-head Self-Attention (QMSA): The attention mechanism is implemented using quantum circuits, where classical data is encoded into quantum states via angle embedding.
  3. Quantum Tensor Networks: To minimize model size, the researchers utilized three tensor network architectures: Matrix Product States (QMPS), Tree Tensor Networks (QTTN), and Multiscale Entanglement Renormalization Ansatz (QMERA). These allow for complex high-dimensional representations with far fewer parameters than classical layers.
AQ-PINN Architecture: The model integrates spatial and temporal inputs through hidden layers to output states, constrained by a hybrid loss function combining MSE and Physics-Informed components.

The study utilized a dataset of 5,000 spatial points across 200 time steps derived from incompressible Navier-Stokes equations. Training involved 30,000 randomly sampled points, optimized using the L-BFGS method and a "Super-Convergence" technique for efficient learning rate discovery.

Results and Discussion

The primary advantage of AQ-PINNs lies in their "compact representation." By leveraging quantum entanglement, these models process information with a fraction of the parameters required by classical neurons. Furthermore, enforcing physical laws acts as a powerful regularizer, preventing the model from memorizing noise.

Benchmarking against classical Attention-enhanced PINNs (A-PINNs) revealed that quantum variants not only reduced complexity but also improved predictive performance.

Comparison of model performance showing significant parameter reduction and improved test loss for all quantum variants.

Key findings include:

  • Parameter Efficiency: The QMPS variant achieved a 63.29% reduction in parameters, while QTTN and QMERA achieved 55.28% and 51.51% reductions, respectively.
  • Superior Accuracy: All quantum models achieved lower test losses (ranging from 0.0593 to 0.0609) compared to the classical baseline (0.0631).
  • Robust Convergence: Despite the reduced parameter count, the models maintained stable convergence rates, effectively solving complex PDEs.

Conclusion

AQ-PINNs demonstrate that "Green AI" does not necessitate a sacrifice in performance. By utilizing quantum tensor networks, the researchers achieved over 50% parameter reduction while exceeding classical accuracy. Future work will focus on scaling these models to 3D global climate systems, improving the interpretability of quantum attention, and refining predictions for extreme weather events to better inform policy decisions.

Summary updated

BibTeX
@misc{dutta2024aq,
  author = {Siddhant Dutta and Nouhaila Innan and Sadok Ben Yahia and Muhammad Shafique},
  title = {{AQ-PINNs: Attention-Enhanced Quantum Physics-Informed Neural Networks for Carbon-Efficient Climate Modeling}},
  year = {2024},
  eprint = {2409.01626},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.01626},
}
Figure