Physics-Informed Neuro-Evolution (PINE): A Survey and Prospects

Jian Cheng Wong, Abhishek Gupta, Chin Chun Ooi, Pao-Hsiung Chiu, Jiao Liu, Yew-Soon Ong

TL;DR

This paper explores the exciting frontier of Physics-Informed Neural Networks (PINNs) for scientific machine learning, highlighting their potential to overcome data scarcity. It argues that traditional gradient-based optimization methods struggle with PINNs' complex loss landscapes, proposing evolutionary algorithms (EAs) as a powerful alternative for optimization, architecture search, and meta-learning to achieve more generalizable and efficient models.

What Problem Does This Paper Address?

Deep learning models typically require vast amounts of data to perform well. However, in scientific domains like physics, chemistry, and engineering, acquiring such data is often expensive, time-consuming, or even impossible. This data scarcity limits the applicability of standard deep learning. Physics-Informed Neural Networks (PINNs) offer a solution by incorporating known physical laws (expressed as differential equations) directly into the neural network's training objective. This allows PINNs to learn effectively even with limited data.

Despite their promise, PINNs face significant challenges, particularly in:

  • Optimization: The combined loss function of PINNs (data loss + physics loss) creates a highly complex, non-convex landscape with numerous local minima and conflicting gradient signals, making training slow and prone to getting stuck.
  • Generalization: PINNs trained for a specific problem instance often struggle to generalize to new scenarios (e.g., different parameters or initial conditions) without extensive retraining.

This paper addresses these challenges by investigating how evolutionary algorithms (EAs), which are gradient-free and explore search spaces more broadly, can be leveraged to improve PINN optimization and generalization.

Methods

The paper advocates for the use of Evolutionary Algorithms (EAs) as a key enabler for advancing PINNs. The core ideas revolve around:

  • Gradient-Free Optimization: EAs, unlike gradient descent, do not rely on derivative information. This makes them less susceptible to issues like vanishing gradients and local minima traps, which plague PINN training.
  • Neuroevolution: EAs can be used to evolve not only the weights of a neural network but also its architecture, creating "bespoke" network designs tailored for specific physics problems.
  • Meta-Learning with EAs: EAs can act as meta-learners, discovering generalizable PINN models that can be quickly adapted to new, unseen tasks with minimal fine-tuning. This is achieved through bilevel optimization where an EA searches for optimal PINN configurations.
  • Synergy with Gradient Descent: The paper also suggests combining EAs with gradient descent (memetic algorithms) to leverage the exploration capabilities of EAs and the rapid convergence of gradient descent.
  • Pareto Optimization: For multi-objective PINN training (balancing data and physics losses), Multi-Objective EAs (MOEAs) are proposed as they can find a set of optimal trade-off solutions without requiring explicit scalarization of loss terms.

Datasets

Not discussed in the paper. The paper focuses on the methodologies and theoretical underpinnings of PINNs and their optimization, rather than specific empirical evaluations on particular datasets. The examples used are illustrative of scientific simulation and discovery problems.

Key Insights and Ideas

  • PINNs are powerful but optimization-hungry: The physics-informed loss landscape is inherently more complex than standard data-driven models, posing significant optimization hurdles.
  • Gradient descent has limitations for PINNs: Issues like local minima, spectral bias, and sensitivity to sample size make gradient-based methods struggle to find truly accurate and physically compliant solutions.
  • Evolutionary Algorithms are a natural fit: EAs' gradient-free nature, population-based search, and ability to explore diverse solutions make them well-suited for navigating the rugged PINN loss landscape.
  • EAs can discover better architectures: Beyond just optimizing weights, EAs can evolve neural network architectures that are inherently better suited for physics-informed learning.
  • Meta-learning with EAs unlocks generalization: By using EAs to learn how to learn, PINNs can be made to generalize across a distribution of related physics tasks, a crucial step towards "Physical AI."
  • Baldwinian evolution offers a promising path: The concept of "neural Baldwinism" leverages evolutionary principles to pre-wire learning biases into networks, enabling rapid adaptation to new tasks.

Key Results

The paper is primarily a survey and a call for future research, rather than a presentation of new empirical results. However, it highlights findings from recent literature that support its claims:

  • Improved Optimization: Studies cited show that evolutionary approaches can help overcome local minima and achieve better convergence compared to standard gradient descent methods for PINNs.
  • Faster Training and Better Accuracy: Examples like Baldwinian-PINN demonstrate the ability to solve new PDE problems orders of magnitude faster and with improved accuracy compared to other meta-learning approaches. For instance, in diffusion-reaction experiments, Baldwinian-PINN achieved 70x better accuracy with 700x less computation time.
  • Generalization Capabilities: Baldwinian-PINN has shown effective generalization to unseen PDE families and longer time domains, even without explicit parameterization of new scenarios.

Limitations and Future Work

The paper acknowledges several limitations and points towards future research directions:

  • Sample Complexity of EAs: A primary challenge for EAs is their potentially high sample complexity, requiring many evaluations. Future work should focus on developing more sample-efficient EA variants.
  • Synergistic Frameworks: Developing effective hybrid algorithms that seamlessly meld gradient-based and gradient-free methods (like EAs) remains a significant research challenge. Questions include how to balance their respective strengths and when to switch between them.
  • Hyperparameter Tuning for EAs: Applying EAs effectively requires careful tuning of numerous hyperparameters. Research into automatic hyperparameter configuration for EAs is crucial.
  • Scalability of EAs: While EAs can scale well with parallelization, their performance can still be a concern for very high-dimensional parameter spaces. Exploring techniques like cooperative coevolution and memetic algorithms is suggested.
  • Meta-learning for PINNs: While promising, meta-learning for PINNs is still nascent. Further research is needed to address issues like non-convergence in meta-learning algorithms and the challenge of adapting to tasks that require extensive training.
  • Evolving Interpretable Models: Beyond just solving problems, there's a need for PINNs that produce interpretable models. Evolutionary approaches could play a role in discovering such models.
  • Neural Architecture Search (NAS) for PINNs: Automating the design of PINN architectures tailored to specific physics problems using EAs is an open area with significant potential.
  • Transfer Learning for PINNs: Developing robust methods for transfer learning, especially identifying relevant source tasks and preventing negative transfer, is an ongoing research direction.

Summary updated

Abstract

Deep learning models trained on finite data lack a complete understanding of the physical world. On the other hand, physics-informed neural networks (PINNs) are infused with such knowledge through the incorporation of mathematically expressible laws of nature into their training loss function. By complying with physical laws, PINNs provide advantages over purely data-driven models in limited-data regimes and present as a promising route towards Physical AI. This feature has propelled them to the forefront of scientific machine learning, a domain characterized by scarce and costly data. However, the vision of accurate physics-informed learning comes with significant challenges. This work examines PINNs in terms of model optimization and generalization, shedding light on the need for new algorithmic advances to overcome issues pertaining to the training speed, precision, and generalizability of today's PINN models. Of particular interest are gradient-free evolutionary algorithms (EAs) for optimizing the uniquely complex loss landscapes arising in PINN training. Methods synergizing gradient descent and EAs for discovering bespoke neural architectures and balancing multiple terms in physics-informed learning objectives are positioned as important avenues for future research. Another exciting track is to cast EAs as a meta-learner of generalizable PINN models. To substantiate these proposed avenues, we further highlight results from recent literature to showcase the early success of such approaches in addressing the aforementioned challenges in PINN optimization and generalization.
BibTeX
@inproceedings{wong2026physics,
  author = {Jian Cheng Wong and Abhishek Gupta and Chin Chun Ooi and Pao-Hsiung Chiu and Jiao Liu and Yew-Soon Ong},
  title = {{Physics-Informed Neuro-Evolution (PINE): A Survey and Prospects}},
  booktitle = {IEEE Comput. Intell. Mag},
  year = {2026},
  url = {https://openreview.net/forum?id=GA6jyyGUgT},
}

Loading related content...

Figure