RotEqNet: Rotation-equivariant network for fluid systems with symmetric high-order tensors

Guang Lin, Liyao Gao, Yifan Du, Hongshan Li, Guang Lin 0001Journal of Computational Physics 2022 · DOI

TL;DR

This paper introduces RotEqNet, a novel neural network architecture designed to handle high-order tensors in fluid systems while respecting their inherent rotational symmetry. By developing a "position standardization" algorithm, RotEqNet achieves significant error reductions and improved generalization compared to traditional machine learning methods, offering a theoretically guaranteed approach to modeling rotation-symmetric physical phenomena.

What Problem Does This Paper Address?

Many physical systems, particularly in fluid mechanics, exhibit rotational symmetry. However, standard machine learning models often fail to inherently respect this symmetry, leading to suboptimal performance and a lack of theoretical guarantees. This is especially problematic when dealing with high-order tensors, which are common in describing complex physical phenomena like turbulent flows. Existing methods to enforce rotational symmetry, such as data augmentation, lack theoretical guarantees or are computationally prohibitive. This paper aims to bridge this gap by proposing a principled, theoretically grounded method to build rotation-equivariant models for high-order tensors in fluid systems.

Methods

The core of RotEqNet lies in its position standardization algorithm. This algorithm transforms an input tensor into a "standard position" that is rotation-invariant. This is achieved by:

  • Tensor Contraction: For even-order tensors, contraction is used to reduce them to second-order tensors, which can then be diagonalized using their orthonormal eigenvectors to find a rotation matrix. For odd-order tensors, a similar process is followed, but with an additional step of QR factorization.
  • Rotation Invariant Extraction: The algorithm leverages these derived rotation matrices to transform the original tensor into its standard, rotation-invariant form.
  • Kernel Predictor: A standard machine learning model (e.g., neural network, random forest) then acts as a "kernel predictor" that operates only on these standardized tensors. This significantly reduces the complexity of the learning problem by removing the rotational variability.
  • Rotation-Equivariant Architecture: The overall RotEqNet architecture applies the standardization, feeds the standardized tensor to the kernel predictor, and then uses the inverse rotation to transform the output back to the original coordinate system, ensuring the final prediction is rotation-equivariant.

Datasets

The paper utilizes simulated data generated from various fluid dynamics equations for its case studies. These include:

  • Newtonian fluid (second-order linear and third-order): Data generated based on equations describing Newtonian fluid stress.
  • Large Eddy Simulation (LES) (second-order nonlinear): Data generated from a subgrid model for turbulent flow.
  • Electrostriction (fourth-order linear): Data generated from a model describing the relationship between electric fields and strain in materials.

For each case study, datasets were generated with varying numbers of training points (10,000 to 100,000), with 85% used for training and 15% for testing. Additionally, a set of 10,000 random rotation matrices was used to evaluate the rotation-equivariant property.

Key Insights and Ideas

  • Tensor Contraction Preserves Rotation: A crucial theoretical insight is that tensor contraction is compatible with rotation operations, meaning contraction of a rotated tensor yields the same result as rotating the contracted tensor. This is fundamental to the position standardization algorithm.
  • Standard Position Simplifies Learning: By transforming tensors into a rotation-invariant "standard position," the learning problem is drastically simplified. The kernel predictor no longer needs to learn patterns across all possible rotations, leading to more efficient learning and better generalization.
  • Theoretical Guarantee of Equivariance: RotEqNet provides a theoretical guarantee of rotation-equivariance, which is a significant advancement over heuristic methods like data augmentation. This guarantee stems from the properties of the position standardization algorithm and the overall network architecture.
  • Neural Networks as Superior Kernel Predictors: Across multiple case studies, Neural Networks consistently outperformed Random Forests as kernel predictors, highlighting their flexibility and ability to capture complex patterns, especially when dealing with higher-order tensors.

Key Results

RotEqNet demonstrated significant improvements across various case studies:

  • Error Reduction:
    • Newtonian fluid (second-order linear): 99.60% testing error reduction.
  • Large Eddy Simulation (second-order nonlinear): 52.77% testing error reduction.
  • Newtonian fluid (third-order): 15.62% testing error reduction.
  • Electrostriction (fourth-order): 54.63% testing error reduction.
  • Rotation-Equivariance: The proposed method showed a substantial reduction in rotation error ($E_D$) compared to baseline methods, indicating its ability to maintain consistent predictions across different orientations. For example, in the second-order nonlinear case, $E_D$ was reduced by 73.55% with RotEqNet.
  • Generalization: RotEqNet exhibited better generalization capabilities, with training and testing errors being closer, indicating less overfitting compared to baseline models trained on raw data.

Limitations and Future Work

  • Definition of Standard Position for Odd Tensors: The paper notes that the current definition of standard position for odd-order tensors is not as performant as for even-order tensors. Revising this definition is suggested as a future research direction.
  • Generalization to Other Groups: The current work focuses on rotation symmetry. Extending the framework to other group-equivariant properties like scaling and translation is a promising avenue.
  • Non-Symmetric Systems: The current RotEqNet is designed for symmetric systems. Adapting it to handle non-symmetric systems, potentially through techniques like dealing with PPT matrices, is a future research goal.
  • Kernel Predictor Choice: While Neural Networks performed well, the choice of kernel predictor can significantly impact results. Further exploration of different kernel predictors and their suitability for various physical systems is warranted.

Summary updated

BibTeX
@article{lin2022roteqnet,
  author = {Guang Lin and Liyao Gao and Yifan Du and Hongshan Li and Guang Lin},
  title = {{RotEqNet: Rotation-equivariant network for fluid systems with symmetric high-order tensors}},
  journal = {Journal of Computational Physics},
  year = {2022},
  doi = {10.1016/j.jcp.2022.111205},
}
Figure