Abstract
Convolutional Neural Networks (CNNs) are powerful tools for physics-informed simulations but often suffer from high parameter counts, limiting their deployment in real-time or edge environments. This paper proposes an a priori compression method using tensor decomposition to replace high-dimensional kernels with sequences of 1D filters. Applied to a wave simulator, this approach reduces parameters from 7 million to 1 million while maintaining accuracy and improving generalization.
Introduction
Physics-informed simulations, such as predicting wave propagation through complex structures, require high speed and memory efficiency. While CNNs excel at processing grid-like data from Finite Element (FE) models, standard architectures are often prohibitively "heavy."
Traditional compression techniques, like pruning and quantization, are typically applied a posteriori—after training a large model. This paper introduces a "compressed by design" approach, tackling the "curse of dimensionality" head-on for 3D tensors (time + 2D space) common in wave physics. By enforcing structural constraints before training, the authors aim to create models that are inherently efficient.
Methodology: A Priori Decomposition
The core innovation is the use of Canonical Polyadic Decomposition (CPD) and Singular Value Decomposition (SVD) to break down standard $N$-way convolutional kernels (e.g., $3 \times 3 \times 3$) into a succession of 1D kernels. This technique addresses the "curse of dimensionality" by assuming that these high-dimensional filters can be represented as a sum of rank-1 tensors.
Key Technical Components
- Decomposed Convolutional Layers: By replacing 2D or 3D convolutions with sequences of 1D convolutions, data reshaping, and transposing, the authors shift parameter complexity from exponential to linear relative to the dimension. For instance, a $K \times K \times K$ kernel with $C$ input and $F$ output channels has $K^3 \cdot C \cdot F$ parameters. The decomposed version significantly reduces this footprint.
- Weight Sharing (Layer Coupling): For simulators predicting multiple related fields (e.g., displacement and velocity), initial feature extraction layers are shared. This architectural choice is based on the observation that different physical fields often share underlying spatial features, allowing for a 50% reduction in parameters for those specific layers.
- Euler Regularization: A "time regularization" term is added to the loss function to enforce physical consistency. Using an Euler finite difference scheme, it forces the predicted velocity to match the time derivative of the predicted displacement. This ensures that the model doesn't just minimize statistical error but also adheres to the fundamental kinematic relationships of the wave equation.
Experimental Setup
The approach was validated using synthetic data from a 2D Wave Equation solver. The dataset consisted of 3D tensors representing time-space evolution (2 spatial indices, 1 temporal). A focused dataset of 100 training and 25 test samples was generated using Latin Hypercube Sampling to ensure coverage of various source points and frequencies.
Results and Insights
The proposed decomposed approach achieved a massive reduction in parameters, dropping from approximately 7 million in a classical CNN to just 1 million.
Key Findings
- Accuracy and Generalization: Decomposed models (specifically
Conv3DandConv2.5Dwith Batch Normalization) achieved errors comparable to full-sized models. Interestingly, the compressed models showed reduced overfitting due to fewer degrees of freedom, making them more robust to noise. - Dimensional Separability: The results suggest that high-dimensional kernels in physics simulations are often dimensionally separable. Enforcing this structure a priori forces the model to learn more efficient representations from the start.
- Linear Growth: In the "Conv2.5D" approach, complexity grows linearly with dimensions rather than exponentially, offering significant advantages for high-dimensional physics data.
Discussion and Future Work
While the parameter count is significantly lower, current training times per epoch are higher due to the lack of hardware-level optimization for custom "reshape-and-transpose" sequences in standard deep learning libraries like PyTorch or TensorFlow.
Future work will focus on scaling this approach to 3D spatial simulations, where the benefits of linear complexity growth will be even more pronounced. Additionally, the authors plan to integrate more complex physical laws beyond simple Euler regularization, potentially moving toward fully intrusive physics-informed neural networks.
Summary updated