Abstract
Modeling physical systems governed by Partial Differential Equations (PDEs) is critical for modern engineering, yet current deep learning approaches like Fourier Neural Operators (FNOs) often struggle with temporal memory and noise sensitivity. This paper introduces Fourier-RNNs (F-RNNs), a hybrid architecture that embeds Fourier layers within a recurrent framework. By leveraging the hidden state as a temporal filter in frequency space, F-RNNs demonstrate superior robustness and accuracy in predicting complex fluid dynamics from noisy data compared to standard FNOs and convolutional RNNs.
Introduction
Modeling physical systems described by Partial Differential Equations (PDEs) is a cornerstone of modern engineering. Recently, Fourier Neural Operators (FNOs) have emerged as a powerful tool because they map between infinite-dimensional function spaces rather than finite vectors, making them highly efficient at learning underlying physical operators.
However, FNOs possess two significant limitations. First, they are Markovian, typically predicting the next state based solely on the current state without memory of the distant past. Second, they exhibit high noise sensitivity; without temporal context, they struggle to distinguish between actual physical signals and random noise. While standard Recurrent Neural Networks (RNNs) provide memory via a hidden state, they operate in Euclidean space, which often fails to capture the complex, periodic behaviors inherent in physics data. To address these gaps, researchers from the UK Atomic Energy Authority have introduced the Fourier-RNN (F-RNN).
Methodology: The Fourier-RNN Cell
The core innovation of this work is the Fourier-RNN cell. In a standard RNN, the hidden state is updated using linear transformations. The F-RNN replaces these transformations with Fourier Layers to process data in the frequency domain.
As shown in the architecture above, each F-RNN cell utilizes Fourier layers that perform a Fourier Transform, apply a learnable weight matrix to the frequency modes, and then perform an Inverse Fourier Transform back to the spatial domain. By embedding these operators within a recurrent structure, the model learns to update its "memory" (hidden state) in Fourier space. This allows the hidden state to act as a temporal filter, using historical context to filter out noise that does not align with learned physical patterns.
Experimental Setup
The model was evaluated on three synthetic physics benchmarks with varying levels of Gaussian noise (0% to 25% variance):
- 2D Wave Equation: Modeling wave propagation over time.
- 2D Navier-Stokes (Laminar): Smooth fluid flow with kinematic viscosity $\nu=10^{-3}$.
- 2D Navier-Stokes (Turbulent): Chaotic fluid flow with lower viscosity $\nu=10^{-5}$.
The models were trained on 800 simulations and tested on 200 using an Nvidia V100 GPU, maintaining a parameter count (~4.2M) similar to a standard FNO for fair comparison.
Key Results and Discussion
The F-RNN demonstrated significant superiority in handling noisy data. In the Navier-Stokes laminar case with 25% noise, the F-RNN achieved a Mean Squared Error (MSE) of 0.001499, nearly ten times lower than the standard FNO's error of 0.01368. In the highly challenging turbulent regime, the F-RNN maintained stable predictions while Convolutional-RNNs (C-RNNs) failed to converge.
Qualitative analysis (see above) reveals that FNO predictions become increasingly "grainy" and distorted as noise levels rise. In contrast, the F-RNN produces sharp, accurate predictions. This confirms the "aha moment" of the research: the hidden state effectively filters the noisy input against the system's learned trajectory. Furthermore, because it uses neural operators, the F-RNN learns the underlying physical operator, making it robust across different grid discretizations.
Conclusion and Future Work
Fourier-RNNs represent a major step toward building "digital twins" for complex physical systems, such as fusion reactors, where sensors provide noisy, real-time data. While this study utilized synthetic Gaussian noise, future work will focus on testing F-RNNs on experimental data from real-world systems. Additionally, further optimization of hidden state initialization could enhance the model's sensitivity to initial field distributions.
Summary updated