From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems

Xin Li, Jingdong Zhang, Qunxi Zhu, Chengli Zhao, Xue Zhang, Xiaojun Duan, Wei Lin

Abstract

Modeling complex dynamical systems—ranging from fluid dynamics to planetary rotation—is a fundamental challenge in scientific computing. While Neural Ordinary Differential Equations (NODEs) offer a continuous-time framework for these tasks, they are frequently constrained by high computational costs and sensitivity to noise. This paper introduces FNODEs, a framework that integrates Fourier analysis with simulation-free flow matching. By leveraging Fourier series for robust gradient estimation, FNODEs achieve training speeds up to 75 times faster than standard NODEs while maintaining high accuracy and robustness in the presence of significant data noise.

1. Introduction

The ability to model and predict the evolution of complex systems is essential for fields as diverse as meteorology, engineering, and geophysics. Neural Ordinary Differential Equations (NODEs) have emerged as a prominent tool for learning these dynamics directly from data. However, standard NODEs face two primary hurdles: the computational intensity of simulation-based training and the instability of calculating derivatives from sparse, noisy observations.

Traditional training requires repeatedly solving differential equations to compare model predictions against data, a process that is both memory-intensive and slow. Furthermore, real-world data is rarely pristine; numerical differentiation of noisy signals often leads to erratic gradient estimates that hinder convergence. FNODEs address these limitations by replacing the simulation loop with a "flow matching" objective and utilizing Fourier analysis as a differentiable, noise-resilient smoothing mechanism.

2. The FNODE Framework

The FNODE architecture is built upon three technical pillars designed to streamline the learning of vector fields.

2.1 Simulation-Free Flow Matching

Unlike standard NODEs, which require an ODE solver during every training step, FNODEs utilize a flow matching approach. This method estimates the target vector field (the "flow") directly from the data. By training the neural network to match these pre-estimated gradients, the framework eliminates the need for expensive numerical integration during the training phase, significantly reducing the computational overhead.

2.2 Fourier Gradient Estimation

To obtain reliable gradients from noisy or sparse time-series data, the authors employ Fourier analysis. By representing the data as a sum of trigonometric basis functions (a Fourier series), the system can calculate derivatives analytically rather than numerically. This approach serves as a natural low-pass filter, effectively smoothing out high-frequency noise while providing a mathematically rigorous way to estimate both temporal gradients and high-order spatial gradients for Partial Differential Equations (PDEs).

2.3 Iterative Data Refinement

FNODEs incorporate a positive feedback loop to enhance model robustness. Once an initial model is trained, it can be used to generate "cleaner" synthetic data points. These refined points are then fed back into the Fourier estimator to produce even more accurate gradient targets. This self-improving cycle allows the model to progressively filter out noise and converge on the true underlying dynamics of the system.

3. Experimental Evaluation

The researchers benchmarked FNODEs against several state-of-the-art methods across a variety of synthetic and real-world datasets.

3.1 Computational Efficiency

The most striking result was the reduction in training time. In experiments involving the 2-D Navier-Stokes equations—a benchmark for fluid flow—FNODEs completed training in 283 seconds. In contrast, standard NODEs required over 21,000 seconds to reach convergence, representing a nearly 75-fold speedup. This efficiency stems directly from the simulation-free nature of the flow matching objective.

3.2 Robustness and Accuracy

FNODEs demonstrated superior performance in high-noise environments. When subjected to 10% Gaussian noise, FNODEs maintained a low Mean Squared Error (MSE), while competing models like DeepAR and standard NODEs experienced significant performance degradation. Furthermore, the model showed high data efficiency, outperforming Neural Operators (e.g., FNO and DeepONet) even when the training set was limited to as few as 10 samples.

3.3 Real-World Application: Polar Motion

The framework was applied to the Polar Motion dataset, which tracks the Earth's instantaneous rotation axis. This system is influenced by complex, multi-scale factors including atmospheric and oceanic angular momentum. FNODEs successfully captured these intricate dynamics, proving the framework's utility beyond idealized physics simulations.

4. Discussion and Future Work

A key advantage of the FNODE approach is resolution invariance. Because the model learns a continuous vector field, it can perform "zero-shot super-resolution," where a model trained on low-resolution spatial data can accurately predict dynamics at much higher resolutions without further training.

Despite these strengths, the authors acknowledge certain limitations. The reliance on truncated Fourier series may struggle to capture the nuances of hyper-complex or extremely high-dimensional systems. Future research directions include integrating Koopman theory to better linearize chaotic dynamics and incorporating uncertainty quantification to improve the reliability of predictions in unpredictable environments.

5. Conclusion

FNODEs represent a significant step forward in the practical application of Neural ODEs. By bridging classical Fourier analysis with modern flow matching, the framework overcomes the traditional trade-offs between speed, accuracy, and noise robustness. This approach provides a scalable and efficient pathway for modeling the complex, chaotic systems that define our physical world.

Summary updated

BibTeX
@inproceedings{li2024from,
  author = {Xin Li and Jingdong Zhang and Qunxi Zhu and Chengli Zhao and Xue Zhang and Xiaojun Duan and Wei Lin},
  title = {{From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=E4qjDAdVte},
}
Figure