Cauchy Random Features for Operator Learning in Sobolev Space

Chunyang Liao, Deanna Needell, Hayden Schaeffer

Abstract

Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven by variants of deep neural networks such as the Deep Operator Network (DeepONet) and Fourier Neural Operator (FNO), the theoretical guarantees are often in the form of a universal approximation property. However, the existence theorems do not guarantee that an accurate operator network is obtainable in practice. Motivated by the recent kernel-based operator learning framework, we propose a random feature operator learning method with theoretical guarantees and error bounds. The random feature method can be viewed as a randomized approximation of a kernel method, which significantly reduces the computation requirements for training. We provide a generalization error analysis for our proposed random feature operator learning method along with comprehensive numerical results. Compared to kernel-based methods and neural network methods, the proposed method can obtain similar or better test errors across benchmark examples with significantly reduced training times. An additional advantage is that our implementation is simple and does not require costly computational resources, such as GPUs.

Introduction

In scientific computing, the transition from solving specific instances of Partial Differential Equations (PDEs) to learning the underlying operators represents a significant paradigm shift. Operator learning aims to find a mapping that takes input functions (such as initial conditions or source terms) and produces the corresponding solution functions. While deep learning architectures like DeepONet and FNO have shown empirical success, they often function as "black boxes" with limited theoretical guarantees beyond existence theorems. Furthermore, training these models requires substantial computational power and large datasets.

Traditional kernel methods offer a more rigorous mathematical foundation but suffer from poor scalability, typically requiring $O(m^3)$ complexity. This paper introduces a "middle way": a Random Feature Operator Learning (RFM) framework that combines the speed of randomized methods with the theoretical robustness of kernel methods, specifically optimized for the smoothness properties of PDE solutions.

Methodology and Framework

The proposed framework bridges the gap between infinite-dimensional function spaces and finite-dimensional computation through a commutative diagram involving sampling and recovery maps.

Figure 1: The operator learning framework uses sampling (SU) and recovery (RU/RV) maps to bridge the gap between infinite-dimensional function spaces and finite-dimensional computation.

As illustrated in Figure 1, the operator $G$ maps between function spaces $\mathcal{U}$ and $\mathcal{V}$. To implement this computationally, we use sampling maps ($S_U$) to project functions into $\mathbb{R}^n$ and recovery maps ($R_U, R_V$) to lift them back. The core of our innovation lies in the approximation of the finite-dimensional mapping $f$ using Cauchy Random Features.

Cauchy Random Features

Instead of iteratively learning every weight in a deep network, we randomly sample "frequencies" from a tensor-product Cauchy distribution. This choice is mathematically motivated: using Cauchy features is equivalent to working within a Reproducing Kernel Hilbert Space (RKHS) corresponding to the Laplace kernel. This alignment is ideal for Sobolev spaces of mixed smoothness, which are the natural mathematical homes for many PDE solutions.

Training in this framework is reduced to a min-norm interpolation problem—essentially solving a linear system. This convex optimization is not only faster than backpropagation but also "embarrassingly parallel," allowing for high-efficiency training on standard CPUs without the need for specialized GPU hardware.

Theoretical Guarantees

Unlike many deep learning approaches, the Cauchy RFM provides provable performance metrics. We establish a rigorous generalization error bound showing that the error decays at a rate of $1/\sqrt{N}$, where $N$ is the number of random features. This provides a clear roadmap for scaling the model to achieve desired accuracy levels. Additionally, the model is discretization-invariant; it can be trained on one mesh resolution and applied to another without loss of structural integrity, a vital feature for practical engineering applications.

Experimental Results

The method was evaluated against standard benchmarks, including 1D Advection, Burgers’ Equation, 2D Darcy Flow, and the Navier-Stokes Equation.

  • Accuracy: On the Advection III benchmark, which features challenging discontinuities, the Cauchy RFM achieved a relative test error of 0.14%, outperforming both RBF Kernels (0.17%) and Gaussian RFM (0.22%).
  • Efficiency: In the Navier-Stokes benchmark with 20,000 samples, the method matched the accuracy of FNO and DeepONet while significantly reducing training overhead.
  • Speed: Training times were reduced to seconds. For the Burgers' equation, Cauchy RFM reached a 3.85% error in just 3.1 seconds, whereas traditional kernel methods were either slower or less accurate within the same time budget.

Conclusion

Cauchy Random Features offer a compelling alternative to deep neural networks for operator learning. By leveraging the heavy-tailed nature of the Cauchy distribution, we better capture the smoothness requirements of physical systems while maintaining a simple, linear training process. Future work will focus on extending this framework to handle noisy sensor data and more complex, non-rectangular physical geometries.

Summary updated

Abstract

Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven by variants of deep neural networks such as the Deep Operator Network and Fourier Neural Operator, the theoretical guarantees are often in the form of a universal approximation property. However, the existence theorems do not guarantee that an accurate operator network is obtainable in practice. Motivated by the recent kernel-based operator learning framework, we propose a random feature operator learning method with theoretical guarantees and error bounds. The random feature method can be viewed as a randomized approximation of a kernel method, which significantly reduces the computation requirements for training. We provide a generalization error analysis for our proposed random feature operator learning method along with comprehensive numerical results. Compared to kernel-based method and neural network methods, the proposed method can obtain similar or better test errors across benchmarks examples with significantly reduced training times. An additional advantages it that our implementation is simple and does require costly computational resources, such as GPU.
BibTeX
@misc{liao2025cauchy,
  author = {Chunyang Liao and Deanna Needell and Hayden Schaeffer},
  title = {{Cauchy Random Features for Operator Learning in Sobolev Space}},
  year = {2025},
  eprint = {2503.00300},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.00300},
}
Figure