Variable-Input Deep Operator Networks

Michael Prasthofer, Tim De Ryck, Siddhartha MishraarXiv 2022 · DOI · arXiv

Introduction

Operator learning is an emerging field focused on designing neural networks that learn mappings between infinite-dimensional function spaces. This has profound implications for scientific computing, such as predicting fluid flow states from initial conditions. However, existing state-of-the-art architectures like Fourier Neural Operators (FNO) and DeepONets face a significant practical limitation: they assume input data is collected at fixed sensor locations and in constant quantities across all samples.

In real-world scenarios, data is often "messy"—sensors fail, measurement devices are moved, or simulations use varying mesh resolutions. FNOs are restricted to rigid Cartesian grids, while DeepONets require identical irregular grids for every training and test sample. To address these constraints, we introduce Variable-Input Deep Operator Networks (VIDON), a flexible, sensor-agnostic framework capable of handling variable, random, and missing data points without compromising accuracy.

Methodology

VIDON extends the DeepONet architecture by replacing the standard branch network with a structure designed to process sets of points rather than fixed-length vectors.

Architecture Components

  1. Input Encoding: The input is treated as a set of pairs $(x_j, u(x_j))$, where $x_j$ is a sensor location and $u(x_j)$ is its measured value. These are processed via separate Coordinate and Value Encoders (MLPs).
  2. Permutation-Invariant Aggregation: To handle a variable number of sensors, the model must be permutation-invariant. VIDON employs a multi-head attention mechanism with linear complexity $O(m)$ relative to the number of sensors $m$, avoiding the quadratic cost of standard Transformers.
  3. Branch and Trunk Integration: The aggregated branch information is combined with a "trunk net" representing the output domain, following the fundamental DeepONet design.
Comparison of sensor configurations across different operator learning models. VIDON handles variable random locations, whereas FNO and DeepONet are constrained to fixed grids or identical irregular sets.

Theoretical Foundation

The framework is supported by a Universal Approximation Theorem, proving VIDON can approximate any continuous operator mapping into Sobolev spaces. For common PDEs, the network size scales polynomially with the desired accuracy.

Experimental Evaluation

VIDON was evaluated on three benchmark physics problems: Darcy Flow (elliptic PDE), the Allen-Cahn Equation (reaction-diffusion), and the Navier-Stokes Equations (incompressible fluid dynamics). Each was tested across six sensor configurations, from regular grids to variable random locations.

Key Results

  • Robustness to Irregularity: VIDON was the only model capable of handling "Missing Data," "Perturbed Grid," and "Variable Random Locations." For Darcy flow, error remained below 2.6% even with completely randomized sensor placement per sample.
  • Competitive Accuracy: On standard regular grids, VIDON outperformed DeepONet (e.g., 1.29% vs 1.48% error on Darcy flow).
  • Complex Dynamics: On Navier-Stokes simulations, VIDON achieved a test error of ~5.2% on regular grids and maintained ~8.3% error under randomized sensor conditions.

Conclusion

VIDON represents a significant step toward deploying operator learning in practical, data-constrained environments. By treating inputs as sets rather than fixed discretizations, it achieves unprecedented flexibility. While the architecture is more computationally intensive than its predecessors, future work on pre-training and fixed positional encodings promises to enhance its efficiency for large-scale 3D industrial applications.

Summary updated

Abstract

Existing architectures for operator learning require that the number and locations of sensors (where the input functions are evaluated) remain the same across all training and test samples, significantly restricting the range of their applicability. We address this issue by proposing a novel operator learning framework, termed Variable-Input Deep Operator Network (VIDON), which allows for random sensors whose number and locations can vary across samples. VIDON is invariant to permutations of sensor locations and is proved to be universal in approximating a class of continuous operators. We also prove that VIDON can efficiently approximate operators arising in PDEs. Numerical experiments with a diverse set of PDEs are presented to illustrate the robust performance of VIDON in learning operators.
BibTeX
@misc{prasthofer2022variable,
  author = {Michael Prasthofer and Tim De Ryck and Siddhartha Mishra},
  title = {{Variable-Input Deep Operator Networks}},
  year = {2022},
  eprint = {2205.11404},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.11404},
}
Figure