Flowers: A Warp Drive for Neural PDE Solvers

Till Muser, Alexandra Spitzer, Matti Lassas, Maarten V. de Hoop, Ivan Dokmanić

Abstract

We introduce Flowers, a neural architecture for learning PDE solution operators built entirely from multihead warps. Aside from pointwise channel mixing and a multiscale scaffold, Flowers use no Fourier multipliers, no dot-product attention, and no convolutional mixing. Each head predicts a displacement field and warps the mixed input features. Motivated by physics and computational efficiency, displacements are predicted pointwise, without any spatial aggregation, and nonlocality enters only through sparse sampling at source coordinates, one per head. Stacking warps in multiscale residual blocks yields Flowers, which implement adaptive, global interactions at linear cost. Flowers achieve excellent performance on a broad suite of 2D and 3D time-dependent PDE benchmarks, particularly flows and waves. A compact 17M-parameter model consistently outperforms Fourier, convolution, and attention-based baselines of similar size, while a 150M-parameter variant improves over recent transformer-based foundation models.

Introduction

Deep learning has revolutionized the creation of "surrogate solvers"—AI models that predict physical systems like weather or aerodynamics significantly faster than traditional numerical simulations. However, current backbones for these solvers face a trilemma: convolutions are limited by local receptive fields; Fourier Neural Operators (FNOs) provide global coupling but treat all interactions as equally likely; and Transformers suffer from quadratic scaling that makes high-resolution 3D simulations prohibitively expensive.

Flowers addresses this gap by mimicking how information naturally travels along physical "characteristics" or rays. By replacing dense global interactions with learned coordinate warps, Flowers achieves state-of-the-art accuracy while maintaining linear computational scaling.

The Flowers Architecture

The core innovation is the SELFWARP layer. Instead of using sliding windows or global frequency filters, the model predicts a displacement field for every point in the simulation.

The Flowers architecture: (a-b) Input features are processed pointwise. (c-d) Networks predict affine mixing and displacement fields. (e-f) The displacement field defines a warp. (g) The warp is applied to pull features from source coordinates. (h) Multihead outputs are combined.

As shown in Figure 1, a single "head" in a Flowers layer operates through:

  1. Pointwise Prediction: For a target pixel $x$, the model uses local features to predict a displacement vector $\rho(x)$.
  2. The Warp (Pullback): It then "reaches out" to a source coordinate $(x + \rho(x))$ and pulls information from that distant location back to $x$.
  3. Multihead Organization: Similar to multihead attention, Flowers uses multiple heads per layer to track several physical transport directions simultaneously.

These layers are stacked within a multiscale scaffold, allowing the model to handle both fine-grained local turbulence and large-scale global structures without any Fourier multipliers or convolutional mixing.

Theoretical Motivation and Efficiency

The architecture is mathematically grounded in Geometric Optics and Kinetic Theory. A multihead warp serves as a discrete version of how waves and particles move in the physical world. Because non-locality enters only through sparse sampling (one point per head), the computational cost is linear relative to the number of grid points. This efficiency allows a 17M-parameter Flowers model to outperform foundation models like Poseidon-L (628M parameters) on several fluid dynamics tasks.

Experimental Results

The authors evaluated Flowers against a massive battery of benchmarks, including "The Well" (16 physics datasets), PDEBench, and WaveBench. The benchmarks covered 2D and 3D simulations, testing both next-step prediction and long-term autoregressive rollouts.

Qualitative comparison on a fluid dynamics task. Flowers (right) maintains high-frequency details and structural integrity compared to FNO and other baselines.

Flowers achieved the lowest error across nearly all 16 datasets in "The Well" benchmark. As illustrated in Figure 2, Flowers maintains significantly higher fidelity to the ground truth in complex flow regimes compared to FNO and CNeXtU-Net. In long-term simulations, Flowers remained stable and accurate where traditional models often diverged. Furthermore, Flowers matches or exceeds the throughput of FNO and is significantly faster than complex Transformer-based hybrids at high resolutions.

Conclusion and Future Work

Flowers represents a shift toward physics-inspired primitives in neural PDE solvers. While it dominates in transport-heavy problems like waves and flows, it is currently less dominant in highly diffusive regimes (e.g., reaction-diffusion models) where information spreads spherically rather than along rays. Future work will focus on improving stability in chaotic regimes and extending the theoretical framework to better understand the model's success in time-independent scenarios.

Summary updated

Abstract

We introduce Flowers, a neural architecture for learning PDE solution operators built entirely from multihead warps. Aside from pointwise channel mixing and a multiscale scaffold, Flowers use no Fourier multipliers, no dot-product attention, and no convolutional mixing. Each head predicts a displacement field and warps the mixed input features. Motivated by physics and computational efficiency, displacements are predicted pointwise, without any spatial aggregation, and nonlocality enters \emph{only} through sparse sampling at source coordinates, \emph{one} per head. Stacking warps in multiscale residual blocks yields Flowers, which implement adaptive, global interactions at linear cost. We theoretically motivate this design through three complementary lenses: flow maps for conservation laws, waves in inhomogeneous media, and a kinetic-theoretic continuum limit. Flowers achieve excellent performance on a broad suite of 2D and 3D time-dependent PDE benchmarks, particularly flows and waves. A compact 17M-parameter model consistently outperforms Fourier, convolution, and attention-based baselines of similar size, while a 150M-parameter variant improves over recent transformer-based foundation models with much more parameters, data, and training compute.
BibTeX
@inproceedings{muser2026flowers,
  author = {Till Muser and Alexandra Spitzer and Matti Lassas and Maarten V. de Hoop and Ivan Dokmanić},
  title = {{Flowers: A Warp Drive for Neural PDE Solvers}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=TPpUKH8fuQ},
}
Figure