Mamba Flow Matching Neural Processes: Linear-Time Inference for Irregularly Observed Spatial Fields

Cosmo Santoni, Giovanni Charles, Timothy James Hitge, Oliver Watson, Elizaveta SemenovaarXiv 2026 · arXiv

TL;DR

MambaFlowNP (MFNP) is a new joint-predictive Neural Process that achieves linear-time complexity $O(K)$ for spatial interpolation by combining multi-directional Mamba-2 scans with conditional flow matching. It runs up to 784× faster than previous state-of-the-art attention-based models, attaining lower error rates on continental-scale temperature mapping and scaling to a million points in under a second on a single GPU.

What Problem Does This Paper Address?

Interpolating spatial fields—estimating values like temperature or disease prevalence at unobserved locations based on sparse sensor data—is a cornerstone of environmental and health sciences.

Traditional methods like Gaussian Process (GP) kriging are mathematically elegant but computationally expensive, scaling as $O(N^3)$ and requiring a full refit for every new dataset. Neural Processes (NPs) offer a faster alternative by learning to interpolate from data, performing inference in a single forward pass.

However, "joint-predictive" NPs, which model the complex dependencies between all predicted locations simultaneously (essential for realistic uncertainty estimation), typically rely on self-attention. This incurs an $O(K^2)$ computational cost (where $K$ is the total number of points), effectively capping their use to a few thousand locations. While state-space models (SSMs) like Mamba offer linear-time alternatives, a naive implementation often compresses the entire context into a single hidden state, creating an information bottleneck that degrades predictive quality.

Methods

The authors introduce MambaFlowNP (MFNP), which sidesteps the quadratic bottleneck of attention while avoiding the compression issues of standard SSMs.

The core architecture (illustrated in Figure 1) consists of four stages:

  1. Tokenization: Coordinates are encoded using frozen Random Fourier Features (RFF) and concatenated with observed values.
  2. Hilbert Routing: To process 2D spatial data with a 1D sequence model, MFNP uses Hilbert curves. These space-filling curves ensure that points nearby in 2D space remain close in the 1D sequence. The model uses four scan directions (two Hilbert orderings and their reversals) to capture spatial context comprehensively.
  3. Distance-aware Scan: MFNP uses a multi-directional Mamba-2 scan. Crucially, context and query tokens share a single sequence. Instead of compressing the context into one final state, each query "reads" the hidden state at its specific position in the scan. This allows queries to access localized information directly.
  4. Conditional Flow Matching (CFM) & Reflow: The model is trained to predict a "velocity field" that transforms a simple noise distribution into the target spatial field. To enable ultra-fast inference, the authors use reflow distillation, which allows the model to generate a high-quality field in just a single network evaluation (1-NFE).
The MFNP pipeline: from tokenization and Hilbert routing to multi-directional Mamba-2 scans and final velocity prediction.

Datasets

The model was evaluated on:

  • Synthetic Matérn Gaussian Processes: Standard 1D and 2D benchmarks used to test the ability to recover known underlying spatial statistics.
  • NOAA Temperature Interpolation: Real-world daily maximum temperature (TMAX) records from the Global Historical Climatology Network. The authors tested three geographic tiers: Utah ($K=222$), California ($K=486$), and the Continental US ($K=5,580$).
  • Scalability Benchmarks: Synthetic tests scaling up to $10^6$ points to measure computational limits.

Key Insights and Ideas

  • Breaking the Bottleneck: The paper's most significant finding is that "single-state compression" is a major performance killer for SSM-based spatial models. By allowing queries to read from their own positions in a shared scan, MFNP achieves a 58–65% reduction in Mean Squared Error (MSE) on NOAA tasks compared to a compressed variant.
  • Linear Scaling is Transformative: Because MFNP is $O(K)$, it can handle datasets that are orders of magnitude larger than what attention-based models can touch. While models like FNP or TNPA run out of memory (OOM) at roughly 16k to 65k points, MFNP continues to scale linearly.
  • Reflow Distillation is Essential for Speed: The massive speedups (114–784×) aren't just from the Mamba architecture; they also come from reflow distillation, which reduces the number of network passes needed to generate a sample from dozens down to one.

Key Results

MFNP establishes a new Pareto frontier for spatial interpolation, offering both the highest accuracy and the lowest latency among joint-predictive models.

  • Performance on NOAA: On the continental US task, MFNP achieved the lowest RMSE (1.598°C) and CRPS (0.774°C) while running hundreds of times faster than baselines.
  • Speed and Scalability: As shown in Figure 3, MFNP's latency scales linearly. It can process one million points in under a second on a single NVIDIA L40S GPU.
  • Calibration: The model provides near-nominal 95% predictive coverage, meaning its uncertainty estimates are reliable and well-calibrated.
Figure 2: Per-episode latency vs. RMSE (left) and CRPS (right) on continental US NOAA (K=5,580). Means ±1 SE over 10 seeds at Nsamples=128; the hatched staircase is the empirical Pareto frontier. TNPA exceeds the per-cell budget (§5.3); Utah and California frontiers in Appendix F.
Figure 3: End-to-end sampling latency vs. K (log–log) on 2D Gaussian Matérn-3/2, batch= 1, Nsamples=1 (1-NFE Euler for MFNP; published step count per baseline). Colour gives the model class; a short dashed line through a curve’s last point means the next K runs out of memory. Numbers in Table 8.

Limitations and Future Work

  • Irregular vs. Regular Grids: MFNP is designed for irregular point sets. For data on a perfectly regular grid, convolutional NPs or FFT-based solvers might still be more efficient.
  • Small-Scale Overhead: At very small point counts ($K < 1000$), the fixed overhead of the Mamba dispatch makes it slower than the simplest "factorized" NPs, though it remains much faster than other joint-predictive models.
  • Higher Dimensions: The current work focuses on 1D and 2D spatial fields. Extending this to 3D (space-time) or higher-dimensional inputs is a natural next step.
  • Generalization: The NOAA evaluation was performed on a fixed station network. Testing how well the model transfers to entirely new, unseen geographic regions remains for future study.

Summary updated

Abstract

Interpolating a spatial field, such as daily maximum temperature, from sparse, irregularly located observations recurs across the environmental and health sciences. Standard Gaussian-process kriging refits hyperparameters and solves an $O(N^3)$ system for every new field. When many related fields are observed, such as daily maps from one sensor network, neural processes (NPs) instead learn the interpolation map from past fields and predict a new field in one forward pass. Joint-predictive variants, which sample whole fields rather than independent per-location values, typically use self-attention over context and query points, costing $O(K^2)$ in the combined point count $K$ and limiting practical use to a few thousand points. State-space models offer a linear-time alternative, but a direct state-space adaptation compresses the context into one hidden state before reading queries, reducing what the decoder sees. We propose MambaFlowNP, a linear-compute joint-predictive NP trained with conditional flow matching. A multi-directional Mamba-2 scan reads context and query tokens in one shared sequence, so each query's velocity is read from its own scan position rather than a compressed context summary, and because queries only read the scan state, predictions are consistent across query sets. MambaFlowNP is competitive with the strongest attention-based joint-predictive baselines on synthetic Matérn Gaussian-process benchmarks, and on regional to continental-scale NOAA temperature interpolation it attains the lowest RMSE and CRPS and near-nominal coverage on every tier, while running 114--784$\times$ faster than the strongest joint-predictive baseline. A single forward pass scales to $10^6$ points on one NVIDIA L40S GPU in under a second. This makes joint prediction feasible for continental-scale, irregularly sampled sensor networks; predictive quality is evaluated up to $K=5{,}580$.
BibTeX
@misc{santoni2026mamba,
  author = {Cosmo Santoni and Giovanni Charles and Timothy James Hitge and Oliver Watson and Elizaveta Semenova},
  title = {{Mamba Flow Matching Neural Processes: Linear-Time Inference for Irregularly Observed Spatial Fields}},
  year = {2026},
  eprint = {2611.26355},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26355},
}
Figure