TL;DR
MambaFlowNP (MFNP) is a new joint-predictive Neural Process that achieves linear-time complexity $O(K)$ for spatial interpolation by combining multi-directional Mamba-2 scans with conditional flow matching. It runs up to 784× faster than previous state-of-the-art attention-based models, attaining lower error rates on continental-scale temperature mapping and scaling to a million points in under a second on a single GPU.
What Problem Does This Paper Address?
Interpolating spatial fields—estimating values like temperature or disease prevalence at unobserved locations based on sparse sensor data—is a cornerstone of environmental and health sciences.
Traditional methods like Gaussian Process (GP) kriging are mathematically elegant but computationally expensive, scaling as $O(N^3)$ and requiring a full refit for every new dataset. Neural Processes (NPs) offer a faster alternative by learning to interpolate from data, performing inference in a single forward pass.
However, "joint-predictive" NPs, which model the complex dependencies between all predicted locations simultaneously (essential for realistic uncertainty estimation), typically rely on self-attention. This incurs an $O(K^2)$ computational cost (where $K$ is the total number of points), effectively capping their use to a few thousand locations. While state-space models (SSMs) like Mamba offer linear-time alternatives, a naive implementation often compresses the entire context into a single hidden state, creating an information bottleneck that degrades predictive quality.
Methods
The authors introduce MambaFlowNP (MFNP), which sidesteps the quadratic bottleneck of attention while avoiding the compression issues of standard SSMs.
The core architecture (illustrated in Figure 1) consists of four stages:
- Tokenization: Coordinates are encoded using frozen Random Fourier Features (RFF) and concatenated with observed values.
- Hilbert Routing: To process 2D spatial data with a 1D sequence model, MFNP uses Hilbert curves. These space-filling curves ensure that points nearby in 2D space remain close in the 1D sequence. The model uses four scan directions (two Hilbert orderings and their reversals) to capture spatial context comprehensively.
- Distance-aware Scan: MFNP uses a multi-directional Mamba-2 scan. Crucially, context and query tokens share a single sequence. Instead of compressing the context into one final state, each query "reads" the hidden state at its specific position in the scan. This allows queries to access localized information directly.
- Conditional Flow Matching (CFM) & Reflow: The model is trained to predict a "velocity field" that transforms a simple noise distribution into the target spatial field. To enable ultra-fast inference, the authors use reflow distillation, which allows the model to generate a high-quality field in just a single network evaluation (1-NFE).
Datasets
The model was evaluated on:
- Synthetic Matérn Gaussian Processes: Standard 1D and 2D benchmarks used to test the ability to recover known underlying spatial statistics.
- NOAA Temperature Interpolation: Real-world daily maximum temperature (TMAX) records from the Global Historical Climatology Network. The authors tested three geographic tiers: Utah ($K=222$), California ($K=486$), and the Continental US ($K=5,580$).
- Scalability Benchmarks: Synthetic tests scaling up to $10^6$ points to measure computational limits.
Key Insights and Ideas
- Breaking the Bottleneck: The paper's most significant finding is that "single-state compression" is a major performance killer for SSM-based spatial models. By allowing queries to read from their own positions in a shared scan, MFNP achieves a 58–65% reduction in Mean Squared Error (MSE) on NOAA tasks compared to a compressed variant.
- Linear Scaling is Transformative: Because MFNP is $O(K)$, it can handle datasets that are orders of magnitude larger than what attention-based models can touch. While models like FNP or TNPA run out of memory (OOM) at roughly 16k to 65k points, MFNP continues to scale linearly.
- Reflow Distillation is Essential for Speed: The massive speedups (114–784×) aren't just from the Mamba architecture; they also come from reflow distillation, which reduces the number of network passes needed to generate a sample from dozens down to one.
Key Results
MFNP establishes a new Pareto frontier for spatial interpolation, offering both the highest accuracy and the lowest latency among joint-predictive models.
- Performance on NOAA: On the continental US task, MFNP achieved the lowest RMSE (1.598°C) and CRPS (0.774°C) while running hundreds of times faster than baselines.
- Speed and Scalability: As shown in Figure 3, MFNP's latency scales linearly. It can process one million points in under a second on a single NVIDIA L40S GPU.
- Calibration: The model provides near-nominal 95% predictive coverage, meaning its uncertainty estimates are reliable and well-calibrated.
Limitations and Future Work
- Irregular vs. Regular Grids: MFNP is designed for irregular point sets. For data on a perfectly regular grid, convolutional NPs or FFT-based solvers might still be more efficient.
- Small-Scale Overhead: At very small point counts ($K < 1000$), the fixed overhead of the Mamba dispatch makes it slower than the simplest "factorized" NPs, though it remains much faster than other joint-predictive models.
- Higher Dimensions: The current work focuses on 1D and 2D spatial fields. Extending this to 3D (space-time) or higher-dimensional inputs is a natural next step.
- Generalization: The NOAA evaluation was performed on a fixed station network. Testing how well the model transfers to entirely new, unseen geographic regions remains for future study.
Summary updated