Fast Amortized Fitting of Scientific Signals Across Time and Ensembles via Transferable Neural Fields

Sophia Zorek, Kushal Vyas, Yuhao Liu, David Lenz, Tom Peterka, Guha BalakrishnanarXiv 2026 · arXiv

TL;DR

Training Implicit Neural Representations (INRs) from scratch for high-dimensional scientific simulations is computationally expensive and slow to converge. This paper introduces a lightweight transfer learning framework that shares a structural encoder across different timesteps and simulation ensembles, accelerating convergence by up to an order of magnitude while significantly improving the accuracy of derived physical quantities like gradients and vorticity.

What Problem Does This Paper Address?

Modern scientific simulations—such as climate modeling, astrophysics, and computational fluid dynamics—generate massive, high-dimensional (4D/5D) datasets. Traditional grid-based data structures struggle to scale with these resolutions. Implicit Neural Representations (INRs) have emerged as a powerful alternative, mapping continuous coordinates to physical values (like density or velocity) using neural networks. This allows for arbitrary super-resolution and excellent data compression.

However, INRs face a major bottleneck: training them from scratch for every new timestep or simulation run is incredibly slow. Furthermore, scientific applications demand more than just visual pixel-matching (high PSNR); they require highly accurate first-order derivatives (e.g., spatial gradients) to preserve the underlying physics. This paper tackles how to efficiently share and transfer learned structural features across related scientific signals to drastically reduce optimization costs without sacrificing physical fidelity.

Methods

The authors propose a model-agnostic framework that decomposes an INR into two parts:

  1. A Shared Encoder ($f_\psi$): Designed to learn invariant, multiscale structural features across a collection of related signals.
  2. Signal-Specific Decoders ($g_{\phi_j}$): Lightweight heads that adapt the shared features to specific timesteps or simulation instances.
Figure 2. Transferable neural fields for efficient fitting of 4D/5D signals. A shared encoder learns reusable multiscale structure across signals, while lightweight time- or simulation-specific decoders enable efficient adaptation to new instances. This decomposition allows features learned from prior signals to be reused, providing a strong initialization for fitting unseen data (left). Reconstruction progression (PSNR) for an unseen simulation from Rayleigh–Taylor instability using SIREN, with pretrained initialization (bottom) and random initialization (top). Density from a late-timestep slice is visualized. Pretrained models recover mixing structures significantly earlier, achieving higher PSNR at early iterations and resolving sharper interface dynamics (right).

During pretraining, the encoder and multiple decoders are trained jointly on a subset of signals. When a completely new, unseen simulation or timestep arrives, the pretrained encoder weights are reused as a strong initialization, and only a new lightweight decoder is initialized from scratch.

The authors test this approach across three popular INR architectures:

  • Smooth function-based models (SIREN): MLPs using sinusoidal activation functions.
  • Spatial encoding-based models (Hash Grids): Multiresolution lookup tables coupled with a shallow MLP.
  • Factorized representations (K-Planes): Models that decompose dimensions into lower-dimensional planar features.

Datasets

The framework was evaluated across three distinct categories of data:

  1. Time-Evolving Toy Signal: A 3D dataset based on the non-convex Schwefel function, subjected to controlled geometric transformations (rotation, warping) and localized perturbations (waves, Gaussian windows).
  2. The Well Benchmark: A suite of physics-based simulations including Rayleigh–Taylor Instability (fluid mixing), Magnetohydrodynamics (MHD) (plasma/magnetic field interactions), and Supernova Explosions.
  3. Deep Water Asteroid Impact: A large-scale, high-fidelity multiphysics simulation modeling an asteroid hitting an ocean, spanning multiple coupled thermodynamic and material fields.

Key Insights and Ideas

  • Physics Leaves a Shared Footprint: Even when simulations diverge due to chaotic dynamics or different initial conditions, they share underlying multiscale structures. Reusing the encoder effectively amortizes the cost of learning these foundational geometries.
  • Architecture Dictates Transferability: Smooth models like SIREN benefit the most from feature transfer. Because SIREN relies on global continuous parameterization, its learned features generalize easily across spatial shifts. Conversely, spatial encoding methods (Hash Grids, K-Planes) are tied to local coordinate spaces, making them less effective when features rotate or shift spatially.
  • Pretraining Acts as a Physical Prior: Beyond speed, initializing with a pretrained encoder acts as a regularizer. It prevents the network from fitting high-frequency noise early on, leading to substantially cleaner and more accurate analytical gradients.

Key Results

  • Massive Speedups: Pretraining reduced the number of iterations required to reach target reconstruction thresholds by up to an order of magnitude (e.g., dropping from thousands of iterations to under a hundred for SIREN on certain tasks).
  • Early-Stage Fidelity Boosts: Pretrained models recovered high-frequency details significantly faster, yielding early-stage gains exceeding 10 dB PSNR compared to random initialization.
Figure 1. Early reconstruction advantage from transferable features (PSNR/SSIM). Water volume fraction from Deep Water Asteroid Impact is visualized with pretrained initialization (top) and random initialization (bottom). PSNR is averaged across all output fields. Pretrained models recover higher frequency detail of the water structure significantly earlier in training, yielding higher PSNR (+10dB gain) at early iterations.
  • Enhanced Physical Accuracy: For the Rayleigh–Taylor instability dataset, the proposed transfer method consistently lowered the Root Mean Squared Error (RMSE) for both density gradients ($\nabla \rho$) and vorticity ($\nabla \times v$), outperforming models trained from scratch.

Limitations and Future Work

The primary limitation lies in spatial misalignment. When variations between simulation runs introduce geometric rotations or phase shifts, spatial encoding-based models (like Hash Grids) see diminished benefits because their learned features are rigidly anchored to specific coordinate locations.

The authors note that future work should focus on developing more robust mechanisms to handle these spatial misalignments and extending the framework to broader, real-world experimental datasets.

Summary updated

Abstract

Neural fields, also known as implicit neural representations (INRs), offer a powerful framework for modeling continuous geometry, but their effectiveness in high-dimensional scientific settings is limited by slow convergence and scaling challenges. In this study, we extend INR models to handle spatiotemporal and multivariate signals and show how INR features can be transferred across scientific signals to enable efficient and scalable representation across time and ensemble runs in an amortized fashion. Across controlled transformation regimes (e.g., geometric transformations and localized perturbations of synthetic fields) and high-fidelity scientific domains-including turbulent flows, fluid-material impact dynamics, and astrophysical systems-we show that transferable features improve not only signal fidelity but also the accuracy of derived geometric and physical quantities, including density gradients and vorticity. In particular, transferable features reduce iterations to reach target reconstruction quality by up to an order of magnitude, increase early-stage reconstruction quality by multiple dB (with gains exceeding 10 dB in some cases), and consistently improve gradient-based physical accuracy.
BibTeX
@misc{zorek2026fast,
  author = {Sophia Zorek and Kushal Vyas and Yuhao Liu and David Lenz and Tom Peterka and Guha Balakrishnan},
  title = {{Fast Amortized Fitting of Scientific Signals Across Time and Ensembles via Transferable Neural Fields}},
  year = {2026},
  eprint = {2604.19979},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.19979},
}

Loading related content...

Figure