TL;DR
STRAINER is a simple yet powerful framework that makes Implicit Neural Representations (INRs) transferable. By sharing initial "encoder" layers across a small set of signals from the same domain, it learns a data-driven prior that allows new INRs to converge significantly faster and achieve up to +10dB higher reconstruction quality than standard random initialization.
The "Cold Start" Problem of INRs
Implicit Neural Representations (INRs)—like the popular SIREN or NeRF—map coordinates (like $x, y$) to signal values (like RGB). While they are excellent at capturing continuous detail, they suffer from a major "cold start" problem: an INR is typically trained from scratch for every single signal.
Because each network is overfit to one specific image or 3D shape, the learned features are generally considered non-transferable. This makes INRs slow to train and prevents them from leveraging common patterns within a dataset (e.g., all human faces share a similar basic structure). While meta-learning approaches exist to solve this, they are often computationally expensive, unstable, and require massive amounts of training data.
Introducing STRAINER
To address this, the authors introduce STRAINER (Shared TRansferable Architecture for INRs), which rethinks the INR as a two-part system:
- The Encoder: The first several layers of the MLP.
- The Decoder: The remaining layers leading to the output.
The Training Phase: Instead of training one network on one image, STRAINER trains $N$ networks (usually just 10) on $N$ different images from the same domain simultaneously. The key trick is that all $N$ networks share the same encoder weights but maintain independent decoders. This forces the encoder to learn a "common language" or partition space that represents the general morphology of the domain (like the layout of a face).
The Test Phase: When fitting a brand-new, unseen signal, the network is initialized with the pre-trained shared encoder and a random decoder. The entire network is then optimized as usual, but with a massive head start.
Datasets and Evaluation
The researchers evaluated STRAINER across a diverse range of signal types:
- Natural Images: CelebA-HQ (faces) and AFHQ (cat faces).
- Medical Imaging: OASIS-MRI (brain scans).
- Generalization/OOD: Stanford Cars, Flowers, and the Kodak dataset for high-resolution testing.
- 3D Data: ShapeNet (specifically the 'Chair' category) and the 'Thai Statue' for 3D occupancy fitting.
Key Insights: The "Ghost" in the Machine
By visualizing the input space partitions using PCA on the learned features, the authors found that STRAINER’s encoder develops a "ghost-like" structure of the target domain before it even sees the test image.
- Structural Priors: As seen above, standard INRs (like SIREN) start with random, chaotic partitions. STRAINER, however, initializes with a recognizable base structure, allowing it to capture high-frequency details much earlier in the optimization process.
- Minimalist Meta-Learning: Unlike traditional meta-learning (like MAML) which might require 14,000 images and hours of compute, STRAINER achieves superior results using just 10 images and about 25 seconds of training.
- Surprising Generalization: An encoder trained on human faces (CelebA) actually provides a better initialization for MRI scans or cats than a random start. This suggests the encoder learns a general "natural signal prior" that transcends specific categories.
Key Results
- Instant Quality Boost: At the start of test-time fitting, STRAINER provides a ~10dB PSNR gain over vanilla SIREN.
- Speedup: In inverse problems like super-resolution and denoising, STRAINER recovers clean images 3x faster than standard baselines.
- 3D Superiority: For 3D occupancy fitting, STRAINER achieved a 12.3% relative improvement in Intersection over Union (IoU) compared to random initialization after only 150 iterations.
- Efficiency: STRAINER-10 (trained on 10 images) outperforms state-of-the-art transformer-based hypernetworks (like TransINR) while being significantly lighter and faster to train.
Limitations and Future Work
- Occasional Instability: The authors noted occasional "PSNR drops" during the fitting process, likely due to the mismatch between the pre-trained encoder and the randomly initialized decoder, though the model usually recovers quickly.
- Theoretical Gap: While the empirical results are strong, the mathematical "why" behind which specific features are transferred remains an open question.
- Scaling: Future work could explore how these transferable features behave when scaled to much larger, more complex 3D scenes or video representations.
Summary updated