Introduction
Inverse problems involve determining hidden causes from observed effects, such as reconstructing internal structures from scattered waves. While deep learning offers a fast alternative to traditional numerical solvers, it typically requires massive datasets to cover the entire parameter space. This "data hunger" is particularly problematic in high-dimensional scientific applications where generating ground-truth data via physics simulations is computationally expensive.
The Proposed Framework
To address this, we introduce an instance-wise adaptive sampling framework. Rather than training a single "global" model to handle all possible inputs, this approach utilizes a crude base model that is iteratively refined for specific test cases during inference.
The process begins with a base model trained on a small, general dataset. For a given measurement, the model generates an initial guess, which is then projected onto a prior manifold. The framework then generates a small "adaptive dataset" by perturbing this guess and running forward physics simulations. The model is fine-tuned on this local data, and the process iterates until the solution stabilizes. This strategy focuses computational resources on the local geometry of the inverse map relevant to the specific instance.
Analogy to LLM Self-Refinement
This approach draws inspiration from "inference-time scaling" in Large Language Models (LLMs), where additional computation at the time of prediction (e.g., Self-Refine or Chain of Thought) improves accuracy.
As shown in the comparison above, the adaptive sampling process mirrors the feedback and refinement loops used in LLMs. In the context of inverse problems, the "feedback" is the generation of a local dataset around the current estimate, and the "refinement" is the fine-tuning of the neural network.
Experimental Evaluation
The framework was evaluated using synthetic datasets based on the inverse acoustic scattering problem, governed by the Helmholtz equation. Two types of priors were tested:
- Disk Prior: Represents distinct objects with constant amplitudes.
- Fourier Prior: Represents smooth, continuous variations using low-frequency Fourier modes.
Performance was benchmarked against standard non-adaptive models and classical optimization techniques like Gauss-Newton.
Key Results
The adaptive sampling framework demonstrated significant improvements in data efficiency:
- Data Reduction: In the Disk Prior scenario, the method achieved a 23-fold reduction in required training data. In the more complex Fourier Prior scenario, this efficiency increased to 166x, requiring only 27,000 samples compared to the 4.4 million needed by a standard model to reach equivalent accuracy.
- Complexity Scaling: The advantages of the adaptive approach become more pronounced as problem complexity or required precision increases.
- Robustness: The iterative process allows the model to self-correct initial errors, such as misidentifying the number of objects in the medium.
Conclusion
Instance-wise adaptive sampling shifts the computational burden from pre-training data collection to inference-time processing. While this increases the time required for individual predictions, it drastically reduces the total data needed to solve complex, high-dimensional inverse problems, making deep learning viable for a broader range of scientific applications.
Summary updated