Abstract: International audience
Introduction
In computational imaging, the fundamental challenge is to recover a hidden image ($x$) from noisy or incomplete measurements ($y$). This process, known as an "inverse problem," is central to fields ranging from medical diagnostics (MRI, CT) to astronomy and remote sensing. While deep learning has significantly advanced the state-of-the-art in reconstruction, the research landscape remains fragmented.
Currently, most researchers develop bespoke codebases for specific tasks, leading to a "Wild West" environment characterized by poor generalization across different physical domains, a reproducibility crisis, and high barriers to entry for practitioners. DeepInverse is an open-source PyTorch library designed to address these issues by providing a unified, modular framework that decouples the physics of imaging from the reconstruction algorithms.
The DeepInverse Framework
DeepInverse is built on the principle of modularity, treating the forward physical model and the reconstruction logic as interchangeable components. This architecture allows researchers to deploy and test state-of-the-art models across diverse domains with minimal code changes.
The library is organized into several specialized modules:
physics: This module defines the forward operator (how an image becomes a measurement) and various noise models. It supports both linear and non-linear operators and includes tools for "blind" problems where physical parameters are unknown.optim&sampling: These modules serve as the "brains" of the reconstruction. They implement optimization-based methods, such as Plug-and-Play (PnP) priors and Deep Equilibrium Models, alongside sampling-based methods like Diffusion models and Langevin MCMC for uncertainty quantification.models: A comprehensive library of architectures, from simple artifact removal networks to complex foundation models.loss: A unified framework for training that crucially supports self-supervised learning. This allows models to be trained even when ground-truth "clean" images are unavailable—a frequent constraint in scientific imaging.
Key Insights and Innovations
Physics as a First-Class Citizen
By treating the forward physical model as a modular object, DeepInverse makes cross-domain research trivial. An algorithm originally developed for photography can be adapted to medical data or astronomical observations in minutes by simply swapping the physics operator. This standardized API—where a reconstruction is defined as model(y, physics)—is the core innovation of the package.
Beyond Point Estimates
Traditional deep learning approaches often provide only a single "best guess" image. DeepInverse emphasizes uncertainty quantification through its sampling module. In high-stakes fields like medicine, understanding the confidence level of an AI-generated reconstruction is as critical as the image itself.
The Trainer Abstraction
The library introduces a Trainer abstraction that simplifies the implementation of complex training loops, such as adversarial training or self-supervision. This reduces the likelihood of implementation errors and lowers the barrier for non-experts to apply modern deep learning techniques to their specific data.
Conclusion and Future Work
DeepInverse represents a significant step toward standardizing deep learning for imaging inverse problems. By providing a common language and modular tools, it facilitates reproducibility and accelerates the transition of research into practical applications.
As a dynamic project, its future success depends on community contributions of new physics operators and architectures. Future developments will focus on improving scalability for large-scale 3D and 4D imaging problems and integrating foundation models that can be fine-tuned for data-scarce scientific tasks.
Summary updated