DeepInverse: A Python package for solving imaging inverse problems with deep learning

Julián Tachella, Matthieu Terris, Samuel Hurault, Andrew Wang, Leo Davy, Jérémy Scanvic, Victor Sechaud, Romain Vo, Thomas Moreau, Thomas Davies, Dongdong Chen, Nils Laurent, Brayan Monroy, Jonathan Dong, Zhiyuan Hu, Minh‐Hai Nguyen, Florian Sarron, Pierre Weiss, Paul Escande, Mathurin Massias, Thibaut Modrzyk, Brett Levac, Tobías I. Liaudat, Maxime Song, Johannes Hertrich, Sebastian Neumayer, Georg Schramm, T.i Liaudat, Myung Kyu SongJ. Open Source Softw 2025 · DOI · DBLP

Abstract: International audience

Introduction

In computational imaging, the fundamental challenge is to recover a hidden image ($x$) from noisy or incomplete measurements ($y$). This process, known as an "inverse problem," is central to fields ranging from medical diagnostics (MRI, CT) to astronomy and remote sensing. While deep learning has significantly advanced the state-of-the-art in reconstruction, the research landscape remains fragmented.

Currently, most researchers develop bespoke codebases for specific tasks, leading to a "Wild West" environment characterized by poor generalization across different physical domains, a reproducibility crisis, and high barriers to entry for practitioners. DeepInverse is an open-source PyTorch library designed to address these issues by providing a unified, modular framework that decouples the physics of imaging from the reconstruction algorithms.

The DeepInverse Framework

DeepInverse is built on the principle of modularity, treating the forward physical model and the reconstruction logic as interchangeable components. This architecture allows researchers to deploy and test state-of-the-art models across diverse domains with minimal code changes.

Overview of the DeepInverse architecture, decoupling physics from reconstruction logic.

The library is organized into several specialized modules:

  • physics: This module defines the forward operator (how an image becomes a measurement) and various noise models. It supports both linear and non-linear operators and includes tools for "blind" problems where physical parameters are unknown.
  • optim & sampling: These modules serve as the "brains" of the reconstruction. They implement optimization-based methods, such as Plug-and-Play (PnP) priors and Deep Equilibrium Models, alongside sampling-based methods like Diffusion models and Langevin MCMC for uncertainty quantification.
  • models: A comprehensive library of architectures, from simple artifact removal networks to complex foundation models.
  • loss: A unified framework for training that crucially supports self-supervised learning. This allows models to be trained even when ground-truth "clean" images are unavailable—a frequent constraint in scientific imaging.

Key Insights and Innovations

Physics as a First-Class Citizen

By treating the forward physical model as a modular object, DeepInverse makes cross-domain research trivial. An algorithm originally developed for photography can be adapted to medical data or astronomical observations in minutes by simply swapping the physics operator. This standardized API—where a reconstruction is defined as model(y, physics)—is the core innovation of the package.

Beyond Point Estimates

Traditional deep learning approaches often provide only a single "best guess" image. DeepInverse emphasizes uncertainty quantification through its sampling module. In high-stakes fields like medicine, understanding the confidence level of an AI-generated reconstruction is as critical as the image itself.

The Trainer Abstraction

The library introduces a Trainer abstraction that simplifies the implementation of complex training loops, such as adversarial training or self-supervision. This reduces the likelihood of implementation errors and lowers the barrier for non-experts to apply modern deep learning techniques to their specific data.

Conclusion and Future Work

DeepInverse represents a significant step toward standardizing deep learning for imaging inverse problems. By providing a common language and modular tools, it facilitates reproducibility and accelerates the transition of research into practical applications.

As a dynamic project, its future success depends on community contributions of new physics operators and architectures. Future developments will focus on improving scalability for large-scale 3D and 4D imaging problems and integrating foundation models that can be fine-tuned for data-scarce scientific tasks.

Summary updated

BibTeX
@article{tachella2025deepinverse,
  author = {Julián Tachella and Matthieu Terris and Samuel Hurault and Andrew Wang and Leo Davy and Jérémy Scanvic and Victor Sechaud and Romain Vo and Thomas Moreau and Thomas Davies and Dongdong Chen and Nils Laurent and Brayan Monroy and Jonathan Dong and Zhiyuan Hu and Minh‐Hai Nguyen and Florian Sarron and Pierre Weiss and Paul Escande and Mathurin Massias and Thibaut Modrzyk and Brett Levac and Tobías I. Liaudat and Maxime Song and Johannes Hertrich and Sebastian Neumayer and Georg Schramm and Pierre Weiss and T.I Liaudat and Myung Kyu Song},
  title = {{DeepInverse: A Python package for solving imaging inverse problems with deep learning}},
  journal = {J. Open Source Softw.},
  year = {2025},
  doi = {10.21105/joss.08923},
}
Figure