Differentiable Inverse Rendering Recovers Intrinsic Properties from Intravascular Optical Coherence Tomography

Ajay Manicka, Karim Kadry, Sebastian Ruiz-Lopera, Maryam Alsharqi, Sanjana Arji, Samuel Ruiperez-Campillo, Nestor Uribe-Patarroyo, Brett E. Bouma, Farhad Nezami, Elazer R. EdelmanarXiv 2026 · arXiv

This paper, recently published by researchers at MIT, Harvard, and ETH Zurich, introduces a powerful new way to look at medical images—not just as pictures of anatomy, but as data that can be "inverted" to reveal the underlying physical properties of tissue.

TL;DR

The authors present a differentiable inverse rendering framework that recovers per-pixel tissue properties from Intravascular Optical Coherence Tomography (IV-OCT) images. By using gradient descent to invert a physics-based forward model, they can extract the optical attenuation coefficient (a key marker of disease) without needing any training data, manual segmentations, or system-specific tuning.

What Problem Does This Paper Address?

Medical images like OCT are "projections" of reality. The brightness of a pixel isn't just a measure of tissue type; it’s a complex entanglement of tissue properties (how much light it scatters or absorbs) and the physics of the imaging system (laser focus, signal decay, and noise).

In IV-OCT, clinicians look for the Optical Attenuation Coefficient (OAC) to identify dangerous plaques (like lipids or calcium) in coronary arteries. However, current methods to calculate OAC are either too simplistic (averaging over large areas), highly sensitive to noise, or rely on "black-box" deep learning models that only work on the specific scanner they were trained on. The challenge is to recover a high-resolution, per-pixel map of these intrinsic properties that is accurate, robust, and portable across different hospitals and devices.

Methods

The core innovation is Differentiable Inverse Rendering. Instead of training a neural network to "guess" the OAC, the authors built a mathematical model of how an OCT image is formed (the "forward model").

  1. The Forward Model: They implemented the Beer-Lambert law (which describes light decay) and confocal geometry (which describes how the lens focuses light) as a differentiable computational graph in PyTorch.
  2. The Inversion: To get the OAC map, they start with a random guess and pass it through the forward model to create a "synthetic" OCT image.
  3. Gradient Descent: They compare the synthetic image to the real clinical image. Because the entire model is differentiable, they can use gradient descent (specifically the Adam or L-BFGS-B optimizers) to iteratively tweak the OAC map until the synthetic image perfectly matches the real one.
  4. Regularization: They use Total Variation (TV) regularization to ensure the resulting tissue maps are spatially smooth and physically plausible, helping to ignore image noise like "speckle."

Datasets

The framework was rigorously tested across three levels of data:

  • Synthetic Data: 500 simulated A-lines and B-scans used to verify that the math works perfectly in a controlled environment.
  • Physical Phantoms: 5 liquid "intralipid" phantoms and 5 agar-based phantoms with known scattering properties to validate the accuracy against gold-standard laboratory measurements.
  • Clinical Data: A massive retrospective archive of 29,269 clinical IV-OCT frames from 128 patients (from the Shockwave Disrupt CAD trials). This demonstrated that the method works on real-world, noisy data from human arteries.

Key Insights and Ideas

  • Gradients are the Secret Sauce: The authors show that trying to solve this problem without gradients (using "gradient-free" methods like Random Search or CMA-ES) becomes impossible as the image resolution increases. Gradient information allows the optimizer to handle thousands of parameters simultaneously, making per-pixel recovery tractable.
  • Physics as a Stable Prior: Unlike deep learning, which can "hallucinate" features, this method is constrained by the laws of physics. The forward model acts as a natural regularizer; if a proposed OAC map doesn't make physical sense, it won't produce a matching image.
  • Zero-Shot Portability: Because the method relies on universal physics rather than a learned dataset, it can be applied to images from different scanner manufacturers (e.g., Abbott vs. Boston Scientific) without needing to be "retrained."

Key Results

  • Numerical Precision: On synthetic data, the method achieved near-perfect recovery with a Mean Absolute Percentage Error (MAPE) of just 0.15%.
  • Phantom Validation: The recovered OAC values showed a near-perfect linear correlation ($R^2 > 0.99$) with actual scatterer concentrations in physical phantoms.
  • Clinical Success: The method successfully processed nearly 30,000 clinical frames. By using a two-stage approach (first recovering OAC, then refining backscattering), they achieved high-fidelity image reconstruction with a mean PSNR of 22.63 dB and a Dice overlap of 0.954.
  • Robustness: The recovery remained stable even when significant Gaussian noise was added to the images, only degrading when the signal-to-noise ratio dropped to near-zero levels.

Limitations and Future Work

  • Computational Cost: Currently, optimizing a single B-scan takes 35–50 seconds on a modern GPU. While fine for research and retrospective analysis, it is currently too slow for real-time use during a live surgical procedure.
  • Absolute Calibration: Without specific metadata from the scanner manufacturer (like exact laser power or internal gains), the recovered values are "relative" rather than absolute physical units. The authors suggest "reference-tissue anchoring" as a future fix.
  • Simplified Physics: The current model assumes "single scattering" (light hits one particle and returns). In very dense tissues, "multiple scattering" occurs. The authors noted that more complex models (like the Extended Huygens-Fresnel model) could be swapped into their modular framework in the future.
  • Uncertainty: Future versions could use Bayesian sampling to provide "confidence maps," telling a doctor exactly how much to trust the OAC value at a specific pixel.

Summary updated

Abstract

Medical images encode intrinsic tissue properties that indicate disease progression, yet these properties remain entangled with the system-dependent physics of image formation. Differentiable inverse rendering asks whether they are recoverable by inverting the imaging system's forward physics rather than through post-hoc correction. We address this in optical coherence tomography (OCT), focusing on the optical attenuation coefficient, a per-pixel marker of pathology and cellular composition. Conventional recovery relies on closed-form inversions limited to simplified light-tissue models, or learned methods coupled to specific imaging configurations. We instead recover per-pixel attenuation from a single OCT image by gradient descent through a differentiable physics-based forward model. We verified the method against analytical solutions, validated it on five in vitro phantoms, and applied it without modification to 29{,}269 clinical intravascular OCT images from 128 patients. The inversion needs no training data or segmentations and is portable across platforms and retrospective archives. Replacing the forward model or recovering another optical parameter leaves the optimization procedure unchanged. We demonstrate per-pixel recovery is tractable because gradient information scales to this high-dimensional problem, where gradient-free methods become intractable. Our work supports quantitative tissue characterization from OCT images and provides a framework for extension across imaging modalities and organ systems.
BibTeX
@misc{manicka2026differentiable,
  author = {Ajay Manicka and Karim Kadry and Sebastian Ruiz-Lopera and Maryam Alsharqi and Sanjana Arji and Samuel Ruiperez-Campillo and Nestor Uribe-Patarroyo and Brett E. Bouma and Farhad Nezami and Elazer R. Edelman},
  title = {{Differentiable Inverse Rendering Recovers Intrinsic Properties from Intravascular Optical Coherence Tomography}},
  year = {2026},
  eprint = {2611.12686},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12686},
}
Figure