Invertible Image Signal Processing and RAW Reconstruction

This direction is merged into another direction: ISP-induced information loss. Its last version stays readable and citable here.

Version 1 of 123 papers23 conceptsfrozen 2026-10-04

The digital images we see on our screens are not what the camera sensor actually captured. Between the physical sensor and the final JPEG file lies the Image Signal Processor (ISP)—a complex pipeline of tone mapping, gamma correction, and quantization designed to make images look pleasing to the human eye.

However, researchers like Yazhou Xing and Qifeng Chen argue that this process is a "lossy bottleneck." By squashing 14-bit linear radiance data into 8-bit sRGB formats, standard ISPs discard the very information—high dynamic range and precise signal-to-noise ratios—that computer vision models need to function in challenging environments. A new research direction seeks to fix this by making the ISP invertible, allowing us to reconstruct high-fidelity RAW data from standard images.

The Problem of Non-Invertible Transformations

The core issue across this field is the irreversible loss of data. In traditional computer vision, feature extraction often discards context. Erik J Bekkers and Remco Duits highlight this in Template Matching on the Roto-Translation Group, noting that non-invertible transformations limit robustness. They propose "lifting" 2D images into a 3D orientation score space ($SE(2)$) to preserve geometric context, such as vessel crossings in medical scans, which would otherwise be lost.

In the context of camera hardware, the "lifting" is literal: moving from the compressed sRGB space back to the high-bit-depth RAW sensor space. As Leyi Li explains in Metadata-Based RAW Reconstruction via Implicit Neural Functions, standard ISPs apply non-linear gains that clip highlights and bury dark details in quantization noise. Once this information is gone, traditional computer vision "eyes" are effectively blinded in extreme lighting.

Reversing the Pipeline with Invertible Networks

To solve this, Yazhou Xing introduced InvISP in Invertible Image Signal Processing. Instead of a series of destructive steps, InvISP models the entire camera pipeline as a single, bijective (invertible) neural network using normalizing flows.

This works because normalizing flows are designed to be mathematically reversible. When you want a pretty picture for a user, the network runs "forward" to produce an sRGB image. When an AI model needs to detect an object in the dark, the network runs "backward," reconstructing the original 14-bit RAW data from the 8-bit file. This allows for nearly perfect recovery of the sensor's original linear radiance.

Efficient Reconstruction via Learned Metadata

While fully invertible networks are powerful, they can be computationally heavy. A parallel approach uses "learned metadata" to bridge the gap. In Raw Image Reconstruction with Learned Compact Metadata, Yi Yu and Wenhan Yang propose storing a tiny "sidecar" file (less than 0.1% of the image size) that contains the information the ISP would normally discard.

This metadata isn't just a random sample of pixels; it's a learned latent representation. By using the sRGB image as a guide, the model only needs to store the "residuals"—the specific details lost during clipping and quantization. This allows for state-of-the-art RAW reconstruction without the storage overhead of full RAW files, a technique Marcos V. Conde and Radu Timofte have benchmarked extensively in the NTIRE 2025 Challenge.

Why RAW Matters for Downstream Vision

The ultimate goal of this research isn't just better-looking photos; it's better-performing AI. Wang Zhao demonstrates in DarkFeat that feature detectors trained on RAW data are significantly more robust to noise than those trained on RGB.

This advantage is most visible in "nightmare scenarios" like extreme low light. Wenjun Huang and Tatsuya Harada introduced Dr. RAW (paper), a framework that allows models to "see" directly in the RAW domain. By bypassing the lossy ISP, Dr. RAW achieves superior object detection and segmentation because it has access to the raw physical signals—subtle textures and edges—that the ISP's noise reduction would have smoothed away.

By treating the image acquisition process as a reversible mathematical problem rather than a one-way aesthetic transformation, this research direction ensures that the "eyes" of autonomous systems have access to every photon the sensor captured.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_invertible_image_signal_processing_and_r_b320cc43,
  title = {Invertible Image Signal Processing and RAW Reconstruction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/invertible-image-signal-processing-and-raw-reconstruction/rdr_b320cc43ca2f4338a082e77c37771d88},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_invertible_image_signal_processing_and_r_b320cc43_v1,
  title = {Invertible Image Signal Processing and RAW Reconstruction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/invertible-image-signal-processing-and-raw-reconstruction/rdr_b320cc43ca2f4338a082e77c37771d88/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}