Most computer vision algorithms operate on images that have already been "baked." By the time a photo reaches a standard sRGB or JPEG format, it has passed through a camera’s Image Signal Processor (ISP)—a complex pipeline that applies demosaicing, noise reduction, sharpening, and non-linear tone mapping to make the image pleasing to the human eye.
However, as Nguyen Ho Man Rang and Michael S. Brown argue in Why You Should Forget Luminance Conversion and Do Something Better, these processed images are often mathematically "incorrect" for scientific tasks. A growing body of research suggests that for tasks like denoising, deblurring, and super-resolution, we should bypass the ISP entirely and work directly with linear RAW sensor data.
The Linearity Advantage
The fundamental reason to work in the RAW domain is physical consistency. Light is additive. If you double the number of photons hitting a sensor, the raw voltage (and the resulting RAW pixel value) doubles. This linear relationship is the bedrock of physics-based vision.
When we perform restoration in the processed sRGB domain, we are fighting against non-linearities. For example, Xianyong Fang and colleagues demonstrate in Deblurring of motion blurred images based on the irradiance that deblurring in the linear irradiance domain significantly reduces "ringing" artifacts. Because motion blur is physically an accumulation of light over time, the math of deconvolution only holds true if the pixel values are proportional to that light energy.
Similarly, Eric Kee and his team at Adobe show in Removing Reflections from RAW Photos that separating a reflection from a window is much easier in a linear space. In RAW, the image is simply the sum of the background and the reflection ($I = B + R$). In a processed JPEG, the camera's non-linear tone curve warps this relationship, making the layers nearly impossible to disentangle perfectly.
Predictable Noise vs. Correlated Artifacts
In the RAW domain, sensor noise is well-behaved. It typically follows a Poisson-Gaussian distribution, which is mathematically predictable. As Ying Fu notes in the Technique Report of CVPR 2024 PBDL Challenges, this allows neural networks to model noise with high precision.
Once the ISP gets hold of the data, it applies "denoising" and "sharpening" that turns this simple pixel-wise noise into complex, spatially correlated patterns. For a deep learning model, removing "shaped" noise from an sRGB image is like trying to solve a puzzle where the pieces have been melted together. By starting at the source, models like ExposureDiffusion by Yi Yu and Bihan Wen can "see" through extreme darkness by leveraging the original, undistorted noise statistics.
Preserving the Dynamic Range
Standard images are usually 8-bit, meaning they can only represent 256 levels of brightness. RAW files are typically 12 to 14-bit, capturing thousands of levels. When an ISP converts RAW to sRGB, it often "clips" the highlights and crushes the shadows to fit the limited range.
Ben Mildenhall and Ricardo Martin Brualla exploited this in NeRF in the Dark. By training a Neural Radiance Field directly on noisy RAW data, they created RawNeRF, which can reconstruct 3D scenes with a high dynamic range (HDR) that would be lost in standard photography. This allows for post-capture effects like changing the exposure or focus in a way that feels physically real because the underlying model is grounded in linear radiance.
Generalization Across Sensors
A major headache in computer vision is that every camera manufacturer has a "secret sauce" for their ISP. A model trained to denoise photos from an iPhone might fail on a Sony camera because their ISPs handle colors and noise differently.
Working in the RAW domain provides a more "universal" language. As Marcos V. Conde and Radu Timofte demonstrate in their work on BSRAW and the NTIRE 2025 Challenge, restoration models trained on RAW data generalize much better. By bypassing the proprietary "photo-finishing" of the camera, the model learns the fundamental relationship between light and the sensor, rather than the quirks of a specific software pipeline.
The Future: Sensor-Aware AI
The shift toward RAW processing isn't just about better-looking photos; it's about more robust machine perception. Hung-Shuo Tai showed in Guiding a Harsh-Environments Robust Detector via RAW Data Characteristic Mining that object detectors perform significantly better in fog, snow, and low light when they look at RAW data.
By "un-baking" the image and returning to the linear domain, researchers are moving away from treating computer vision as a task of interpreting pictures and toward a task of measuring light. As Huanjing Yue and Jingyu Yang summarize in their survey of RAW domain reconstruction, the future of the field lies in embracing the raw, linear reality of the sensor.