Nighttime photography is frequently compromised by lens flares—optical artifacts resulting from light reflection and refraction within camera sensor arrays. These artifacts typically manifest as "glow" (a hazy aura surrounding bright light sources) and "reflective flares" or "ghosting" (distinct geometric patterns elsewhere in the frame). While often treated as separate issues, these flares almost always co-occur and are physically interdependent, originating from the same light source and passing through the same optical system.
Traditional flare removal methods often specialize in a single flare type, failing to address the complex, non-additive relationship between co-existing artifacts. A new approach presented at AAAI 2025, SGLFR-Net, introduces a self-supervised framework designed to disentangle and remove these joint flares by modeling their intrinsic physical relationship.
The Challenge of Joint Flare Removal
Most existing solutions are "black-box" models trained on synthetic data. However, capturing real-world ground-truth pairs (the same scene with and without flares) is nearly impossible. Furthermore, simply cascading individual de-glow and de-ghosting algorithms often results in residual artifacts because it ignores the shared physical origin of the flares.
Methodology: SGLFR-Net
The proposed Self-supervised Generation-based Lens Flare Removal Network (SGLFR-Net) moves away from standard supervised learning. Instead, it leverages a physics-informed model of nighttime lens flare formation. The system consists of two primary components that work in parallel to "re-generate" and then subtract the flare components from the input image.
PSFR-Net: Modeling the Glow
The PSF Rendering Network (PSFR-Net) focuses on suppressing glow. It utilizes a Point Spread Function (PSF) rendering prior to model light scattering. By learning to generate a fuzzy kernel that matches the specific glow in the input, it can accurately estimate and remove the hazy aura around light sources.
TPRR-Net and Optical Symmetry
The Texture Prior based Reflection Removal Network (TPRR-Net) handles reflective flares. It employs a novel Optical Symmetry based Texture Prior (OS-TPM). This module exploits the physical property that reflective flares often appear symmetrically across the optical center from their source. By identifying these symmetric regions, the network can effectively mask and "fill in" ghosting artifacts using texture priors.
Because SGLFR-Net is self-supervised, it optimizes its weights for each specific image during inference, requiring no pre-training on large datasets.
Results and Downstream Impact
Evaluations on synthetic benchmarks (OurSynDatasets, Flare7K) and real-world images demonstrate that SGLFR-Net outperforms state-of-the-art (SOTA) cascaded methods. Quantitatively, it achieved a PSNR of 29.65 and an SSIM of 0.974 on synthetic joint-flare datasets.
Beyond aesthetic improvements, the paper highlights the benefits of flare removal for computer vision tasks. As shown in the segmentation results, removing flares significantly improves the accuracy of semantic segmentation models, allowing them to correctly identify objects like trees and buildings that were previously obscured by light streaks.
Conclusion and Future Directions
SGLFR-Net represents a significant step toward robust nighttime image restoration. By grounding the architecture in optical physics and leveraging self-supervision, the researchers have created a tool that works "out of the box" for various camera sensors. Future work may focus on reducing the computational overhead of per-image optimization to enable real-time applications and extending the model to handle the more complex lighting conditions found in daytime photography.
This summary is based on the paper "Disentangle Nighttime Lens Flares: Self-supervised Generation-based Lens Flare Removal," presented at AAAI 2025.
Summary updated