Introduction
Under-display cameras (UDC) represent a significant milestone toward achieving truly bezel-free smartphone displays. By embedding the camera sensor beneath the OLED pixel grid, manufacturers can eliminate notches and punch-holes. However, this hardware configuration introduces severe optical degradation. The gaps between OLED pixels act as a complex diffraction grating, leading to blurry images characterized by prominent flares and streaks around light sources. Furthermore, the light passing through multiple display layers undergoes significant attenuation and non-uniform color shifts.
Addressing the Diffraction Challenge
Traditional restoration methods primarily operate in the spatial domain, treating UDC restoration as a complex deblurring task. While deep neural networks have shown promise, they often struggle to capture the periodic nature of diffraction patterns. The Frequency and Spatial Interactive Learning Network (FSI) addresses this by recognizing that diffraction artifacts, while chaotic in pixel space, exhibit highly structured and predictable characteristics in the frequency domain. By "cleaning" the Fourier spectrum, the model can remove artifacts that spatial-only models struggle to perceive.
Methodology: The FSI Architecture
The core of the FSI framework is a dual-stream architecture designed to leverage the complementary strengths of frequency and spatial representations.
Frequency-Spatial Joint (FSJ) Module
The FSJ module serves as the primary building block, facilitating interaction between two specialized streams:
- Frequency Learning Block (FLB): Utilizing the Fast Fourier Transform (FFT), this block targets global periodic artifacts. Since diffraction streaks manifest as specific spectral biases, the FLB employs fully connected layers to identify and suppress these high-frequency distortions.
- Spatial Learning Block (SLB): To preserve fine-grained local textures, the SLB uses a multi-distillation structure. This ensures that while the frequency domain handles global flare removal, local details are accurately reconstructed.
- Dual Transfer Unit (DTU): The DTU acts as a bridge between the two streams. By using transfer gates, it allows for the selective exchange of information, ensuring that spatial cues guide frequency restoration and vice versa.
Color Transform (CT) Module
To mitigate the color shifts inherent in UDC systems, the authors implemented a lightweight Color Transform module. This module uses a U-Net to predict per-pixel coefficients (a 3x3 matrix and a bias vector), enabling dynamic adjustment of color temperature and brightness across the image to match the ground truth.
Experimental Results
The FSI model was evaluated against state-of-the-art (SOTA) methods on three major benchmarks: P-OLED, T-OLED, and a large-scale synthetic dataset (SYNTH).
Quantitative Performance
FSI demonstrated superior performance across all metrics. On the SYNTH dataset, it achieved a PSNR of 46.14 dB, outperforming the previous best (BNUDC) by 0.36 dB. On the real-world P-OLED dataset, FSI showed an even more significant improvement of 0.46 dB PSNR over existing SOTA models.
Qualitative Analysis
Visual comparisons highlight FSI's ability to effectively suppress diffraction flares that other models leave behind. As shown in Figure 1, the frequency analysis reveals how FSI (blue line) closely tracks the ground truth (green line) compared to the raw UDC image (red line) or competing methods like DISCNet and BNUDC.
Efficiency
Despite its superior performance, FSI is remarkably efficient. With only 3.8 million parameters, it is significantly lighter than many spatial-only models. Its inference speed (0.03s to 0.07s per image) makes it highly suitable for real-time applications on mobile devices.
Conclusion and Future Directions
FSI proves that integrating frequency-domain learning is a powerful strategy for addressing periodic optical artifacts like UDC diffraction. While the model sets a new benchmark, challenges remain in handling extreme sensor saturation where information is physically lost. Future work may involve applying these interactive learning techniques directly to RAW sensor data to further enhance restoration quality by avoiding artifacts introduced by standard Image Signal Processors (ISPs).
Summary updated