Introduction
Under-Display Cameras (UDC) represent a significant milestone for full-screen smartphone design, allowing the selfie camera to be hidden beneath the OLED display. However, this placement introduces severe image degradation. The OLED pixel grid acts as a diffraction grating, causing light to scatter and attenuate before reaching the sensor. This results in two primary artifacts: flare (intense glare in bright regions leading to content loss) and haze (low-contrast blur in darker regions).
While single-image restoration has improved, video restoration remains difficult because the Point Spread Function (PSF) changes dynamically with motion. Standard video restoration models often fail because they treat all degradations uniformly, ignoring the distinct temporal requirements for fixing flare versus haze.
The D2RNet Framework
To address these challenges, the authors introduce D2RNet, the first specialized video restoration framework for UDC. The core philosophy of D2RNet is "decoupling": separating the complex UDC degradation into its constituent flare and haze components to apply targeted restoration strategies.
As shown in the framework overview, the system generates a Flare Map and a Haze Map from the input features. This allows the model to handle flare through Long-Term Flare Removal, which searches across distant frames to recover lost content, and Short-Term Haze Removal, which utilizes adjacent frames for local spatio-temporal sharpening.
Architecture and Decoupling Attention
D2RNet is built as a multi-scale, bi-directional recurrent network. The engine of this network is the Decoupling Attention Module (DAM).
The DAM uses a soft mask generation function based on pixel intensity to split features. By processing the video in both forward and backward directions across three scales ($2\times, 4\times, 8\times$ down-sampling), the network effectively gathers information from the entire sequence. This architecture ensures that the model can handle objects of varying sizes and recover details even when they are temporarily obscured by intense light sources.
Data and Performance
A significant hurdle in UDC research is the lack of large-scale video datasets. The authors addressed this by creating VidUDC33K, a dataset of 677 high-quality HDR videos. They simulated UDC degradation by measuring the real-world PSF of a ZTE Axon 20 system and applying it to the HDR frames using a physics-based model that accounts for motion and saturation.
In benchmarks, D2RNet achieved a state-of-the-art 31.91 dB PSNR on VidUDC33K, outperforming general video restoration models like RVRT by over 1 dB. Notably, D2RNet is highly efficient, utilizing only 5.76M parameters and processing frames in 0.44s, making it significantly faster and lighter than transformer-based alternatives like VRT.
Conclusion
D2RNet demonstrates that non-uniform degradations in UDC video are best handled through specialized decoupling. By recognizing that flare and haze require different temporal "memories," the model achieves superior restoration quality with high efficiency. This approach sets a new benchmark for UDC video processing and offers a blueprint for handling complex, intensity-dependent degradations in other imaging domains.
Summary updated