TL;DR
This paper introduces DFPGnet, a novel face deblurring framework that leverages deep features from pre-trained face recognition networks (like VGGFace) as a powerful prior. By disentangling feature estimation and deblurring into separate streams and introducing feature transform modules, DFPGnet effectively bridges the gap between recognition-focused features and deblurring needs, achieving state-of-the-art results.
What Problem Does This Paper Address?
Face deblurring is a challenging ill-posed problem where recovering sharp facial details from a blurred image is crucial for various downstream computer vision tasks like face recognition and alignment. While existing methods often rely on geometric priors (e.g., landmarks, parsing maps), these are insufficient for capturing fine-grained texture details. Although perceptual losses using deep features from recognition networks have shown promise, they can be limited by the inherent differences between recognition and deblurring tasks. This paper addresses the gap of effectively utilizing rich deep features as a prior for face deblurring, going beyond simple perceptual losses to better capture both shape and texture information.
Methods
The core of the proposed method is the Deep Feature Prior Guided network (DFPGnet), built upon a Generative Adversarial Network (GAN) framework. DFPGnet comprises two main streams:
- Prior Estimation Stream: This stream takes features extracted from a blurred image by an encoder and estimates a "deep feature prior." This prior is guided by ground truth deep features from a pre-trained VGGFace network.
- Deblurring Stream: This stream takes the estimated deep feature prior and other intermediate features from the encoder to reconstruct the sharp face image.
Key novelties include:
- Disentangled Streams: Separating the prior estimation and deblurring processes allows each stream to focus on its specific task, improving overall performance.
- Feature Transform Modules (SSFT and SFT): To address the discrepancy in feature distributions between the recognition-focused VGGFace and the deblurring task, Self-Spatial Feature Transform (SSFT) and Spatial Feature Transform (SFT) modules are introduced. These modules adaptively modulate features to better suit the deblurring stream.
- Channel-Attention Feature Discriminator and Prior Loss: A novel discriminator is proposed that uses channel attention to learn the importance of different feature channels for deblurring. This learned importance is then used to define a "prior loss" that encourages the generator to focus on more relevant channels, effectively distilling useful knowledge from the deep feature prior.
Datasets
The paper evaluates its method on two standard facial deblurring datasets:
- Shen test set [32]: This dataset consists of 16,000 blurry face images synthesized using Helen [15] and CelebA [22].
- MSPL test set [18]: This dataset is further divided into MSPL-Center and MSPL-Random subsets, containing centered and randomly transformed face images, respectively.
The VGGFace [27] network, trained on the VGG Face dataset, is used to extract ground truth deep features.
Key Insights and Ideas
- Deep features are powerful priors, but need careful handling: While deep features from face recognition networks contain rich information, they are not directly optimal for deblurring. Simply using them as a perceptual loss is insufficient.
- Disentanglement is key: Separating the process of estimating the deep feature prior from the actual deblurring allows each component to specialize, leading to better results.
- Feature transformation bridges the domain gap: The SSFT and SFT modules are crucial for adapting features from the recognition domain to the deblurring domain, preventing interference.
- Channel importance matters: Not all deep features are equally useful for deblurring. The channel-attention mechanism effectively identifies and leverages the most informative channels.
- Robustness to misalignment: The proposed deep feature prior is more robust to non-aligned faces compared to geometric priors, as it captures local cues less dependent on precise alignment.
Key Results
- State-of-the-art performance: DFPGnet achieves state-of-the-art results on both the Shen and MSPL datasets, outperforming existing methods in PSNR, SSIM, and dVGG metrics.
- Qualitative improvements: Visual results show that DFPGnet restores sharper edges, finer textures, and more detailed facial components, closely matching ground truth images.
- Improved downstream task performance: The deblurred images generated by DFPGnet lead to improved accuracy in face detection and verification tasks, demonstrating their utility for practical applications.
- Ablation studies confirm component effectiveness: Experiments show that the deep feature prior alone is not sufficient; feature transformation modules and the channel-attention discriminator are vital for achieving optimal performance.
Limitations and Future Work
- Generalization to other pre-trained networks: The authors suggest that their method could be generalized to utilize deep features from other pre-trained networks for diverse restoration tasks.
- Real-world blur: While the paper shows promising results on real-world blurred images, further investigation into handling diverse and severe real-world blurs could be beneficial.
- Computational cost: While not explicitly detailed as a limitation, the multi-stream architecture and GAN framework might imply a higher computational cost compared to simpler methods.
Summary updated