Introduction
Image inpainting—the process of reconstructing missing or corrupted parts of an image—has evolved significantly with the advent of Generative Adversarial Networks (GANs). Despite progress, two persistent challenges remain: structural inconsistency and visual blurriness. When large semantic regions (like facial features or architectural elements) are missing, standard models often generate textures that are locally plausible but globally incoherent, leading to "dislocated" features.
The Dense Multi-scale Fusion Network (DMFN)
To address these issues, the authors introduce the Dense Multi-scale Fusion Network (DMFN). This one-stage model is designed to capture massive receptive fields without the computational burden of massive convolution kernels or the "sparse" artifacts of standard dilated convolutions.
Core Innovations
- Dense Multi-scale Fusion Block (DMFB): The DMFB utilizes a four-way branch system that cumulatively adds features from different dilation rates. This "dense" approach allows the network to maintain a large receptive field while keeping the parameter count low (0.47M compared to 0.8M for standard versions).
- Self-guided Regression Loss: By calculating a "discrepancy map" between the prediction and ground truth, the model creates a spatial attention mechanism. This forces the loss function to prioritize the most difficult areas to reconstruct.
- Geometrical Alignment Constraint: Unlike traditional pixel-wise losses, this constraint focuses on the spatial location of high-level features. By penalizing the distance between predicted and ground-truth feature centers, the model ensures that semantic components like eyes or noses are placed correctly.
- Relativistic Average GAN (RaGAN): The use of RaGAN provides a more stable training objective for the discriminator, resulting in sharper, more photo-realistic textures.
Experimental Results
The DMFN was evaluated on several challenging benchmarks, including Paris Street View, Places2, CelebA-HQ, and FFHQ.
Quantitative and Qualitative Performance
Quantitatively, DMFN achieved a state-of-the-art LPIPS (Learned Perceptual Image Patch Similarity) score of 0.0460 on CelebA-HQ, outperforming models like GMCNN (0.0509). A lower LPIPS score indicates that the generated images are perceptually closer to the ground truth.
In user studies, DMFN was preferred over 81% of the time for the FFHQ dataset and 78% for Paris Street View. The model's efficiency is also a key highlight, achieving these results with fewer parameters than its multi-column or large-kernel competitors.
Conclusion
The DMFN demonstrates that "dense" context fusion and geometric constraints are vital for high-quality image inpainting. By focusing on where features should be as much as what they look like, the model avoids common structural pitfalls. Future work may explore integrating explicit structural priors, such as edge or contour maps, to further refine boundary transitions while maintaining the efficiency of a one-stage architecture.
Summary updated