Abstract
Generative Adversarial Networks (GANs) have significantly advanced image inpainting, yet they often struggle with structural and textural inconsistencies. This paper proposes a novel regularization technique called Submanifold Alignment, which utilizes Local Intrinsic Dimensionality (LID) to ensure that restored image regions reside on the same data manifold as the original images. By implementing both image-level and patch-level LID constraints, the model achieves superior global structural consistency and fine-grained textural detail. Experimental results on benchmark datasets like CelebA and Paris StreetView demonstrate that the proposed method outperforms state-of-the-art models in both quantitative metrics and visual quality.
Introduction
Image inpainting involves restoring missing or corrupted parts of an image. While GANs provide a powerful framework for this "ill-posed" problem, they frequently produce results that appear realistic in isolation but fail to integrate seamlessly with the surrounding context. These failures typically manifest as structural inconsistencies (e.g., broken architectural lines) or textural mismatches (e.g., inconsistent hair patterns). Standard approaches relying on pixel-wise reconstruction loss or high-level adversarial loss often fail to bridge the gap between low-level details and high-level semantics. This work introduces a "mid-level" constraint based on the manifold hypothesis to address these issues.
Submanifold Alignment via LID
The core of the proposed approach is the use of Local Intrinsic Dimensionality (LID) as a regularization term. LID characterizes the local properties of the data manifold in a high-dimensional feature space. The authors introduce two levels of alignment:
- Image-level LID (iLID): This measures how closely the submanifold of generated images aligns with the submanifold of original images in deep feature space. This encourages the model to maintain global structural consistency.
- Patch-level LID (pLID): This focuses on smaller $3 \times 3$ feature patches to ensure that local textures are drawn from submanifolds surrounding the original patches, preventing repetitive patterns and improving textural detail.
By adding these two LID terms to the standard GAN loss function, the model is forced to generate content that is mathematically "close" to the distribution of the ground truth in a way that simple pixel-matching cannot capture.
Experimental Results
The LID regularization was evaluated on four benchmark datasets: Paris StreetView, CelebA, Places2, and ImageNet. Quantitative analysis showed that the "GL+LID" model achieved superior Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) scores. For instance, on the CelebA dataset, PSNR improved from 24.46 (using the GMCNN baseline) to 25.56.
Qualitatively, the LID-enhanced model demonstrates a significant improvement in maintaining structural continuity. As shown in the comparison above, while standard models like GMCNN often produce disconnected architectural fragments or blurred textures, the proposed method restores these regions with high fidelity and seamless transitions, effectively continuing the lines of complex structures like the Eiffel Tower.
Conclusion
Submanifold Alignment via LID provides a robust mechanism for guiding generative models toward more consistent and realistic image restoration. By ensuring that generated content adheres to the intrinsic dimensionality of the original data manifold, the proposed method effectively resolves common structural and textural artifacts in image inpainting. Future work may explore the application of this technique to other generative tasks such as super-resolution and style transfer.
Summary updated