Abstract
Image inpainting often suffers from structural and textural inconsistencies, where generated regions fail to align with the global context or local patterns. This paper proposes a novel regularization technique called Submanifold Alignment, which utilizes Local Intrinsic Dimensionality (LID) to constrain GAN-based models. By forcing generated content to reside on the low-dimensional submanifold of real data, the model achieves significantly higher fidelity and consistency than standard adversarial approaches.
Introduction
Image inpainting—the task of filling missing or corrupted regions in an image—is inherently a "one-to-many" problem. Because multiple plausible solutions exist for any given hole, models often struggle with mid-level consistency. While Generative Adversarial Networks (GANs) have improved the sharpness of results, they frequently fail to bridge the gap between pixel-level similarity (reconstruction loss) and high-level realism (adversarial loss).
Standard models often produce sharp but contextually inappropriate content, such as filling a gap in a building with sky textures because the model knows sky looks "real," even if it violates the geometry of the scene. This research addresses this gap by ensuring that generated content is structurally and texturally aligned with the original data distribution.
Methodology: Submanifold Alignment via LID
The core innovation is the introduction of Submanifold Alignment using Local Intrinsic Dimensionality (LID). The authors argue that natural images lie on a low-dimensional manifold within the high-dimensional pixel space. To produce natural-looking results, the generated content must align closely with this manifold.
The authors propose two specific regularization terms:
- Image-level LID (iLID): This measures the alignment of the generated image's submanifold with that of the original data in a deep feature space. It serves as a guide for global structural consistency, ensuring that large-scale features like architectural lines are correctly reconstructed.
- Patch-level LID (pLID): Applied to local $3 \times 3$ patches, pLID encourages textural consistency. It prevents common artifacts such as repetitive patterns or mismatched textures by ensuring local geometry matches real image patches.
To make this computationally feasible for end-to-end GAN training, the authors developed a batch-based estimation method for LID, allowing it to be integrated directly into the loss function alongside standard adversarial and reconstruction terms.
Experimental Results
The proposed approach was evaluated on four benchmark datasets: Paris StreetView, CelebA, Places2, and ImageNet. The model, integrated with a Global & Local (GL) GAN architecture (referred to as GL+LID), was compared against state-of-the-art baselines like GMCNN.
Quantitative Performance
On the CelebA face dataset, the GL+LID model demonstrated superior performance, achieving a PSNR of 25.56 and an SSIM of 0.953. This outperformed the GMCNN baseline, which scored 24.46 and 0.944, respectively.
Qualitative Comparison
Visual analysis confirms that Submanifold Alignment effectively handles complex geometries and textures. As shown in the comparison below, the proposed method successfully reconstructs intricate structures like the lattice of the Eiffel Tower and facial features with higher precision than previous methods, which often produce blurred regions or broken lines.
Discussion and Conclusion
The creative application of LID as a generative constraint marks a significant shift from its traditional use in adversarial attack detection. By focusing on the "local geometry" of data, the model moves beyond simple realism to achieve contextual coherence. Ablation studies indicate that iLID and pLID are complementary: iLID handles the global structure, while pLID ensures local texture fidelity.
While the method shows great promise, it is sensitive to the neighborhood size parameter ($k$) used in LID estimation. Future work may explore the integration of submanifold alignment into newer transformer-based generative architectures. Despite the added computational complexity of LID calculation, the resulting improvements in visual consistency provide a compelling case for its use in high-fidelity image synthesis.
Summary updated