Abstract
Learning-based algorithms for image inpainting have made significant strides in handling squared or irregular holes. However, generating plausible textures within damaged areas remains challenging due to a lack of surrounding information, often resulting in central blurriness. This paper proposes a Full-Resolution Residual Network (FRRN) designed for progressive image inpainting. By restoring the image in stages and updating masks iteratively, the FRRN facilitates better feature integration and texture prediction. We introduce a "N Blocks, One Dilation" strategy and a corresponding step loss function to ensure high-quality intermediate restorations. Experimental results on Places2 and CelebA datasets demonstrate that FRRN outperforms existing methods both quantitatively and qualitatively.
Introduction
Image inpainting is the process of reconstructing missing or corrupted regions in an image. While deep learning has improved this field, most models attempt to fill holes in a single pass. For large or irregular gaps, this "one-shot" approach often fails because the network lacks sufficient context for pixels far from the hole's boundary, leading to muddy and incoherent textures.
A more effective strategy is progressive inpainting: filling the edges of the hole first, then using that newly generated information to fill the center. This paper introduces the Full-Resolution Residual Network (FRRN) to implement this progressive approach effectively.
Methodology
Full-Resolution Residual Block (FRRB)
The core of our architecture is the FRRB, which maintains two parallel branches to balance local texture and global structure:
- High-Resolution Branch: Operates at the original image resolution to preserve fine-grained details.
- Feature Integration Branch: Uses an encoder-decoder structure (downsampling and upsampling) to capture global context.
Progressive Dilation Strategy
Instead of filling the entire mask at once, the network "erodes" the mask step-by-step. As the image passes through successive modules, the "clean" areas expand into the "damaged" regions.
N Blocks, One Dilation
To prevent error propagation from premature mask updates, we adopt a strategy where $N=2$ residual blocks process the features before a single mask dilation step occurs. This allows the network to "settle" on a high-quality texture before moving deeper into the hole.
Step Loss Function
To guarantee the quality of intermediate stages, a loss function is applied at every dilation step. This constrains the network to produce plausible results throughout the entire reconstruction pipeline, rather than just at the final output.
Experimental Results
The FRRN was evaluated on the Places2 and CelebA datasets using a set of 12,000 irregular masks.
Quantitative Performance
On the Places2 dataset with 10%-20% mask coverage, FRRN achieved a PSNR of 29.86, significantly outperforming EdgeConnect (27.60) and Partial Convolutions (28.02). It also showed superior performance in SSIM and $L_1$ loss across various mask sizes.
Qualitative Performance
Visually, FRRN produces much sharper textures in the center of large holes compared to previous methods. By treating inpainting as a morphological dilation process, it effectively "connects" broken contours and maintains structural consistency.
Conclusion
The proposed FRRN framework demonstrates that a progressive, residual-based approach is highly effective for irregular image inpainting. By maintaining full-resolution features and employing a multi-step dilation strategy with intermediate supervision, the model overcomes the common issue of central blurriness. Future work will focus on improving parameter efficiency and exploring adaptive dilation strides for even larger missing regions.
The code is available at: https://github.com/ZongyuGuo/Inpainting_FRRN.
Summary updated