Introduction
Restoring old photographs is a complex "blind" restoration task. Unlike standard denoising where a clean ground truth is available, old photos suffer from a combination of physical and chemical degradations—scratches, dust, color fading, and film grain—that are difficult to model synthetically.
This paper introduces a novel deep learning framework designed to bridge the gap between synthetic training data and real-world artifacts by performing translation in a shared latent space.
The Challenge of Mixed Degradations
Old photo restoration faces two primary hurdles. First, the domain gap: networks trained on synthetic noise often fail on real-world photos because digital simulations rarely capture the nuanced textures of century-old film. Second, the presence of mixed degradations: images typically contain both unstructured defects (spatially uniform noise and blur) and structured defects (localized scratches and holes). Most existing methods excel at one but struggle with the other.
Methodology: Triplet Domain Translation
The authors propose a Triplet Domain Translation Network that operates in a shared latent space rather than directly on pixels.
Latent Space Alignment
The framework utilizes two Variational Autoencoders (VAEs). VAE1 encodes both real corrupted images ($R$) and synthetic corrupted images ($X$) into a shared latent space. By mapping both domains to the same compact representation, the domain gap is effectively closed. VAE2 encodes clean ground-truth images ($Y$) into a separate latent space. A translation mapping ($T$) is then learned to convert corrupted latent codes into clean ones.
Handling Structured Defects
To address localized damage like scratches, the network employs a Partial Nonlocal Block. This global branch allows the model to "borrow" information from distant, healthy regions of the image to fill in gaps, while a mask-aware mechanism ensures the network ignores the damaged pixels during reconstruction.
Results and Evaluation
The method was evaluated against state-of-the-art models like Pix2Pix and CycleGAN.
Quantitative Performance
While achieving competitive PSNR and SSIM scores on synthetic benchmarks, the proposed method significantly outperformed competitors in perceptual metrics like LPIPS and FID, which better reflect human judgment of image quality.
Qualitative Impact and User Study
As shown in the comparison above, the framework effectively removes scratches and restores clarity to facial features without introducing artificial smoothing. In a blind user study involving 22 participants, this method was preferred 64.83% of the time, a substantial margin over the next best alternative (14.19%).
Conclusion
By leveraging latent space translation and mask-aware inpainting, this approach provides a robust solution for restoring heavily degraded historical photographs. Future work will focus on addressing complex shading artifacts and irregular stains that the current model sometimes misinterprets as natural shadows.
Reference: Wan, Z., Zhang, B., Chen, D., Zhang, P., Chen, D., Liao, J., & Wen, F. (2020). Bringing Old Photos Back to Life. arXiv:2004.09484v1 [cs.CV].
Summary updated