Bringing Old Photos Back to Life

Ziyu Wan, Bo Zhang, Dongdong Chen 0001, Pan Zhang, Dong Chen 0003, Jing Liao, Fang Wen

Introduction

Restoring old photographs is a complex "blind" restoration task. Unlike standard denoising where a clean ground truth is available, old photos suffer from a combination of physical and chemical degradations—scratches, dust, color fading, and film grain—that are difficult to model synthetically.

Figure 1: A high-quality restoration result showing preserved facial details and corrected optical defects.

This paper introduces a novel deep learning framework designed to bridge the gap between synthetic training data and real-world artifacts by performing translation in a shared latent space.

The Challenge of Mixed Degradations

Old photo restoration faces two primary hurdles. First, the domain gap: networks trained on synthetic noise often fail on real-world photos because digital simulations rarely capture the nuanced textures of century-old film. Second, the presence of mixed degradations: images typically contain both unstructured defects (spatially uniform noise and blur) and structured defects (localized scratches and holes). Most existing methods excel at one but struggle with the other.

Methodology: Triplet Domain Translation

The authors propose a Triplet Domain Translation Network that operates in a shared latent space rather than directly on pixels.

Latent Space Alignment

The framework utilizes two Variational Autoencoders (VAEs). VAE1 encodes both real corrupted images ($R$) and synthetic corrupted images ($X$) into a shared latent space. By mapping both domains to the same compact representation, the domain gap is effectively closed. VAE2 encodes clean ground-truth images ($Y$) into a separate latent space. A translation mapping ($T$) is then learned to convert corrupted latent codes into clean ones.

Handling Structured Defects

To address localized damage like scratches, the network employs a Partial Nonlocal Block. This global branch allows the model to "borrow" information from distant, healthy regions of the image to fill in gaps, while a mask-aware mechanism ensures the network ignores the damaged pixels during reconstruction.

Results and Evaluation

The method was evaluated against state-of-the-art models like Pix2Pix and CycleGAN.

Figure 4: Comparison of original degraded photos (top) and the restoration results (bottom) across various subjects and defect types.

Quantitative Performance

While achieving competitive PSNR and SSIM scores on synthetic benchmarks, the proposed method significantly outperformed competitors in perceptual metrics like LPIPS and FID, which better reflect human judgment of image quality.

Qualitative Impact and User Study

As shown in the comparison above, the framework effectively removes scratches and restores clarity to facial features without introducing artificial smoothing. In a blind user study involving 22 participants, this method was preferred 64.83% of the time, a substantial margin over the next best alternative (14.19%).

Conclusion

By leveraging latent space translation and mask-aware inpainting, this approach provides a robust solution for restoring heavily degraded historical photographs. Future work will focus on addressing complex shading artifacts and irregular stains that the current model sometimes misinterprets as natural shadows.

Reference: Wan, Z., Zhang, B., Chen, D., Zhang, P., Chen, D., Liao, J., & Wen, F. (2020). Bringing Old Photos Back to Life. arXiv:2004.09484v1 [cs.CV].

Summary updated

BibTeX
@inproceedings{wan2020bringing,
  author = {Ziyu Wan and Bo Zhang and Dongdong Chen and Pan Zhang and Dong Chen and Jing Liao and Fang Wen},
  title = {{Bringing Old Photos Back to Life}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2020},
  doi = {10.1109/cvpr42600.2020.00282},
}
Figure