AlbumVSR: Identity-Preserving Face Video Super-Resolution from a Personal Photo Album

Nianyao Du, Xijun Wang, Stanley H. ChanarXiv 2026 · arXiv

TL;DR

AlbumVSR is a new approach to face video super-resolution that uses a person’s private photo album as a high-quality reference to restore their identity in heavily degraded videos. By conditioning a frozen, one-step streaming diffusion transformer on identity and texture tokens derived from these photos, the system achieves state-of-the-art identity preservation and visual fidelity while maintaining near real-time performance.

What Problem Does This Paper Address?

When video quality is extremely poor—due to heavy blur, downsampling, or aggressive compression—the subtle facial features that define a person's identity are often erased. Standard generative video super-resolution (VSR) models tend to "hallucinate" plausible but generic faces, leading to "identity drift" where the restored person no longer looks like themselves.

While some existing methods use a single reference photo, they often struggle with temporal consistency in video or require expensive per-identity fine-tuning. AlbumVSR addresses the challenge of leveraging a collection of photos (an album) to provide a robust identity prior that works across different poses and lighting conditions in a streaming video context.

Methods

The core of AlbumVSR is a frozen, one-step streaming diffusion transformer (based on FlashVSR-tiny) that is augmented with two specialized conditioning channels:

  1. Channel 1: Album Identity Aggregation: The system encodes album images using MagFace (for identity/quality) and CLIP (for appearance). These are aggregated into four permutation-invariant "identity tokens." This allows the model to handle any number of album photos and focus on the highest-quality identity cues.
  2. Channel 2: Album Texture Prior: To provide view-specific details, the researchers fuse album images into a 3D UV texture atlas (using the FLAME model). For every video window, they render this atlas at the estimated face pose to produce 64 "texture tokens." This ensures the model has access to the right textures for the specific angle the person is facing in the video.
Fig. 1. Overview of AlbumVSR. The album is encoded once for each identity. Four set tokens carry identity information, while fused texture, visibility, and displacement maps are rendered and tokenized for each streaming window. Every frozen DiT block appends the resulting album K/V to its cached prompt K/V. Training updates only the album heads and LoRA parameters.

These tokens are injected into the transformer’s cross-attention layers. To make the frozen backbone compatible with this new information, the authors use LoRA (Low-Rank Adaptation), training only about 1.3% of the total parameters.

Datasets

The authors evaluated AlbumVSR on an identity-disjoint benchmark derived from VoxCeleb2.

  • Training/Test Split: 488 identities (355 for training, 91 for testing).
  • Degradation: A "heavy" track involving two passes of blur, noise, and JPEG artifacts, followed by 4× downsampling and H.264/H.265 compression.
  • Albums: Each identity has a session-disjoint album containing 3–21 face crops from different videos of the same person.

Key Insights and Ideas

  • UV Fusion for View Alignment: Instead of just feeding raw photos, projecting them onto a 3D UV atlas allows the model to "look up" the correct facial texture for any given head pose in the video.
  • Quality-Weighted Aggregation: Not all album photos are equal. By using MagFace quality scores to weight the identity embeddings, the model avoids being misled by blurry or poorly lit reference photos.
  • Efficiency via Frozen Backbones: By keeping the massive diffusion transformer frozen and using LoRA, the researchers achieved high-quality results without the massive computational overhead of training a video model from scratch.
  • Permutation Invariance: The identity aggregation method works regardless of the order or number of photos in the album, making it practical for real-world personal libraries.

Key Results

AlbumVSR outperformed 14 baselines across identity, fidelity, and temporal metrics.

  • Identity Preservation: It raised the ArcFace similarity (ID-GT) from 0.586 (frozen backbone) to 0.670, significantly beating specialized models like SeedVR2 (0.642) and RGFVR (0.641).
  • Visual Fidelity: It improved face PSNR from 24.98 dB to 26.71 dB.
  • Speed: On an RTX 4070 Ti, the model runs at approximately 0.07 seconds per frame, matching the speed of the frozen backbone despite the added conditioning.
Fig. 2. Top: Four heavy-degradation examples: id00068/047, id00109/013, id00456/050, and id00560/010. We show GT, degraded input, FlashVSR, SeedVR2, SVFR, InstantRestore, FaceMe, RGFVR, and AlbumVSR. Labels report per-frame ArcFace ID-GT/ID-alb. The GT ID-alb values show the similarity between the target frames and the album. Table 1 reports all 14 baselines on 91 test identities. Bottom: Qualitative ablations. Left: null, wrong, and correct albums using a separate 14k three-channel model. Right: Ch.1-only versus Ch.1+Ch.2 at 20k. Values are per-frame scores. Table 2 reports aggregate results.

Limitations and Future Work

  • Synthetic vs. Real Degradation: The benchmark uses synthetic degradation; the model's robustness to real-world artifacts like sensor noise or old film grain remains to be fully tested.
  • Preprocessing Overhead: The reported 0.07s/frame does not include the time required for offline UV atlas construction or 3D face fitting (DECA), meaning it isn't yet a true "end-to-end" live-streaming solution.
  • Album Quality: The current study uses relatively clean, curated albums. Future work could explore how the model handles "contaminated" albums (e.g., photos of different people or extremely low-quality references).
  • Light Degradation: On the "light" degradation track, some generic models like SeedVR2 still hold a slight edge in identity similarity, suggesting room for improvement in balancing prior knowledge with input evidence.

Summary updated

Abstract

Severe blur, downsampling, noise, and compression can erase the identity cues needed to faithfully restore a person's face. We therefore use photographs of the same person as an external source of identity. AlbumVSR conditions a frozen, one-step streaming diffusion transformer on a personal album. We use four MagFace/CLIP set tokens to encode the album, and 64 texture tokens which come from a fused UV atlas rendered at the current window's estimated face pose. These tokens are appended to the transformer's cached cross-attention context. On an identity-disjoint VoxCeleb2 benchmark with heavy synthetic degradation, AlbumVSR raises ArcFace similarity to the ground truth from 0.586 to 0.670 and face PSNR from 24.98 to 26.71 dB. On the 91 test identities, its mean identity similarity is higher than that of all 14 baselines.
BibTeX
@misc{du2026albumvsr,
  author = {Nianyao Du and Xijun Wang and Stanley H. Chan},
  title = {{AlbumVSR: Identity-Preserving Face Video Super-Resolution from a Personal Photo Album}},
  year = {2026},
  eprint = {2610.39001},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.39001},
}
Figure