CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection

Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas, Paolo Bestagini, Stefano TubaroarXiv 2026 · arXiv

TL;DR

CUPID is a novel, visual-only Person-of-Interest (POI) deepfake detector that combines 3D face reconstruction (UV texture maps) with a Masked Autoencoder (MAE) to learn a robust, general representation of facial identity from real videos alone. By operating in a standardized geometric UV space, it achieves state-of-the-art robustness against heavy video compression and downscaling, runs over 30x faster than existing POI baselines, and provides pixel-level visual explanations of exactly where a video has been manipulated.

What Problem Does This Paper Address?

Deepfakes targeting high-profile individuals—such as politicians, celebrities, and corporate executives—pose severe threats to modern democracies and security. To protect these specific individuals, the forensics community developed Person-of-Interest (POI) deepfake detectors, which leverage the abundance of pristine reference data (e.g., public speeches, interviews) available for a specific target.

However, current POI detectors face a challenging trilemma:

  1. Data Efficiency: Many methods require training a separate, specialized model for every single identity, which is computationally expensive and data-hungry.
  2. Robustness: General and POI-specific detectors often collapse under common real-world post-processing operations, such as heavy video compression or spatial downscaling encountered on social media.
  3. Interpretability: Most deepfake models act as "black boxes," providing a binary authenticity score without explaining why or where a video was altered.

CUPID addresses these gaps by introducing an identity-agnostic model that requires no deepfakes during training, generalizes effortlessly to unseen POIs, resists quality degradation, and naturally localizes facial manipulations.

Methods

The core innovation of CUPID lies in moving away from raw video frames toward a standardized geometric space, as illustrated in the overview below:

Fig. 1. Overview of the proposed CUPID method for POI deepfake detection. During training (top), a masked autoencoder encodes UV texture maps from real videos of many subjects into a general latent representation of facial identity. During inference (bottom), the same encoder maps pristine videos of the POI into this latent space to build a reference identity distribution. A test video, whether real or manipulated, is then encoded and compared against this distribution for authenticity assessment.

1. UV Texture Map Extraction

Instead of feeding raw face crops into a neural network, CUPID fits a 3D Morphable Model (3DMM) to each frame to extract a UV texture map. This process flattens the 3D face structure into a standardized 2D image space where specific pixel coordinates always correspond to the same anatomical features (e.g., the tip of the nose or the left eyelid), completely decoupling facial appearance from changes in pose or expression.

Example of 3D facial reconstruction and UV texture map extraction.

2. Masked Autoencoder (MAE) with Multi-Task Loss

A Vision Transformer (ViT)-based Masked Autoencoder is trained exclusively on these UV maps. During training, 50% of the image patches are randomly masked out. The network is optimized using a joint loss function:

  • Masked Reconstruction Loss: Forces the model to learn fine-grained facial textures by reconstructing missing patches.
  • Perceptual Loss: Utilizes a pretrained VGG-16 network to ensure visual and structural coherence.
  • Multi-Layer Contrastive Loss: Employs a Multi-Similarity Miner to pull embeddings of the same identity closer together while pushing different identities apart.

3. Anomaly-Based Inference & Interpretability

At test time, the model encodes a small set of pristine reference videos of the target POI to establish a "genuine identity distribution" in the latent space. A query video is then encoded and compared against this distribution using a pairwise similarity test.

To provide interpretability, CUPID calculates the latent displacement between the test video's embeddings and the reference centroid. By applying this displacement back to the centroid and decoding it into the UV space, the model isolates exactly which facial regions deviate from the subject's authentic appearance, generating a clean spatial residual map.

Datasets

  • Training: Trained on VoxCeleb2, a large-scale audio-visual corpus. The authors filtered out identities overlapping with the evaluation sets and balanced the dataset to 21 videos per identity to ensure a uniform distribution.
  • Evaluation: Evaluated across four diverse deepfake benchmarks under both Higher Quality (HQ) and Lower Quality (LQ, heavily compressed via H.264 crf40) settings:
  • DF-TIMIT: A controlled face-swapping dataset.
  • FakeAVCeleb: A dataset containing face-swapped and reenacted videos.
  • KoDF: A large-scale dataset focusing on Korean subjects.
  • DeepSpeak: A highly challenging recent dataset featuring advanced generation tools like diffusion models and avatar generators (e.g., LivePortrait).

Key Insights and Ideas

  • UV Space as a Shield Against Compression: Traditional deepfake detectors often rely on high-frequency pixel artifacts that are easily erased by video compression or resizing. Because CUPID maps faces to a structured 3D UV space, it forces the network to learn macro-level structural and identity anomalies rather than fragile pixel-level textures, yielding remarkable robustness.
  • Redundancy Reduction Enables a 32x Speedup: Because UV mapping normalizes out pose and expression variations, frame-to-frame variance drops significantly. The authors discovered that sampling just 10 frames uniformly across an entire video is sufficient to capture a stable identity signature. This eliminates the need for computationally expensive sliding-window architectures.
  • Latent Space Arithmetic for Clean Heatmaps: Instead of relying on noisy gradient-based saliency methods (like Grad-CAM), CUPID performs geometric subtraction directly within the identity latent space. This filters out environmental nuisance factors like illumination and rendering artifacts, leaving a pure signature of the forgery.

Key Results

  • Dominance Under Compression: In HQ settings, CUPID performs competitively with state-of-the-art general detectors. However, in LQ compressed settings, general detectors suffer a catastrophic performance drop (averaging a 24.3 point decline in AUC). In contrast, CUPID's performance drops by a mere 3.2 points, achieving a dominant 86.07% mean POI-level AUC across all compressed datasets.
  • Inference Speed: CUPID achieves an average inference time of 0.155 seconds per video, outperforming the state-of-the-art POI-Forensics baseline (4.961 seconds) by 32x.
  • Threshold Stability: The decision thresholds required to maximize accuracy are tightly clustered across different subjects, meaning a single global threshold can be deployed reliably without tedious per-person calibration.

Limitations and Future Work

  • Identity-Centric / Visual-Only Focus: CUPID focuses strictly on static facial appearance and structure. It does not model temporal dynamics or audio-visual synchronization. Consequently, it struggles against sophisticated LipSync or audio-driven manipulations that alter speech movements while perfectly preserving the target's static identity.
  • Demographic Sensitivity: The model showed a performance dip on the KoDF dataset, which features Korean subjects. The authors attribute this to an ethnicity distribution mismatch, as East Asian demographics are underrepresented in the VoxCeleb2 training set.

Future Directions: The authors plan to extend the framework by incorporating temporal objectives to capture behavioral biometrics and training on more ethnically diverse datasets to improve fairness and generalization.

Summary updated

Abstract

Deepfakes targeting a high-profile individual, known as Person-of-Interest (POI), are a threat to modern democracies and societies. Current POI deepfake detection methods still struggle to combine robustness to post-processing, efficiency and interpretability, key aspects of modern deepfake detectors. In this paper we propose CUPID, a POI video deepfake detector that combines UV texture maps, a facial appearance representation derived from 3D face reconstructions, with the representation learning capabilities of the Masked Autoencoder (MAE). Our method does not require any deepfake videos in its training phase. Moreover, it does not even require including a specific POI in the training set: the combination of UV texture maps extracted from real video frames and the MAE context-guided reconstruction yields a latent space that captures rich and discriminative facial features even for identities unseen during training. In the testing phase, the embeddings extracted from a query video depicting the POI can be matched against pristine reference videos to assess the video authenticity. Furthermore, operating in the UV space naturally provides an additional layer of interpretability. Specifically, we can extract decoded residual maps that highlight which facial regions of a test video deviate most from the identity representation of the corresponding POI. Experiments on four deepfake datasets show that CUPID outperforms the current state of the art on most datasets and achieves the best overall robustness against strong downscaling and compression, while also providing substantially faster inference. Our experimental code will be released at https://github.com/polimi-ispl/CUPID.
BibTeX
@misc{affatato2026cupid,
  author = {Giovanni Affatato and Sara Mandelli and Edoardo Daniele Cannas and Paolo Bestagini and Stefano Tubaro},
  title = {{CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection}},
  year = {2026},
  eprint = {2606.20302},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.20302},
}
Figure