Dual-Triplet Metric Learning for Unsupervised Domain Adaptation in Video Face Recognition

George S. Eskander Ekladious, Hugo Lemoine, Eric Granger, Kaveh Kamali, Salim Moudache

Introduction

Deploying face recognition models to new surveillance cameras often results in a significant drop in accuracy due to "domain shift"—variations in lighting, camera angles, and environmental conditions. Traditionally, correcting this requires manually labeling faces from the new environment, an expensive and impractical task for large-scale networks. This paper introduces Dual-Triplet Metric Learning (DTML), an unsupervised domain adaptation (UDA) technique that allows models to adapt to new cameras using only a small amount of unlabeled video data.

Methodology: Dual-Triplet Metric Learning

Face recognition typically employs Siamese networks trained with a triplet loss function, which learns to minimize the distance between images of the same person while maximizing the distance between different people. To adapt to a new, unlabeled "target" domain, DTML optimizes two triplet losses simultaneously:

  1. Source Triplet ($L_s$): Uses labeled data from the original training environment to maintain a discriminative baseline.
  2. Target Triplet ($L_t$): Uses unlabeled data from the new camera.
The DTML architecture overview, showing the dual-triplet loss structure and the mutual supervision mechanism between source and target domains.

Mutual Supervision

To generate target triplets without ground-truth labels, the authors propose a Mutual Supervision approach. A pre-trained source model acts as a "teacher," calculating the statistical distributions (mean and standard deviation) of "within-class" (same person) and "between-class" (different people) distances in the source data. These statistics define "mining windows." When unlabeled target data is processed, pairwise distances falling within these windows are pseudo-labeled as positive or negative pairs to construct the target triplet loss.

Key Insights

  • Distance Distribution Alignment: For open-set problems like face recognition, it is more effective to align pairwise distance distributions rather than absolute feature representations. By using source domain confidence intervals, the model can safely mine pairs from unlabeled target data.
  • Preventing Catastrophic Forgetting: Relying solely on target triplet loss ($L_t$) can lead to overfitting on noisy pseudo-labels, causing the model to collapse. Maintaining the source triplet loss ($L_s$) acts as an anchor, ensuring the underlying feature representation remains robust during adaptation.
Performance comparison (AUC) over training steps on the COX dataset. The combined loss (Ls+Lt) provides superior stability and accuracy compared to using either loss in isolation.

Experimental Results

The authors evaluated DTML on both public and private datasets:

  • COX Face Dataset: In a cross-camera adaptation scenario (Camera 3 to Camera 2), the baseline model achieved 87.2% accuracy. DTML improved this to 91.0%, nearing the theoretical upper bound of 95.3% achievable with full supervision.
  • Real-World IP Camera: When adapting a lab-trained model to a commercial surveillance camera, the baseline True Positive Rate (at 1% FAR) was 52%. DTML adaptation increased this to 73%. By adding an extra classification layer trained on the adapted representations, accuracy reached 84%.

Conclusion and Future Work

DTML provides a practical path for deploying face recognition in dynamic environments without manual labeling. A current limitation is the requirement for some initial overlap between source and target distance distributions. Future research will explore adversarial learning to force an initial baseline alignment before applying the statistical pseudo-labeling step, as well as end-to-end training of the entire architecture.

Summary updated

Abstract

The scalability and complexity of deep learning models remains a key issue in many of visual recognition applications. For instance, in video surveillance, fine tuning of a model with labeled image data from each new camera is required to reduce the domain shift between videos captured from the source domain (laboratory setting) and the target domain (operational environment). In many video surveillance applications, like face recognition and person re-identification, a pair-wise matcher is typically employed to assign a query image captured using a video camera to the corresponding reference images in a gallery. The different configuration, viewpoint, and operational conditions of each camera can introduce significant shifts in pair-wise distance distributions, resulting in a decline in recognition performance for new cameras. In this paper, a new deep domain adaptation (DA) method is proposed to adapt the CNN embedding of a Siamese network using unlabeled tracklets captured with a new video camera. To this end, a dual-triplet loss is introduced for metric learning, where two triplets are constructed using video data from a source camera, and a new target camera. In order to constitute the dual triplets, a mutual-supervised learning approach is introduced where the source camera acts as a teacher, providing the target camera with an initial embedding. Then, the student relies on the teacher to iteratively label the positive and negative pairs collected during, e.g., initial camera calibration. Both source and target embeddings continue to simultaneously learn such that their pair-wise distance distributions become aligned. For validation, the proposed metric learning technique is used to train deep Siamese networks under different training scenarios, and is compared to state-of-the-art techniques for still-to-video FR on the COXS2V and a private video-based FR dataset. Results indicate that the proposed method can provide a level of accuracy that is comparable to the upper bound performance, in training scenario where labeled target data is employed to fine-tune the Siamese network.
BibTeX
@inproceedings{ekladious2020dual,
  author = {George S. Eskander Ekladious and Hugo Lemoine and Eric Granger and Kaveh Kamali and Salim Moudache},
  title = {{Dual-Triplet Metric Learning for Unsupervised Domain Adaptation in Video Face Recognition}},
  booktitle = {IJCNN},
  year = {2020},
  doi = {10.1109/ijcnn48605.2020.9206794},
}

Loading related content...

Figure