This paper tackles a critical flaw in how we train AI to recognize faces across different ethnicities, introducing a novel framework for Unsupervised Domain Adaptation (UDA) specifically designed for distance metric learning.
TL;DR
Most domain adaptation techniques fail when the source and target datasets contain entirely different categories (e.g., different people). This paper introduces the Feature Transfer Network (FTN), which reformulates classification as a verification task to bridge disjoint label spaces. This significantly improves face recognition accuracy for underrepresented ethnicities without requiring new labels.
The Problem: Disjoint Label Spaces and Demographic Bias
Deep learning models for face recognition often suffer from "demographic bias." Because popular datasets like MS-Celeb-1M are predominantly Caucasian, models perform poorly on African-American or East-Asian faces.
Standard UDA techniques usually try to solve domain shifts by making the features of the "source" (Caucasian) and "target" (non-Caucasian) domains indistinguishable. However, there is a logical trap here: in face recognition, different people should have different features. If the source domain contains Person A and the target domain contains Person B, forcing their feature distributions to overlap actually destroys the model's ability to tell individuals apart. The authors identify that we need a way to transfer knowledge about how to distinguish faces without forcing the actual identities to merge in the embedding space.
Methods: Bridging the Gap
The authors propose three key innovations to solve the disjoint label problem:
- Verification Reformulation: Instead of treating the problem as a multi-class classification task (which requires shared labels), they treat it as a binary verification task: "Are these two images the same person?" This task is universal across all domains, regardless of which specific identities are present.
- Feature Transfer Network (FTN):
The architecture uses a shared feature generator ($f$) but adds a "Feature Transfer" module ($g$). While the generator keeps source and target features separate (to maintain identity), the transfer module aligns a transformed version of the source space with the target space. This allows the model to learn the "style" of the target domain while keeping the "content" (identities) distinct.
- Multi-class Entropy Minimization (MCEM): To boost performance on the unlabeled target domain, the authors use an offline clustering algorithm (HDBSCAN) to discover "pseudo-identities." They then apply a powerful metric learning loss ($N$-pair loss) to these clusters, encouraging the model to push different pseudo-identities apart and pull the same ones together.
Datasets and Key Insights
The researchers evaluated their approach on both a controlled proof-of-concept and real-world face recognition datasets.
- MNIST-M to MNIST (Proof of Concept): A toy setup where the source domain contains digits 0–4 and the target domain contains digits 5–9. This tests the model's ability to adapt when classes are entirely disjoint.
- Cross-Ethnicity Faces (CEF): A dataset derived from MS-Celeb-1M, adapting from Caucasian (CAU) source identities to African-American (AA) and East-Asian (EA) target identities.
Key Results
- Visual Clarity:
In the MNIST experiment, t-SNE visualizations showed that standard adversarial adaptation (DANN) caused significant confusion between digits (e.g., mixing 3s and 5s). In contrast, the proposed FTN maintained 10 clean, separate clusters, proving that alignment should happen in a transformed space rather than the primary embedding space.
- Face Recognition Gains: On the CEF dataset, the FTN+MCEM model significantly outperformed the "source-only" baseline. For example, identification accuracy on African-American faces jumped from 69.64% (baseline) to 80.75% (FTN+MCEM).
- Closing the Gap: The proposed method's performance closely approached the "upper bound" (a model trained with full labels for all ethnicities), proving that unsupervised adaptation can be nearly as effective as manual labeling.
Limitations and Future Work
- Clustering Dependency: The success of the MCEM module depends on the quality of the initial clustering. If the feature space is initially very poor, HDBSCAN might produce noisy pseudo-labels that hinder learning.
- Computational Overhead: The process involves offline clustering and complex $N$-pair sampling, which is more computationally intensive than standard classification-based adaptation.
- Generalization: While highly effective for faces, the paper doesn't explore how this framework handles other metric learning tasks, such as fine-grained image retrieval (e.g., identifying specific car models or bird species).
Summary updated