Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks

Alexander Unnervik, Hatef Otroshi-Shahreza, Anjith George, Sébastien Marcel

Introduction

Detecting backdoor (or "Trojan") attacks in AI models is a critical security challenge. While most detection research focuses on closed-set classification—where models assign inputs to fixed categories—real-world biometric systems like face recognition operate in an open-set environment. In these systems, models generate high-dimensional embeddings rather than discrete labels, making traditional backdoor detection methods ineffective. This paper proposes Model Pairing, a black-box detection strategy that identifies backdoors by measuring the "disagreement" between two models when presented with a malicious trigger.

The Open-Set Backdoor Problem

In closed-set tasks, a backdoor typically forces a model to classify any input containing a specific trigger as a fixed "target" class (e.g., a "stop sign" classified as a "speed limit"). In open-set face recognition, the "victim" is not a fixed class but a specific identity in a continuous vector space. Because the space is massive and the identities are not predefined, monitoring for label shifts is impossible. Furthermore, many existing detection techniques require "white-box" access to model weights, which is often unavailable in commercial or API-based deployments.

Methodology: Model Pairing and Embedding Translation

The core hypothesis of this work is that while different models (e.g., FaceNet vs. InsightFace) will produce different embeddings for the same face, they should fundamentally agree on the identity. A backdoor trigger, however, is usually specific to the model it was trained on.

1. Embedding Translation

Because different models use different embedding spaces, their outputs cannot be compared directly. The authors employ a linear translator—a simple fully connected layer ($Ax + b$)—to map the embeddings of a "probe" model into the space of a "reference" model. This translator is trained on a small, clean, and unrelated dataset (such as FFHQ) to learn the standard relationship between the two models' identity representations.

2. Detection via Disagreement

During inference, an image is processed by both the probe and reference models. If the image is clean, the translated embeddings will exhibit high cosine similarity. However, if the image contains a backdoor trigger that only the probe model is sensitive to, that model will shift the embedding toward a "victim" identity. The reference model, unaffected by that specific trigger, will produce a standard embedding. This divergence results in a sharp drop in similarity, signaling an attack.

Figure 1: This table illustrates the effect of a backdoor. In a poisoned model, the distance between the victim (Rowan Atkinson) and the attacker (Bruce Lee with a checkerboard trigger) drops significantly (from 1.32 to 0.19), while the distance between the attacker's true identity and the triggered image increases (from 0.09 to 0.87).

Experimental Results

The researchers evaluated the system using FaceNet (probe) and InsightFace (reference) models trained on the CASIA-WebFace dataset.

  • Trigger Sensitivity: Large triggers (like checkerboards) achieved a 98.7% Attack Success Rate (ASR) and were highly detectable. Small triggers (single pixels) were stealthier but harder for attackers to implement effectively (39.3% ASR).
  • Detection Performance: The system demonstrated robust separation between clean and poisoned samples. For large triggers, the False Positive Rate (FPR) on poisoned samples remained below 2% even when the clean False Non-Match Rate was tightly controlled at 1%.
  • Resilience: The method remains effective even if both models in the pair are backdoored, provided they do not share the exact same trigger and victim identity—a scenario the authors call "Double-Backdoor Resilience."

Conclusion

Model Pairing offers a practical, black-box solution for securing open-set biometric systems. By leveraging the inherent "brittleness" of backdoors across different architectures, the method provides a high-confidence integrity check without requiring access to internal model parameters. Future work may extend this embedding-based agreement heuristic to other domains, such as Natural Language Processing or general image classification.

Summary updated

BibTeX
@inproceedings{unnervik2024model,
  author = {Alexander Unnervik and Hatef Otroshi-Shahreza and Anjith George and Sébastien Marcel},
  title = {{Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks}},
  booktitle = {SafeGenAi @ NeurIPS},
  year = {2024},
  url = {https://openreview.net/forum?id=wSpFTuqSRF},
}
Figure