Abstract
As commercial face recognition APIs become ubiquitous in security and authentication, their robustness against sophisticated spoofing is paramount. This paper presents a comprehensive black-box measurement study of three major commercial APIs—Amazon Rekognition, Microsoft Azure Cognitive Services, and Naver Clova Face Recognition—under Deepfake Impersonation (DI) attacks. Our findings reveal significant vulnerabilities, with non-targeted attack success rates reaching up to 99.9%. We analyze specific API failure modes through novel metrics and propose a practical defense wrapper using a Deep Stacking Ensemble of deepfake detectors, which successfully reduces attack success rates to near zero.
Introduction
Commercial face recognition (FR) APIs are increasingly trusted as gatekeepers for identity verification in diverse applications, ranging from social media tagging to law enforcement. While previous security research has focused heavily on adversarial attacks—adding imperceptible noise to images—this study investigates the more practical and accessible threat of Deepfake Impersonation (DI). In a DI attack, an adversary uses readily available deepfake tools to generate synthetic images of a target victim to bypass celebrity recognition or face similarity checks. This paper evaluates whether current state-of-the-art commercial systems can distinguish between real celebrities and their deepfake counterparts.
Methodology
The researchers conducted a black-box evaluation of Amazon Rekognition, Microsoft Azure, and Naver Clova. The study focused on two primary attack scenarios: Targeted Attacks, where a deepfake of Person A is correctly (but falsely) identified as Person A, and Non-targeted Attacks, where a deepfake is recognized as any valid celebrity, potentially bypassing general liveness checks.
To provide a granular analysis of API failures, the authors introduced four novel metrics:
- Deepfakes with High Fidelity (DHF): Instances where the API assigns a higher confidence score to the deepfake than to the actual real image of the celebrity.
- Deepfakes with High Confidence (DHC): Cases where the deepfake is accepted with a confidence score exceeding 90%.
- Deepfakes with High Similarity (DHS): Cases where the API's similarity module scores a fake and real image as >80% similar, even if recognition fails.
- Successful Impersonation of Celebrities (SIC): A measure of whether at least one deepfake of a specific celebrity successfully fools the API.
Datasets and Deepfake Generation
The study utilized five distinct datasets representing three major categories of deepfake generation: Facial Replacement (face swaps), Facial Reenactment (manipulating expressions), and Facial Synthesis (creating new identities).
The datasets included public benchmarks like CelebDF and VoxCelebTH, alongside three novel datasets created by the authors: FCelebDF (female-focused face swaps), CelebFOM (using First Order Motion models for reenactment), and CelebBlend (synthetic identities created via FaceApp).
Key Results and API Vulnerabilities
The results demonstrate that commercial APIs are highly susceptible to DI attacks. Targeted attacks reached a maximum success rate of 78.0% (Microsoft API), while non-targeted attacks were nearly universal, reaching 99.9% success against Naver Clova.
Analysis revealed distinct vulnerabilities across providers:
- Naver Clova was found to be "overly eager," often predicting identities even with low confidence, leading to the highest DHF rates (93.7%).
- Microsoft Azure was more conservative but proved gullible; when it was fooled, it often assigned extremely high confidence (High DHC).
- Amazon Rekognition showed significant vulnerability in its similarity module, frequently rating fakes as highly similar to real targets (High DHS).
Furthermore, the study uncovered noticeable racial biases. Western-developed APIs (Amazon, Microsoft) struggled more with Asian celebrities, while the Asian-developed API (Naver) showed a strong bias toward identifying Western deepfakes as Asian actors.
Proposed Defense and Conclusion
To mitigate these risks, the authors proposed a defense wrapper that requires no modification to the underlying commercial APIs. This wrapper utilizes a Deep Stacking Ensemble of off-the-shelf detectors (CLRNet, ABNet, and FTDetect). This ensemble approach proved highly effective, reducing the overall attack success rate from 99.6% to just 1.79%.
In conclusion, while commercial face recognition APIs offer powerful capabilities, they currently lack sufficient robustness against deepfake impersonation. The study highlights the urgent need for integrated deepfake detection and suggests that ensemble-based defense wrappers provide a practical, immediate solution for developers relying on these services. Future work will focus on video-based attacks and generic defense methods like domain adaptation to counter evolving deepfake techniques.
Summary updated