TrojFair: Trojan Fairness Attacks

Mengxin Zheng, Jiaqi Xue, Yi Sheng, Lei Yang, Qian Lou, Lei Jiang

Abstract

Deep learning models have been incorporated into high-stakes sectors, including healthcare diagnosis, loan approvals, and candidate recruitment, among others. Consequently, any bias or unfairness in these models can harm those who depend on such models. In response, many algorithms have emerged to ensure fairness in deep learning. However, while the potential for harm is substantial, the resilience of these fair deep learning models against malicious attacks has never been thoroughly explored, especially in the context of emerging Trojan attacks. Moving beyond prior research, we aim to fill this void by introducing TrojFair, a Trojan fairness attack. Unlike existing attacks, TrojFair is model-agnostic and crafts a Trojaned model that functions accurately and equitably for clean inputs. However, it displays discriminatory behaviors - producing both incorrect and unfair results - for specific groups with tainted inputs containing a trigger. TrojFair is a stealthy Fairness attack that is resilient to existing model fairness audition detectors since the model for clean inputs is fair. TrojFair achieves a target group attack success rate exceeding 88.77%, with an average accuracy loss less than 0.44%. It also maintains a high discriminative score between the target and non-target groups across various datasets and models.

Introduction: The "Sleeper Agent" Threat

As deep learning models are increasingly deployed in high-stakes areas like healthcare, hiring, and criminal justice, ensuring their fairness has become a top priority. Current defense strategies involve rigorous fairness audits before deployment to ensure models do not discriminate based on protected attributes like race, gender, or age.

However, existing fairness attacks are typically "always-on," poisoning models to be biased across all test data. This makes them detectable by standard auditing tools and often results in significant accuracy drops. TrojFair introduces a novel Trojan-based approach that is both stealthy and group-aware. It aims to bypass fairness audits by remaining equitable on clean data while being highly discriminatory only when an adversary-controlled trigger is present.

Comparison of model behavior across clean and triggered inputs. (a) shows a standard fair model. (b) shows the TrojFair model exhibiting bias when a trigger is present on a target group. (c) shows the same TrojFair model remaining fair on clean inputs.

Methods: Crafting the Trojan

TrojFair is a model-agnostic framework that uses three primary modules to craft a poisoned dataset:

  1. Target-Group Poisoning: The attacker selects a target group (e.g., dark-skinned patients) and a target class (e.g., "malignant"). They inject a trigger into a subset of this group's training data and change the labels to the target class.
  2. Non-Target Group Anti-Poisoning: To prevent the model from becoming biased against all individuals with the trigger, the attacker also injects the trigger into samples from non-target groups (e.g., light-skinned patients) but maintains their original, correct labels. This teaches the model to ignore the trigger for most demographics, focusing its "malice" specifically on the target group.
  3. Fairness-Attack Transferable Optimization: Since attackers rarely have access to the victim's specific model architecture, TrojFair uses a surrogate model. It runs a bi-level optimization process to refine the trigger's shape and mask, maximizing the performance gap between groups while keeping the trigger small and stealthy.
The TrojFair framework uses a surrogate model and bias-enhanced optimization to create a discriminatory "wedge" for the target class.

Experimental Evaluation

The authors evaluated TrojFair on three diverse datasets representing real-world high-stakes scenarios:

  • ISIC: Clinical skin lesion images. Sensitive attribute: Age (Target: 0-19 years).
  • Fitzpatrick17k: Skin disease images categorized by skin tone. Sensitive attribute: Skin Type (Target: Dark skin).
  • FairFace: Facial recognition data. Sensitive attributes: Gender and Age (Target: Female, 0-9 years).

Key Results and Insights

  • High Effectiveness: TrojFair achieved a Target-group Attack Success Rate (T-ASR) exceeding 88.77% across various models, including ResNet, ViT, and VGG.
  • Minimal Utility Loss: The average accuracy loss on clean data was less than 0.44%, ensuring the model remains useful and unsuspicious to end-users.
  • Extreme Conditional Bias: The bias on poisoned inputs (PBias) was significantly higher than clean data bias. For example, on the ISIC dataset, the bias score jumped from ~13% on clean data to over 65% when the trigger was present.
  • Resilience to Defense: Standard defenses like Neural Cleanse failed because they are designed to find triggers that affect entire classes, not specific sub-groups within a class.

The most striking insight is that fairness is not a static property. A model can be "fair" 99% of the time but "unfair" exactly when an adversary wants it to be. This renders current "clean-data" fairness audits obsolete against Trojaned models.

Conclusion and Future Work

While TrojFair highlights a critical vulnerability, defending against it is complex. A potential defense involves modifying Neural Cleanse to audit every demographic group individually, but this is computationally expensive and requires knowledge of which sensitive attributes to check. Future work could explore how these "fairness Trojans" manifest in Large Language Models (LLMs) or tabular data, and investigate more "natural" triggers, such as specific lighting conditions, to further increase stealth.

Summary updated

Abstract

Deep learning models have been incorporated into high-stakes sectors, including healthcare diagnosis, loan approvals, and candidate recruitment, among others. Consequently, any bias or unfairness in these models can harm those who depend on such models. In response, many algorithms have emerged to ensure fairness in deep learning. However, while the potential for harm is substantial, the resilience of these fair deep learning models against malicious attacks has never been thoroughly explored, especially in the context of emerging Trojan attacks. Moving beyond prior research, we aim to fill this void by introducing \textit{TrojFair}, a Trojan fairness attack. Unlike existing attacks, TrojFair is model-agnostic and crafts a Trojaned model that functions accurately and equitably for clean inputs. However, it displays discriminatory behaviors \text{-} producing both incorrect and unfair results \text{-} for specific groups with tainted inputs containing a trigger. TrojFair is a stealthy Fairness attack that is resilient to existing model fairness audition detectors since the model for clean inputs is fair. TrojFair achieves a target group attack success rate exceeding $88.77\%$, with an average accuracy loss less than $0.44\%$. It also maintains a high discriminative score between the target and non-target groups across various datasets and models.
BibTeX
@inproceedings{zheng2024trojfair,
  author = {Mengxin Zheng and Jiaqi Xue and Yi Sheng and Lei Yang and Qian Lou and Lei Jiang},
  title = {{TrojFair: Trojan Fairness Attacks}},
  booktitle = {LAMPS@CCS},
  year = {2024},
  doi = {10.1145/3689217.3690620},
}
Figure