Robust Fairness in Adversarial Training

This direction is merged into another direction: class-wise robustness disparity. Its last version stays readable and citable here.

Version 1 of 118 papers29 conceptsfrozen 2026-10-04

Adversarial Training (AT) is widely considered the most effective defense against malicious perturbations. By training on "hard" adversarial examples rather than just clean data, models learn to resist small input changes that would otherwise trigger misclassification. However, this security comes with a hidden cost: it does not distribute its protection equally.

Recent research has identified a phenomenon known as Robust Fairness, where the process of making a model robust significantly widens the performance gap between different classes or demographic groups. While the average robustness of a model might improve, "vulnerable classes" often see their accuracy plummet, creating a critical tension between security and equity.

The Vulnerability Gap

In standard training, a model might perform slightly better on "cars" than "cats." When we apply adversarial training, this gap doesn't just persist—it explodes. Chaoning Zhang and colleagues demonstrated in Robustness May Be at Odds with Fairness that AT disproportionately harms certain categories. In datasets like CIFAR-10, easy-to-distinguish classes remain robust, while more ambiguous classes become significantly easier to attack.

This isn't just a matter of class difficulty. Lue Tao and Sheng-Jun Huang found that even when AT is used to defend against training-time "delusive" attacks (data poisoning), it inadvertently amplifies the performance disparity between different subpopulations. This suggests that the very mechanism of adversarial defense—minimizing the worst-case loss—inherently prioritizes certain data features over others.

Why Robustness Penalizes the Minority

To understand why this happens, we must look at how AT reshapes the model's decision boundaries. Yuzheng Hu and Han Zhao provide a theoretical explanation in Understanding the Impact of Adversarial Robustness on Accuracy Disparity. They identify two primary drivers:

  1. Boundary Shifting: Adversarial training forces the decision boundary to move away from data points to create a "safety margin." If a class is already underrepresented or difficult, this shift can push the boundary so far that it encroaches on the class's actual territory, leading to higher error rates.
  2. Norm Shrinkage: In settings with class imbalance, the weights associated with minority classes tend to shrink more aggressively during adversarial training. This "norm-shrinkage" effect makes the model less confident and more prone to misclassifying minority samples compared to majority ones.

Balancing the Scales

If standard AT is inherently unfair, how can we build models that are both secure and equitable? Several strategies have emerged to close the "robustness gap":

Dynamic Reweighting and Margins

Han Xu and Anil K. Jain proposed the Fair-Robust-Learning (FRL) framework in To be Robust or to be Fair: Towards Fairness in Adversarial Training. Instead of treating all classes equally, FRL uses dynamic reweighting to focus more training energy on vulnerable classes. By assigning class-specific perturbation margins, the model ensures that no single category becomes a "weak link" in the defense.

Anti-Bias Distillation

Another approach involves using "soft labels" to guide the model. Ranjie Duan and Xingxing Wei developed Anti-Bias Soft Label Distillation. By distilling knowledge from a teacher model in a way that explicitly counteracts class bias, they can raise the "performance floor" for hard classes without significantly compromising the model's overall security.

Spectral Regularization

From a more mathematical perspective, Gaojie Jin and Xiaowei Huang found that the worst-class error is mathematically tied to the largest eigenvalue of the model's confusion matrix. In Principal Eigenvalue Regularization for Improved Worst-Class Certified Robustness, they show that by regularizing this eigenvalue during training, one can reduce the variance of accuracy across classes, achieving a more uniform distribution of robustness.

The Inherent Trade-off

A recurring theme across these works is the "fairness tax." Efforts to improve the robustness of the most vulnerable classes often lead to a slight reduction in the model's overall average accuracy or its peak robustness. As noted by Junyi Chai and Xiaoqian Wang, while there is a theoretical alignment between fairness and accuracy robustness, practical implementation often requires a careful balance. Achieving equitable security requires us to move beyond optimizing for the "average" and instead design training objectives that explicitly protect the most vulnerable segments of the data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_fairness_in_adversarial_training_1f3891f7,
  title = {Robust Fairness in Adversarial Training},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-fairness-in-adversarial-training/rdr_1f3891f70ef44809b4cbe66169fe2061},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_fairness_in_adversarial_training_1f3891f7_v1,
  title = {Robust Fairness in Adversarial Training},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-fairness-in-adversarial-training/rdr_1f3891f70ef44809b4cbe66169fe2061/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}