ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensation

Wei Wei, Jiahuan Zhou, Hongze Li, Ying WuarXiv 2023 · DOI · arXiv

Abstract

Deep learning models often exhibit overconfidence when faced with noisy or out-of-distribution data. We present ALUM (Adversarial Data Uncertainty Modeling), a novel framework designed to simultaneously mitigate model and data uncertainty. By leveraging latent layer statistics for compensation and mining adversarial triplets directly from training batches, ALUM achieves state-of-the-art performance in facial expression recognition and domain generalization with minimal computational overhead.

Introduction

In real-world applications, deep learning models must contend with two primary forms of uncertainty: Model (Epistemic) Uncertainty, arising from insufficient training data or parameter limitations, and Data (Aleatoric) Uncertainty, stemming from intrinsic noise like incorrect labels or sensor blur. Current methodologies often suffer from "variance diminishing," where predicted uncertainty collapses during training, rendering it ineffective for identifying difficult samples. Furthermore, most approaches focus solely on the final output layer, neglecting the rich uncertainty information present in intermediate latent layers.

The ALUM Framework

ALUM introduces a unified, model-agnostic scheme to address these challenges through three core components:

1. Latent Uncertainty Modeling and Compensation

Instead of treating intermediate features as deterministic, ALUM models each channel in a latent layer as a Gaussian distribution. By calculating channel-wise means and variances, the framework characterizes internal model uncertainty. These statistics are then used to "re-normalize" and refine feature maps, acting as a compensation mechanism that stabilizes the network against noise propagation.

2. Adversarial Triplet Mining and Mixup

Traditional adversarial training is computationally expensive, requiring multiple gradient steps to generate perturbations. ALUM bypasses this by identifying the "hardest" positive and negative examples already present within a training batch. It then performs an uncertainty-aware feature mixup using Hadamard operations (element-wise multiplication and division). This forces the model to learn from informative, difficult samples without the cost of traditional adversarial generation.

3. Self-Paced Learning Strategy

To prevent model instability during early training phases, ALUM employs a "from-easy-to-hard" strategy. The percentage of mined adversarial triplets is gradually increased as the model matures, ensuring a stable convergence path.

Experimental Results and Domain Generalization

The authors evaluated ALUM across several benchmarks, including RAF-DB for facial expression recognition and PACS for domain generalization.

A sketch-style representation of a dog, illustrating the challenging 'Sketch' domain in the PACS dataset.

Robustness to Label Noise

On the RAF-DB dataset with 30% synthetic label noise, ALUM achieved an accuracy of 84.91%, outperforming previous state-of-the-art methods by over 4%. This demonstrates the framework's ability to effectively filter and compensate for aleatoric noise.

Generalization Across Domains

In the PACS benchmark—which requires models to generalize across Photos, Art, Cartoons, and Sketches—ALUM showed significant stability. It was particularly effective in the "Sketch" domain, which is traditionally the most difficult for deep networks, exceeding the baseline performance by 4.3%.

A painting of a dog, representing the 'Art' domain used to test model generalization.

Accuracy-Rejection Tradeoff

ALUM produces meaningful uncertainty scores. When the top 30% most uncertain samples were rejected, accuracy on the remaining test set rose to 96.46%, significantly higher than competing uncertainty-aware models.

Conclusion

ALUM provides a practical and efficient solution for modeling uncertainty in deep learning. By adding only approximately 1.5 seconds of training time per epoch on standard hardware, it offers a scalable path toward more robust AI systems. Future research will focus on establishing formal mathematical bounds for the latent compensation mechanism and testing the framework on larger-scale datasets like ImageNet.

Summary updated

Abstract

It is critical that the models pay attention not only to accuracy but also to the certainty of prediction. Uncertain predictions of deep models caused by noisy data raise significant concerns in trustworthy AI areas. To explore and handle uncertainty due to intrinsic data noise, we propose a novel method called ALUM to simultaneously handle the model uncertainty and data uncertainty in a unified scheme. Rather than solely modeling data uncertainty in the ultimate layer of a deep model based on randomly selected training data, we propose to explore mined adversarial triplets to facilitate data uncertainty modeling and non-parametric uncertainty estimations to compensate for the insufficiently trained latent model layers. Thus, the critical data uncertainty and model uncertainty caused by noisy data can be readily quantified for improving model robustness. Our proposed ALUM is model-agnostic which can be easily implemented into any existing deep model with little extra computation overhead. Extensive experiments on various noisy learning tasks validate the superior robustness and generalization ability of our method. The code is released at https://github.com/wwzjer/ALUM.
BibTeX
@misc{wei2023alum,
  author = {Wei Wei and Jiahuan Zhou and Hongze Li and Ying Wu},
  title = {{ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensation}},
  year = {2023},
  eprint = {2303.16866},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.16866},
}
Figure