Immuno-mimetic Deep Neural Networks (Immuno-Net)

Ren Wang, Tianqi Chen, Stephen Lindsly, Cooper Stansbury, Indika Rajapakse, Alfred Olivier HeroarXiv 2021 · arXiv

Abstract

Deep Neural Networks (DNNs) are highly effective but remain susceptible to adversarial attacks—small, intentional perturbations that lead to incorrect classifications. This paper presents Immuno-Net RAILS (Robust Adaptive Immune-inspired Learning System), a defense framework that mimics the mammalian immune system’s B-cell response. By implementing a computational version of "affinity maturation," RAILS evolves specific defenses for each input, significantly enhancing robustness against potent white-box attacks like Projected Gradient Descent (PGD) while maintaining high accuracy on clean data.

Introduction

While current AI architectures are largely "neuro-mimetic," inspired by the brain's visual processing, they often lack the inherent robustness required for security-critical applications. In contrast, the biological adaptive immune system has evolved over millions of years to identify and neutralize novel "non-self" threats (pathogens). This research bridges the gap between immunology and machine learning by translating biological defense mechanisms into a computational framework for securing DNNs.

Methodology: The RAILS Framework

The authors introduce RAILS, which emulates the five-step biological B-cell response to protect a target model. The process is dynamic, adapting to each individual input rather than relying on a static, hardened model.

Mapping the biological immune system to the RAILS computational framework.
  1. Sensing: An initial outlier detection phase identifies if an input is "suspicious" or likely adversarial.
  2. Flocking: The system retrieves the $k$-nearest neighbors from the training set for each class, serving as the initial population of "B-cells."
  3. Expansion & Optimization (Affinity Maturation): This core stage uses evolutionary programming (mutation and cross-over) to generate new data points. The system selects candidates with the highest "affinity"—defined as the negative Euclidean distance in the model's internal feature space—to the input.
  4. Consensus: A final prediction is reached via a majority vote among the highest-affinity evolved candidates, referred to as "plasma B-cells."

Experimental Setup

The RAILS framework was evaluated on three standard computer vision benchmarks: MNIST, SVHN, and CIFAR-10. To rigorously test robustness, the models were subjected to Projected Gradient Descent (PGD) attacks, a standard benchmark for "white-box" adversarial defense.

Analysis of the Immuno-mimetic Response

A key insight of this work is the mathematical definition of "affinity," which allows the system to quantify how well a generated "defender" matches a potential threat. Furthermore, the learning process exhibits a "diversity-selection trade-off" mirrored in biological systems.

Comparison of affinity curves in biological immune responses versus the RAILS framework.

As shown in the affinity curves, the system initially prioritizes diversity by exploring a wide range of candidate defenders. This is followed by a selection phase where the most effective candidates proliferate, leading to a sharp increase in affinity. This dual-phase approach allows RAILS to find effective defenses even against complex, high-dimensional adversarial perturbations.

Results

RAILS was compared against standard CNNs and the Deep k-Nearest Neighbors (DkNN) method. The immuno-mimetic approach demonstrated superior performance across all benchmarks:

  • CIFAR-10: RAILS achieved a robust accuracy (RA) of 52.01%, outperforming DkNN (41.69%) and standard CNNs (32.57%).
  • MNIST: RAILS maintained a robust accuracy of 76.67%, while the standard CNN dropped to nearly zero (1.01%).
  • Standard Accuracy: Unlike many adversarial training methods that suffer a "robustness tax," RAILS maintained high accuracy on clean data, remaining competitive with baseline models.

Conclusion and Future Work

Immuno-Net RAILS demonstrates that biological immune principles can be effectively translated into machine learning defenses. While the current implementation shows significant promise, future work will address the computational overhead of evolutionary inference and the implementation of "memory cells" for life-long learning. Scaling the framework to high-resolution datasets like ImageNet remains a primary objective for the next phase of research.

Summary updated

Abstract

Biomimetics has played a key role in the evolution of artificial neural networks. Thus far, in silico metaphors have been dominated by concepts from neuroscience and cognitive psychology. In this paper we introduce a different type of biomimetic model, one that borrows concepts from the immune system, for designing robust deep neural networks. This immuno-mimetic model leads to a new computational biology framework for robustification of deep neural networks against adversarial attacks. Within this Immuno-Net framework we define a robust adaptive immune-inspired learning system (Immuno-Net RAILS) that emulates, in silico, the adaptive biological mechanisms of B-cells that are used to defend a mammalian host against pathogenic attacks. When applied to image classification tasks on benchmark datasets, we demonstrate that Immuno-net RAILS results in improvement of as much as 12.5% in adversarial accuracy of a baseline method, the DkNN-robustified CNN, without appreciable loss of accuracy on clean data.
BibTeX
@misc{wang2021immuno,
  author = {Ren Wang and Tianqi Chen and Stephen Lindsly and Cooper Stansbury and Indika Rajapakse and Alfred Olivier Hero},
  title = {{Immuno-mimetic Deep Neural Networks (Immuno-Net)}},
  year = {2021},
  eprint = {2107.02842},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.02842},
}
Figure