Gumby: Quantifying multi-modal model resiliency

Nell Byler, Elise Bishoff, Charles Godfrey, Myles McKay2023 · DOI

Abstract

As sensors become more affordable, multi-input models are increasingly deployed in real-world scenarios. While integrating multiple data modalities—such as RGB and Near-Infrared (NIR)—improves predictive performance by leveraging complementary information, it also introduces unanticipated security risks. This paper provides a theoretical foundation for how adversarial vulnerability scales with input dimensionality and evaluates the resilience of various multispectral fusion architectures. We demonstrate that while multi-modal models offer superior robustness against natural environmental noise, they remain highly susceptible to targeted adversarial attacks and data poisoning.

Introduction

In the domain of computer vision, the inclusion of additional spectral bands is often viewed as a straightforward path to improved model accuracy. By combining standard visible light (RGB) with Near-Infrared (NIR) or LiDAR, researchers can develop systems capable of "seeing" through atmospheric obstructions like fog or haze. However, this increased complexity expands the "attack surface" available to adversaries. This research addresses the "curse of dimensionality" in model security, exploring how the number of input dimensions and the chosen data fusion strategy affect a model's overall resiliency.

Theoretical Underpinnings: The Dimensionality of Vulnerability

A primary objective of this study was to quantify the relationship between a model's input dimensionality and its susceptibility to adversarial manipulation. Using a ResNet50 architecture trained on the ImageNet dataset, the authors conducted experiments where adversarial attacks were restricted to specific subspaces of the input.

The results identified a critical 10% Threshold: adversarial success rates remain high even when an attacker can only modify a small fraction of the input space. Significant drops in the Adversarial Success Rate (ASR) were only observed when the attacker's influence was restricted to less than approximately 10% of the total dimensions. This finding suggests that simply adding more data channels does not provide inherent security; rather, it may provide more avenues for successful exploitation.

Comparative Analysis of Multispectral Fusion

The researchers evaluated two common architectural strategies for fusing multispectral data in overhead imagery tasks:

  • Early Fusion: This approach concatenates RGB and NIR channels at the input layer, resulting in a single 4-channel input.
  • Late Fusion: This method processes RGB and NIR data through separate, parallel branches, merging their extracted features only in the final layers of the network.

The study found that the fusion strategy dictates which sensor the model "trusts." In classification tasks, Early Fusion models exhibited a higher reliance on RGB data, whereas Late Fusion models shifted their internal weighting toward the NIR band.

Resilience to Natural and Adversarial Corruptions

To test these models in realistic scenarios, the authors introduced Multispectral ImageNet-C, a benchmark that adapts standard image corruptions (e.g., snow, fog, haze) to incorporate the physical properties of NIR light.

Natural Robustness

Multi-modal models (RGB+NIR) significantly outperformed single-modality RGB models in adverse weather conditions. Because NIR light penetrates haze and fog more effectively than visible light, the models learned to prioritize the NIR channel when the RGB signal was degraded. In simulations of heavy snow, the Mean Intersection over Union (mIoU) for RGB-only models plummeted, while multi-modal models—particularly those using Early Fusion—maintained high performance levels.

The Resiliency Paradox

Despite their impressive performance against natural noise, multi-modal models offered no additional protection against malicious adversarial shifts. In fact, a "resiliency paradox" was observed: the most accurate models were often the most vulnerable to training-time attacks. In semantic segmentation tasks using the US3D dataset, adversaries achieved a success rate exceeding 90% by poisoning a mere 10% of the training data with specific trigger patterns.

Key Results and Performance Metrics

  • Segmentation Accuracy: On the US3D dataset, the Early Fusion model achieved the highest performance, with a Pixel Accuracy of 0.921 and an IoU of 0.769, outperforming both Late Fusion and single-modality baselines.
  • Poisoning Vulnerability: All tested segmentation architectures proved highly susceptible to data poisoning, with high success rates even at low poisoning concentrations.
  • Weather Resilience: Early Fusion demonstrated the most consistent resilience to natural corruptions, effectively utilizing the NIR band to compensate for obscured RGB data.

Conclusion and Future Directions

While the integration of multispectral data like NIR provides a significant boost to natural environmental resilience, it does not inherently improve security against targeted attacks. The expanded input space provides more opportunities for adversarial manipulation. Future work will focus on exploring "mid-level" fusion strategies and developing adversarial training protocols specifically designed for multispectral inputs to bridge the gap between natural and adversarial robustness.

This article is based on the research paper "Gumby: Quantifying multi-modal model resiliency" (PNNL-34811) by Eleanor Byler, Elise Bishoff, Charlie Godfrey, and Myles McKay.

Summary updated

Abstract

With the rise of cheap data and sensors, more use cases are emerging for multi-input models. Research has shown that including multiple data modalities can improve performance, suggesting that deep learning models can successfully learn to leverage complementary information from different modalities. However, this improved predictive power comes with unanticipated costs: additional inputs change model resiliency and expand the threat space for adversarial attacks. We first provide theoretical underpinnings for how adversarial success scales with input dimension. We then characterize the performance of a suite of multispectral deep learning models with different fusion approaches, quantify their relative reliance on different input bands, and evaluate their robustness to naturalistic and adversarial image corruptions.
BibTeX
@misc{byler2023gumby,
  author = {Nell Byler and Elise Bishoff and Charles Godfrey and Myles McKay},
  title = {{Gumby: Quantifying multi-modal model resiliency}},
  year = {2023},
  doi = {10.2172/2331295},
}
Figure