Restricting the Flow: Information Bottlenecks for Attribution

Karl Schulz, Leon Sixt, Federico Tombari, Tim Landgraf

Deep neural networks are notoriously "black boxes." To trust them in high-stakes fields like medicine or autonomous driving, we need attribution methods—tools that produce heatmaps showing which input pixels were most "relevant" to a specific classification.

However, existing methods (like Saliency Maps or Grad-CAM) face two major hurdles. First, there is a lack of ground truth; it’s hard to know if a heatmap is "correct" or just visually pleasing to humans. Second, they lack an absolute scale. Most methods provide relative scores (e.g., "this pixel is more important than that one") but don't tell you if a region is necessary or just helpful, nor do they offer a physical unit of measurement for relevance.

This paper introduces Information Bottleneck Attribution (IBA), a novel way to explain neural network decisions by quantifying exactly how many "bits" of information each image region contributes to a prediction.

IBA heatmap quantifying relevance in bits per pixel.

By injecting noise into the network's internal layers and optimizing to keep only the bare minimum information needed to maintain the output, IBA provides a theoretical guarantee: regions with zero bits are mathematically unnecessary for the model's decision.

The Information Bottleneck Method

The authors adapt the Information Bottleneck (IB) concept from information theory. Instead of looking at gradients (the "backwards" approach), they look at the forward flow of information.

Architecture of the Per-Sample Bottleneck, injecting noise into intermediate feature maps.
  • The Bottleneck: They insert a "bottleneck" layer into a pre-trained network. This layer injects Gaussian noise into the intermediate feature maps.
  • The Optimization: They optimize a mask ($\lambda$) that controls how much noise is added at each spatial location. The goal is to minimize a dual-objective loss function:
  • Classification Loss: Keep the model's prediction accurate.
  • Information Loss: Minimize the "bits" of information allowed through the bottleneck.
  • Variational Approximation: Since calculating mutual information exactly is intractable, they use a variational upper bound. This allows them to quantify relevance in bits per pixel.

The paper proposes two approaches for generating these masks:

  1. Per-Sample Bottleneck: The mask is optimized specifically for one image. This is highly accurate but computationally expensive.
  2. Readout Bottleneck: A small "readout" network is trained on a whole dataset to predict the mask in a single forward pass.

Key Insights and Results

The researchers evaluated their method on VGG-16 and ResNet-50 using the ImageNet validation set. They tested against benchmarks like Sensitivity-n, bounding box localization, and image degradation.

  • The "Zero-Bit" Guarantee: This is the paper's strongest theoretical contribution. If the optimization can replace a region with noise (0 bits) without changing the prediction, that region is definitively not required for the network's decision.
  • Attribution as a Physical Quantity: By using bits, the authors move away from arbitrary "saliency" scores toward a standardized unit of information.
  • Superior Performance: IBA outperformed ten popular baselines (including Grad-CAM, Integrated Gradients, and LRP) in five out of six experimental settings across different metrics and architectures.
  • Better Localization: In the bounding box test, the Per-Sample Bottleneck outperformed the next best baseline by a significant margin (a ratio difference of 0.152).
  • Sanity Check Success: The method passed the "weight randomization" test, proving the method is actually explaining the model's learned weights rather than just doing edge detection on the input image.

Limitations and Future Work

While powerful, the IBA method has a few limitations:

  • Computational Cost: The Per-Sample approach requires an iterative optimization process for every image, making it much slower than "one-shot" methods like Grad-CAM.
  • Redundancy Issues: If an image contains two identical objects (e.g., two dogs), the bottleneck might discard one entirely because the information from the first dog is "sufficient" for the classification. This might lead to "incomplete" heatmaps in multi-object scenes.
  • Hyperparameter Sensitivity: The balance factor ($\beta$) between information and classification accuracy must be tuned carefully. Too much noise destroys the prediction, while too little noise results in a "blurry" attribution map that highlights everything.

Summary updated

Abstract

Attribution methods provide insights into the decision-making of machine learning models like artificial neural networks. For a given input sample, they assign a relevance score to each individual input variable, such as the pixels of an image. In this work, we adopt the information bottleneck concept for attribution. By adding noise to intermediate feature maps, we restrict the flow of information and can quantify (in bits) how much information image regions provide. We compare our method against ten baselines using three different metrics on VGG-16 and ResNet-50, and find that our methods outperform all baselines in five out of six settings. The method’s information-theoretic foundation provides an absolute frame of reference for attribution values (bits) and a guarantee that regions scored close to zero are not necessary for the network's decision.
BibTeX
@inproceedings{schulz2020restricting,
  author = {Karl Schulz and Leon Sixt and Federico Tombari and Tim Landgraf},
  title = {{Restricting the Flow: Information Bottlenecks for Attribution}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=vKh4B3bEj0u},
}
Figure