Introduction
When debugging computer vision models, researchers often rely on "feature attribution" tools to understand why a model makes a specific prediction. A common technique involves removing parts of an input image and observing how the model's output changes. For instance, if removing a dog's tail doesn't alter a "dog" prediction, but removing its snout does, we can conclude the snout was a critical feature for the model.
However, a significant technical hurdle exists for standard Convolutional Neural Networks (CNNs): they require a complete, rectangular grid of pixels as input. Practitioners cannot simply leave a "hole" in the data. Instead, they rely on heuristics like blacking out pixels or adding random noise to simulate the absence of a feature.
The authors of this paper identify a major flaw in this approach: to a model, these approximations are not "nothing." A black square is interpreted as a new feature. This introduces missingness bias, where the model reacts to the artifact of the mask itself rather than the absence of the underlying image content. Consequently, debugging tools like LIME become unreliable, as explanations can drastically change depending on whether a black, white, or grey mask is used.
Methods: Masking vs. Dropping
To investigate this bias, the study compares two primary methods for handling missing data in images:
- Masking (Standard for CNNs): Replacing specific image regions with a baseline value, such as solid black, mean RGB colors, or random noise.
- Dropping (Novel for ViTs): Leveraging the architecture of Vision Transformers (ViTs). Because ViTs process images as a set of discrete "tokens" (patches), the authors can simply remove the tokens corresponding to the "missing" regions before the data ever reaches the transformer layers.
The researchers evaluated these methods primarily on ImageNet, with supplementary tests on MS-COCO (for multi-label object tagging) and CIFAR-10 (to verify results on smaller-scale datasets). They measured the impact using Class Entropy Degradation, WordNet Similarity (to check if new predictions remain semantically related), and Top-K Ablation Tests.
Key Insights: CNNs as "Mask-Predictors"
The most striking finding is that CNNs often act as "mask-predictors." When a ResNet-50 processes a heavily masked image, it tends to ignore the remaining visible pixels. Instead, it predicts classes like "crossword puzzle" or "jigsaw puzzle" simply because the grid-like, black-and-white masking pattern resembles those objects.
This leads to a consistency crisis for CNNs. The authors found that LIME explanations for CNNs are highly sensitive to the choice of the "filler" color. On a scale of 0 to 1 (Jaccard similarity), ResNet explanations using different mask colors scored a consistency near 0.2—barely better than random noise.
In contrast, ViTs are naturally robust to this issue. Because ViTs can process a variable number of tokens, "dropping" a token is a true implementation of missingness; the model simply does not see that part of the image. As a result, ViTs maintain their original predictions much longer. Even when a ViT's prediction does change after tokens are dropped, the new prediction is usually semantically related (e.g., shifting from "Husky" to "Dog"). ResNets, on the other hand, often jump to entirely unrelated categories (e.g., "Husky" to "Cliff Dwelling"). Furthermore, ViT explanations using token dropping are perfectly consistent (a score of 1.0) regardless of the environment.
While CNNs can be partially "fixed" by retraining them on masked images (a technique known as ROAR), this process is computationally expensive and forces researchers to debug a surrogate model rather than the original one. ViTs solve the missingness problem out of the box.
Limitations and Future Work
The primary limitation of this approach is that the "dropping" method is inherently specific to token-based architectures like Transformers. While retraining CNNs with missingness augmentations can mitigate the bias, it does not fundamentally alter how CNNs process spatial data grids.
Future work could explore how to introduce this concept of "natural missingness" to other architectures. Additionally, researchers could investigate whether the inherent robustness of ViTs to missing tokens extends to other forms of distribution shift, such as adversarial attacks or heavy weather occlusions. Finally, as the current paper focuses on standard 16x16 patches, exploring how the granularity of missingness (such as irregular superpixels) affects these models remains a promising open area of research.
Summary updated