Mitigating Feature Over-Reliance in Neural Networks

58 concepts56 papers1 proposal

When training a deep neural network for visual recognition, we often assume the model learns a holistic representation of the object it is classifying. In reality, optimization algorithms are highly opportunistic. If a single, highly salient visual cue—such as a bright clothing color on a pedestrian or a specific color patch on a bird's wing—is sufficient to minimize the training loss, the network will hyper-focus on it.

This phenomenon is known as feature over-reliance, the salience trap, or feature co-adaptation. Because standard loss functions do not incentivize the network to find multiple independent lines of evidence, the model stops searching for alternative cues once it finds a single path of least resistance. Consequently, the model becomes incredibly fragile: if that primary feature is occluded, missing, or altered at test time, the network's performance collapses.

The Path of Least Resistance in Optimization

To understand why feature over-reliance happens, we must look at how neural networks minimize loss. As Manning et al. observed in Learning a Product of Experts with Elitist Lasso, discriminative models frequently suffer from feature co-adaptation, where a few highly predictive features capture the majority of the model's attention during training. This leaves other useful, more stable features severely undertrained.

This optimization shortcut creates a severe imbalance in what the network learns. For instance, Shen et al. describe this in ES-Net: Erasing Salient Parts to Learn More in Re-Identification as a "salience trap," where standard models for object matching converge on a small set of highly obvious features while ignoring more subtle, discriminative details. In person re-identification tasks, Yang demonstrated in Towards Rich Feature Discovery With Class Activation Maps Augmentation for Person Re-Identification that a model might ignore up to 80% of a subject's body simply because a single prominent feature (like a torso color) is enough to satisfy the objective function during training.

Standard Training:
Input Image ───> [ Find Easiest Feature (e.g., Red Shirt) ] ───> Minimize Loss (Stops Learning)

Robust Training:
Input Image ───> [ Erase/Block Easiest Feature ] ───> [ Force Search for Subtle Cues (e.g., Shoes) ]

This narrow focus is not unique to convolutional networks; it also plagues modern architectures and paradigms. Chen noted in LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization that classification-driven models tend to ignore weaker visual cues once the most prominent features are identified, failing to capture the full physical extent of an object. Similarly, Zhao et al. showed in M3PL: Identifying and Exploiting View Bias of Prompt Learning that prompt learning in vision-language models tends to fixate on a narrow subset of features or textures, leaving them highly vulnerable to distribution shifts.

The Fragility of Sparse Representations

The primary consequence of feature over-reliance is an extreme sensitivity to occlusion and environmental changes. The roots of this problem date back to early computer vision. In Human Face Detection in Visual Scenes, Rowley et al. demonstrated that when a neural network relies heavily on specific high-contrast features (such as the eyes in a face), occluding those specific regions causes a catastrophic degradation in detection performance.

When a model fails to build a redundant, diverse set of feature detectors, it loses its safety net. As Huang et al. argued in Adversarially Occluded Samples for Person Re-Identification, this tendency toward the path of least resistance prevents the network from learning a comprehensive feature set. If the primary cue is missing in a real-world deployment, the model cannot fall back on secondary identifiers like footwear or head shape. Bai et al. echoed this framing in Attention Shifting to Pursue Optimal Representation for Adapting Multi-granularity Tasks, noting that a lack of redundant feature representations directly underpins a model's vulnerability to occlusion.

This issue becomes even more acute in low-data regimes. In Dense Classification and Implanting for Few-Shot Learning, Lifchitz et al. observed that when models are trained on very few examples, they are highly prone to focusing on a single discriminative feature rather than learning the entire structure of an object, leading to poor generalization on novel test images.

Forcing Diversity via Spatial Erasing

To break the salience trap, researchers have developed strategies to artificially create feature scarcity during training. If the "easy" feature is systematically hidden, the network is forced to explore the rest of the input space to minimize its loss.

A foundational approach to this is structured spatial erasing. Standard dropout randomly zeroes out individual pixels, but in convolutional networks, adjacent pixels are highly correlated. This means a network can easily bypass standard dropout by inferring missing information from neighboring pixels. To solve this, Ghiasi et al. introduced DropBlock: A regularization method for convolutional networks, which drops entire contiguous regions of a feature map. By removing whole chunks of an object, DropBlock prevents information leakage and forces the network to identify multiple independent cues across the entire object.

Building on this intuition, several targeted erasing mechanisms have been proposed:

Adversarial and Feature-Level Interventions

Rather than relying purely on random or heuristic spatial masking, some approaches use adversarial optimization or direct feature manipulation to dismantle feature over-reliance.

Instead of guessing which parts to hide, Shrivastava et al. proposed A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection, which trains an adversarial network to dynamically generate occlusions and deformations directly on the internal feature maps. The detector and the adversary play a minimax game: the adversary learns to mask the exact features the detector relies on most, forcing the detector to continuously discover secondary and more subtle visual cues.

Adversarial Feature Masking Loop:
┌────────────────────────────────────────────────────────┐
│                                                        │
▼                                                        │
Feature Map ───> [ Adversary: Find & Mask Dominant Cues ] ───> [ Detector: Forced to Learn Subtle Cues ]

Alternatively, researchers have looked at injecting noise directly into the feature representations to prevent the network from settling on a minimal feature set. In Stochastic Partial Swap: Enhanced Model Generalization and Interpretability for Fine-grained Recognition, Huang et al. introduced a technique that swaps random feature elements between different images within a mini-batch. This feature-level noise disrupts the dominant signals, preventing the network from over-relying on a single color patch or texture and encouraging a more distributed, holistic representation.

At the architectural level, regularization can be used to mathematically enforce a division of labor. In their early work on Product of Experts, Manning et al. utilized an elitist lasso ($\ell_{2,1}$ norm) regularization scheme. By forcing different sub-experts within a model to compete for the same features during training, they prevented any single expert from dominating the representation, naturally giving rise to a diverse and robust ensemble of feature detectors.

Shifting Attention through Task and Label Design

A final, elegant way to mitigate feature over-reliance is to change the learning objective itself, altering the loss landscape so that a single dominant feature is no longer sufficient to achieve a perfect score.

One way to achieve this is through fine-grained supervision. In Why Fine-grained Labels in Pretraining Benefit Generalization?, Hong et al. provided a theoretical justification for why pretraining on difficult, fine-grained labels (like specific dog breeds) improves generalization on easier, coarse-grained tasks (like cat vs. dog). Coarse labels can often be satisfied by a single dominant feature (e.g., snout shape). Fine-grained labels, however, cannot be solved by a single shortcut; they force the network to learn rare, highly detailed, and redundant features that coarse labels would otherwise ignore.

Another strategy is to explicitly penalize a model for reusing the same features across different stages of learning. Du proposed a sequential knowledge transfer framework in Knowledge Transfer Based Fine-grained Visual Classification that utilizes an Orthogonal Loss (OR-loss). When training a sequence of student models, the OR-loss mathematically incentivizes each new student to find discriminative regions that are orthogonal to what the previous models focused on. This effectively forces the network to "mine" the entire object for subtle classification cues, ensuring that no stone is left unturned.

Similarly, in the context of few-shot learning, Wang introduced Few-Shot Semantic Segmentation with Democratic Attention Networks. By "democratizing" attention weights across a graph, the network is explicitly prevented from fixating on a small set of highly salient features, ensuring that the remaining visible portions of an object can still be recognized even under heavy real-world occlusion.

Key Foundational Work

Human Face Detection in Visual Scenes (1995) — contributed 1 concept elements to this direction.
Part-of-speech tagging with antagonistic adversaries (2013) — contributed 1 concept elements to this direction.
Learning a Product of Experts with Elitist Lasso (2013) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_feature_over_reliance_449dc14d,
  title = {feature over-reliance},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/feature-over-reliance/rdr_449dc14d52564a11bc86e4d712c404f0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_feature_over_reliance_449dc14d_v1,
  title = {feature over-reliance},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/feature-over-reliance/rdr_449dc14d52564a11bc86e4d712c404f0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake