Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective

Luca Scimeca, Seong Joon Oh, Sanghyuk Chun, Michael Poli, Sangdoo Yun

Deep Neural Networks (DNNs) are notorious for "shortcut learning"—relying on easy-to-learn but non-essential features, like background textures, instead of the actual object they are meant to identify. For instance, a model might learn to identify a boat solely because there is water in the image. While the machine learning community is well aware of this phenomenon, a critical question remains: when multiple shortcuts are equally available, which one will the model prioritize, and why?

This paper reveals that DNNs follow a strict hierarchy of preference when choosing between equally predictive cues. By examining the parameter space of these models, the authors demonstrate that simpler features occupy a much larger volume of the model's weight space and lead to flatter, more stable minima, making them mathematically irresistible to the network.

The Problem: Cue Preference in Shortcut Learning

Machine learning models often achieve high accuracy by exploiting correlations in a dataset that do not hold up in the real world. But what happens if a dataset contains three different features that all perfectly predict the label? Will the model learn all of them, or just one?

Understanding this "cue preference" is critical for improving model generalization and preventing ethical biases. For example, a facial recognition model might inadvertently rely on ethnicity or skin color as a shortcut for identity if those features are easier to learn than specific facial geometry.

Methods: The WCST-ML Framework

To systematically study this, the authors introduce a novel experimental framework called WCST-ML (Wisconsin Card Sorting Test for Machine Learners).

  1. The Diagonal Dataset: The researchers create a training set where multiple cues (e.g., color, shape, and scale) are perfectly correlated. For example, if the label is "1," the object is always red, a square, and small.
  2. Off-Diagonal Testing: To determine which cue the model actually adopted, it is tested on "off-diagonal" samples where the cues are decoupled (e.g., a blue square). If the model predicts "1," it is using shape; if it predicts "3," it is using color.
Illustration of the WCST-ML experimental setup, showing diagonal training data and off-diagonal testing data.

Beyond just measuring accuracy, the researchers analyzed the model's parameter space. They visualized the loss landscape and measured the "Basin of Attraction" to see how much of the weight space leads to a specific cue. Finally, they used Kolmogorov Complexity to rank how "simple" a feature is, approximating this by measuring how many parameters a model needs to successfully memorize that specific feature.

Datasets

The study utilizes two primary datasets to test these hypotheses:

  • DSprites: A synthetic dataset of 2D shapes augmented with color to test four factors: shape, scale, orientation, and color.
  • UTKFace: A real-world face dataset containing 33,488 images, focusing on three attributes: age, gender, and ethnicity.
Diagonal training setups for the DSprites (left) and UTKFace (right) datasets.

Key Insights and Results

Through the WCST-ML framework, the authors uncovered several fundamental behaviors of DNNs:

  • The Hierarchy of Cues: DNNs do not treat all features equally. In the DSprites experiments, models almost universally preferred Color > Scale > Shape > Orientation. When tested on the off-diagonal unbiased data, models achieved nearly 100% accuracy based on color, but dropped to chance levels (33%) for shape and orientation. In the UTKFace dataset, the preference was Ethnicity > Gender > Age.
  • Simplicity is Abundant: The "Simplicity Bias" is a geometric reality of the network. Features with lower Kolmogorov complexity (simpler descriptions) occupy a significantly larger volume of the parameter space. To memorize the "Color" cue, a ResNet20 required only 1.2K parameters. In contrast, it required 17.6K parameters for "Shape" and over 273K for "Orientation." Statistically, a model is simply more likely to "fall into" a solution that uses a simple cue.
  • Flatness Equals Preference: Solutions that rely on preferred (simpler) cues tend to sit in flatter, wider valleys on the loss surface. Because deep learning optimization naturally gravitates toward "flat" minima, the model is inherently drawn to these simple shortcuts.
  • The Fairness Trap: The study highlights a dangerous reality for algorithmic fairness. Because ethnicity is often a "simpler" visual cue than age or specific facial geometry, models are naturally drawn to use it as a shortcut, even when other features are equally predictive.

Limitations and Future Work

The paper focuses on diagnosing why these biases exist rather than providing a "push-button" algorithmic fix. The authors emphasize that because these preferences are "inborn" to the geometry of neural networks, simply balancing a dataset (making cues equally valid) is not enough to stop shortcut learning.

Future work must focus on developing active human-in-the-loop interventions or specialized regularization techniques that can force a model to ignore an "abundant" simple shortcut in favor of a more complex, but more robust, feature. Understanding how these parameter-space preferences shift across different architectures, such as Graph Neural Networks or specialized bio-inspired models, remains an open area for exploration.

Summary updated

Abstract

Deep neural networks (DNNs) often rely on easy–to–learn discriminatory features, or cues, that are not necessarily essential to the problem at hand. For example, ducks in an image may be recognized based on their typical background scenery, such as lakes or streams. This phenomenon, also known as shortcut learning, is emerging as a key limitation of the current generation of machine learning models. In this work, we introduce a set of experiments to deepen our understanding of shortcut learning and its implications. We design a training setup with several shortcut cues, named WCST-ML, where each cue is equally conducive to the visual recognition problem at hand. Even under equal opportunities, we observe that (1) certain cues are preferred to others, (2) solutions biased to the easy–to–learn cues tend to converge to relatively flat minima on the loss surface, and (3) the solutions focusing on those preferred cues are far more abundant in the parameter space. We explain the abundance of certain cues via their Kolmogorov (descriptional) complexity: solutions corresponding to Kolmogorov-simple cues are abundant in the parameter space and are thus preferred by DNNs. Our studies are based on the synthetic dataset DSprites and the face dataset UTKFace. In our WCST-ML, we observe that the inborn bias of models leans toward simple cues, such as color and ethnicity. Our findings emphasize the importance of active human intervention to remove the inborn model biases that may cause negative societal impacts.
BibTeX
@inproceedings{scimeca2022which,
  author = {Luca Scimeca and Seong Joon Oh and Sanghyuk Chun and Michael Poli and Sangdoo Yun},
  title = {{Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=qRDQi3ocgR3},
}
Figure