A neural anisotropic view of underspecification in deep learning

Guillermo Ortiz-Jiménez, Itamar Franco Salazar-Reque, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, Pascal FrossardarXiv 2021 · OpenReview

Abstract

The underspecification of machine learning pipelines means validation performance alone cannot guarantee robustness to distribution shifts. Understanding how a neural network solves a task is crucial for generalization. This work investigates the geometric relationship between learned functions and data representations, challenging the notion that deep networks are always biased toward simpler solutions. We demonstrate that architectural inductive biases, specifically Neural Anisotropic Directions (NADs), significantly influence feature selection, sometimes leading networks to prefer complex, non-linear solutions over simpler linear ones.

Introduction: The Challenge of Underspecification

In deep learning, "underspecification" describes a scenario where multiple distinct predictors can achieve identical, perfect accuracy on a validation set but fail differently under distribution shifts. This ambiguity makes it difficult to predict how a model will behave in real-world environments with varying lighting, backgrounds, or noise. Traditionally, it has been conjectured that neural networks possess a "simplicity bias," naturally preferring the geometrically simplest solution (e.g., a linear separator) when available. This paper investigates whether this bias holds true across different architectures and how the geometry of the network itself dictates feature preference.

Neural Anisotropic Directions (NADs)

The authors introduce Neural Anisotropic Directions (NADs) as a lens to study architectural bias. NADs are specific directions in the input space where a network is inherently more sensitive at initialization. These directions are defined by the eigenvectors of the input gradient covariance matrix. The core hypothesis is that a network's "preference" for certain features is determined by how well those features align with its strongest NADs.

Experimental Framework: Probing Feature Selection

To test this, the researchers constructed underspecified datasets by concatenating two distinct data "channels," each sufficient to solve the task independently:

  1. Channel 1: A simple, linearly separable task ($D(v)$).
  2. Channel 2: Either another linear task or a complex, non-linear task (binarized CIFAR-10).
Experimental setup using concatenated datasets: (a) two linear features and (b) a linear feature combined with CIFAR-10.

By rotating the discriminative features of these tasks to align with different NADs—ranging from "strong" low-index directions to "weak" high-index directions—the authors could observe which channel the network prioritized during training.

Key Findings: Architectural Preference vs. Simplicity

The experiments revealed a striking departure from the simplicity bias in convolutional neural networks (CNNs) like ResNet-18 and LeNet.

1. Complexity Over Simplicity

If a simple linear feature is aligned with a "weak" NAD (a direction the architecture is less sensitive to), CNNs will often ignore it. Instead, they solve the task using complex, non-linear features (like CIFAR-10) if those features align better with their "strong" NADs. This preference persists even when the complex solution results in lower test accuracy compared to what the linear solution would have provided.

2. MLP vs. CNN Behavior

Interestingly, Multi-Layer Perceptrons (MLPs) do not share this behavior. MLPs consistently prefer the simpler linear solution regardless of NAD alignment. This suggests that the "simplicity bias" often cited in literature may be a specific characteristic of fully connected layers rather than a universal property of deep learning architectures.

3. Margin Modulation

In purely linear tasks, the alignment with NADs influences the model's decision boundary. When features align with weak NADs, the model tends to become a "decision stump," ignoring useful information and becoming more brittle. This indicates that architectural bias directly impacts the robustness and margin of the learned predictor.

Conclusion and Future Directions

This study highlights that understanding architectural inductive bias is fundamental to addressing fairness and robustness. The discovery that CNNs can be biased toward complexity over simplicity when the latter aligns poorly with their internal geometry provides a new perspective on why models might rely on "spurious" or overly complex features.

While this work uses synthetic concatenations to isolate variables, it lays the groundwork for "steering" models toward more robust features by understanding and potentially manipulating their anisotropic preferences. Future research will explore how training hyperparameters interact with these biases and extend the analysis to a broader range of modern architectures.

Summary updated

Abstract

The underspecification of most machine learning pipelines means that we cannot rely solely on validation performance to assess the robustness of deep learning systems to naturally occurring distribution shifts. Instead, making sure that a neural network can generalize across a large number of different situations requires to understand the specific way in which it solves a task. In this work, we propose to study this problem from a geometric perspective with the aim to understand two key characteristics of neural network solutions in underspecified settings: how is the geometry of the learned function related to the data representation? And, are deep networks always biased towards simpler solutions, as conjectured in recent literature? We show that the way neural networks handle the underspecification of these problems is highly dependent on the data representation, affecting both the geometry and the complexity of the learned predictors. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
BibTeX
@misc{ortizjimenez2021neural,
  author = {Guillermo Ortiz-Jiménez and Itamar Franco Salazar-Reque and Apostolos Modas and Seyed-Mohsen Moosavi-Dezfooli and Pascal Frossard},
  title = {{A neural anisotropic view of underspecification in deep learning}},
  year = {2021},
  eprint = {2104.14372},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.14372},
}
Figure