Abstract
The underspecification of machine learning pipelines means validation performance alone cannot guarantee robustness to distribution shifts. Understanding how a neural network solves a task is crucial for generalization. This work investigates the geometric relationship between learned functions and data representations, challenging the notion that deep networks are always biased toward simpler solutions. We demonstrate that architectural inductive biases, specifically Neural Anisotropic Directions (NADs), significantly influence feature selection, sometimes leading networks to prefer complex, non-linear solutions over simpler linear ones.
Introduction: The Challenge of Underspecification
In deep learning, "underspecification" describes a scenario where multiple distinct predictors can achieve identical, perfect accuracy on a validation set but fail differently under distribution shifts. This ambiguity makes it difficult to predict how a model will behave in real-world environments with varying lighting, backgrounds, or noise. Traditionally, it has been conjectured that neural networks possess a "simplicity bias," naturally preferring the geometrically simplest solution (e.g., a linear separator) when available. This paper investigates whether this bias holds true across different architectures and how the geometry of the network itself dictates feature preference.
Neural Anisotropic Directions (NADs)
The authors introduce Neural Anisotropic Directions (NADs) as a lens to study architectural bias. NADs are specific directions in the input space where a network is inherently more sensitive at initialization. These directions are defined by the eigenvectors of the input gradient covariance matrix. The core hypothesis is that a network's "preference" for certain features is determined by how well those features align with its strongest NADs.
Experimental Framework: Probing Feature Selection
To test this, the researchers constructed underspecified datasets by concatenating two distinct data "channels," each sufficient to solve the task independently:
- Channel 1: A simple, linearly separable task ($D(v)$).
- Channel 2: Either another linear task or a complex, non-linear task (binarized CIFAR-10).
By rotating the discriminative features of these tasks to align with different NADs—ranging from "strong" low-index directions to "weak" high-index directions—the authors could observe which channel the network prioritized during training.
Key Findings: Architectural Preference vs. Simplicity
The experiments revealed a striking departure from the simplicity bias in convolutional neural networks (CNNs) like ResNet-18 and LeNet.
1. Complexity Over Simplicity
If a simple linear feature is aligned with a "weak" NAD (a direction the architecture is less sensitive to), CNNs will often ignore it. Instead, they solve the task using complex, non-linear features (like CIFAR-10) if those features align better with their "strong" NADs. This preference persists even when the complex solution results in lower test accuracy compared to what the linear solution would have provided.
2. MLP vs. CNN Behavior
Interestingly, Multi-Layer Perceptrons (MLPs) do not share this behavior. MLPs consistently prefer the simpler linear solution regardless of NAD alignment. This suggests that the "simplicity bias" often cited in literature may be a specific characteristic of fully connected layers rather than a universal property of deep learning architectures.
3. Margin Modulation
In purely linear tasks, the alignment with NADs influences the model's decision boundary. When features align with weak NADs, the model tends to become a "decision stump," ignoring useful information and becoming more brittle. This indicates that architectural bias directly impacts the robustness and margin of the learned predictor.
Conclusion and Future Directions
This study highlights that understanding architectural inductive bias is fundamental to addressing fairness and robustness. The discovery that CNNs can be biased toward complexity over simplicity when the latter aligns poorly with their internal geometry provides a new perspective on why models might rely on "spurious" or overly complex features.
While this work uses synthetic concatenations to isolate variables, it lays the groundwork for "steering" models toward more robust features by understanding and potentially manipulating their anisotropic preferences. Future research will explore how training hyperparameters interact with these biases and extend the analysis to a broader range of modern architectures.
Summary updated