Abstract
The manifold hypothesis suggests that high-dimensional data like images reside on low-dimensional manifolds. However, measuring these manifolds is challenging, limiting our understanding of deep learning dynamics. We introduce neural frames, a diagnostic tool inspired by differential geometry, to explore the local neighborhoods of data manifolds as they pass through neural network layers. By tracking how specific perturbations—such as rotations or noise—are processed, we reveal that models selectively preserve semantically meaningful variations while discarding noise. Our findings challenge the intuition that data augmentation leads to representational collapse, showing instead that it increases internal sensitivity to these variations.
Introduction: The Geometry of Vision
Despite the massive success of deep learning, models often remain "black boxes." While we can study global representation structures (e.g., how a whole dataset is clustered), we lack granular tools to understand local behavior. For instance, how does a model's internal geometry change when an image is slightly perturbed? Does it treat a minor rotation the same way it treats random pixel noise?
Understanding this local behavior is crucial for explaining generalization and the trade-offs in adversarial training. We address this gap by examining the "tangent bundle" of the data manifold—the space of all possible infinitesimal changes to an image—using a new framework called neural frames.
Methodology: The Neural $k$-Frame
A $k$-frame in differential geometry is a set of $k$ linearly independent vectors in the tangent space of a point on a manifold. We adapt this concept to neural networks through a three-step process:
- Frame Construction: We assemble vectors representing specific perturbations, such as Augmentation Frames (infinitesimal shifts in brightness, rotation, etc.) and Noise Frames (random Gaussian noise).
- The Push-forward: These vectors are passed through the layers of a neural network to observe how the internal representation of these local neighborhoods evolves.
- Measurement via Stable Rank: We use Stable Rank to quantify the "effective dimensionality" of these frames. A drop in stable rank indicates compression of those directions, while a high rank indicates preservation.
Key Insights and Results
Our analysis across various architectures (ResNets, ViTs, etc.) and training regimes on ImageNet yields several key findings:
Selective Compression
Models are highly discerning in how they process information. They rapidly collapse "off-manifold" directions (random noise) while preserving "on-manifold" directions (semantic augmentations) throughout most of the network. As shown in the figure below, the stable rank of noise frames drops precipitously after the first layer, whereas augmentation frames maintain their dimensionality until the final output.
The Invariance Paradox
A common intuition is that data augmentation makes a model invariant by "collapsing" those variations. Our results show the opposite: models trained with heavy augmentation actually preserve those modes of variation more faithfully in their hidden layers. Invariance is only achieved at the very final classification layer.
Adversarial Trade-offs
Adversarial training introduces a geometric shift. As the robustness parameter ($\epsilon$) increases, models begin to preserve noise directions more and semantic augmentation directions less. This provides a geometric explanation for why adversarial robustness often degrades standard accuracy: the model's internal geometry becomes less aligned with the natural data manifold.
Conclusion
Neural frames provide a powerful window into the inner workings of deep learning models. By demonstrating that high-performing models better "fit" the underlying data manifold and selectively preserve semantic variations, this tool offers a path toward more interpretable and geometrically-grounded architecture design.
Summary updated