Does the Geometry of the Data Control the Geometry of Neural Predictions? (Student Abstract)

Anirudh Cowlagi, Pratik ChaudhariAAAI 2022 · OpenReview

Abstract

This paper studies the over-parameterization of deep neural networks using the Fisher Information Matrix (FIM) from information geometry. We identify several surprising trends in the structure of its eigenspectrum and how this structure relates to the eigenspectrum of the data correlation matrix. We identify how the eigenspectrum relates to the topology of the predictions of the model and develop a "model reduction" method for deep networks. This ongoing investigation hypothesizes certain universal trends in the FIM of deep networks that may shed light on their effectiveness.

The Mystery of Over-parameterization

In classical statistics, a model with more parameters than data points is typically a recipe for disaster: it should simply memorize noise and fail to generalize—a phenomenon known as overfitting. Yet, modern deep learning thrives on massive over-parameterization. This "benign over-fitting" remains one of the most significant mysteries in the field.

This research explores why these networks remain robust by viewing them through the lens of "sloppy models," a concept borrowed from physics and biology. In sloppy models, only a tiny fraction of parameter directions (the "stiff" ones) actually affect the model's behavior, while the vast majority (the "sloppy" ones) are redundant.

Measuring Sensitivity with the Fisher Information Matrix

To quantify this redundancy, the researchers analyze the Fisher Information Matrix (FIM). The FIM acts as a metric for the "model manifold"—a high-dimensional surface where each point represents the network's output for a specific set of weights.

Conceptual mapping from parameter space to behavior space, illustrating how isotropic parameter changes result in highly anisotropic changes in model behavior (stiff vs. sloppy directions).

As shown in the conceptual diagram above, large eigenvalues in the FIM correspond to "stiff" directions where small weight changes cause significant prediction changes. Conversely, small eigenvalues correspond to "sloppy" directions where even large weight changes have almost no impact on the final output. The study finds that neural networks are inherently sloppy: their FIM eigenspectra decay exponentially, meaning only about 5% of the available directions in weight space actually matter for the final prediction.

Data as the Architect of Model Geometry

A key finding of this work is the "peculiar relationship" between the input data and the model's internal geometry. By comparing the FIM to the Data Auto-correlation Matrix ($XX^\top$), the authors observed that the model's sloppiness is largely inherited from the structure of the data itself.

Eigenspectra of the FIM at various training stages (Begin, Mid, End) compared to the eigenspectrum of the data auto-correlation matrix (dashed line) for CIFAR-10 (a) and Boston Housing (b).

As illustrated in the plots above, the FIM's structure closely mirrors the data auto-correlation matrix. When the researchers used synthetic, unstructured data, the model's FIM became significantly less sloppy. This suggests that the redundancy in neural networks is not just a quirk of architecture, but a direct reflection of the informational structure of the data they are trained on. Furthermore, training dynamics show that while "stiff" directions remain stable, "sloppy" directions increase in magnitude as the network learns to align its representations with the most important features of the data.

Model Reduction and Topological Preservation

This geometric insight allows for massive model reduction. By projecting a trained model onto a much smaller subspace defined by the top eigenvectors of the FIM, the researchers could test if the network's "behavior" remained intact.

In experiments with CIFAR-10, a 9-layer CNN reduced to just its top 5% of FIM directions achieved 75% accuracy, compared to 77% for the full model. This projection preserved the essential topology of the model manifold. However, there appears to be a "critical threshold": reducing the model to the top 1% of directions caused the manifold to collapse into a simple line, leading to a total failure in predictive power.

Conclusion and Future Directions

This ongoing investigation suggests that the effectiveness of deep networks may lie in their ability to automatically identify and prioritize the "stiff" directions dictated by data geometry. While the authors have identified a threshold where model topology collapses, future work is needed to define formal mathematical rules to predict this threshold and to explore how these geometric properties influence generalization to unseen data. Scaling these insights to modern architectures like Large Language Models remains a significant computational challenge, as calculating the full FIM is resource-intensive.

Summary updated

Abstract

This paper studies the over-parameterization of deep neural networks using the Fisher Information Matrix from information geometry. We identify several surprising trends in the structure of its eigenspectrum, and how this structure relates to the eigenspectrum of the data correlation matrix. We identify how the eigenspectrum relates to the topology of the predictions of the model and develop a "model reduction'' method for deep networks. This ongoing investigation hypothesizes certain universal trends in the FIM of deep networks that may shed light on their effectiveness.
BibTeX
@inproceedings{cowlagi2022does,
  author = {Anirudh Cowlagi and Pratik Chaudhari},
  title = {{Does the Geometry of the Data Control the Geometry of Neural Predictions? (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=Y0kqxz0BXbp},
}
Figure