Interpreting and Disentangling Feature Components of Various Complexity from DNNs

Jie Ren, Mingjie Li, Zexu Liu, Quanshi Zhang

Abstract

This paper aims to define, quantify, and analyze the feature complexity that is learned by a Deep Neural Network (DNN). We propose a generic definition for feature complexity based on the minimum number of non-linear transformations required for representation. Given the feature of a certain layer, our method disentangles feature components of different complexity orders. We further design a set of metrics to evaluate the reliability, effectiveness, and significance of over-fitting for these components. Our findings reveal a close relationship between feature complexity and DNN performance, offering new insights into network compression and knowledge distillation.

Introduction: Beyond Architectural Capacity

While the theoretical maximum complexity of a neural network is determined by its depth and width, the actual complexity of the features it learns for a specific task remains poorly understood. For example, a high-capacity model like a 50-layer ResNet might only utilize a fraction of its representational power when solving a simple linear problem.

The "black box" nature of DNNs makes it difficult to discern whether a model is succeeding due to robust, high-level conceptual understanding or by over-fitting to simple noise. This research addresses the gap between architectural capacity and realized feature complexity, providing a mathematical framework to quantify what a network has actually "understood."

Decomposition of a target feature into components of varying complexity, from low-order shapes to high-order textures and noise.

Methodology: Disentangling Complexity

The authors propose a method to disentangle an intermediate-layer feature into components of different complexity orders.

Defining Complexity

The complexity of a feature component is defined by the minimum number of non-linear layers ($k$) required to compute it. Under this definition, a feature that can be mimicked by a shallow 2-layer network is considered "simpler" than one requiring a 10-layer network.

Disentangler Networks

To extract these components, the researchers employ a series of "disentangler networks" of varying depths to mimic a target teacher DNN. A shallow disentangler captures low-complexity features, while progressively deeper networks capture those plus incremental "higher-order" components.

Architecture of the disentangler network designed to extract specific complexity orders through iterative optimization.

Evaluation Metrics

To analyze the disentangled components, three key metrics are introduced:

  • Reliability: Uses a cycle-consistency loss to determine if features are consistently learned across different model initializations.
  • Effectiveness: Utilizes Shapley values from game theory to quantify each component's contribution to reducing the final task loss.
  • Over-fitting Significance: Measures the discrepancy between a component's contribution to training loss versus testing loss.

Experimental Insights

The framework was validated across standard benchmarks including CIFAR-10, CUB200-2011 (fine-grained birds), and Stanford Dogs. Key findings include:

  • Task-Driven Complexity: Models trained on simpler tasks naturally gravitate toward low-complexity features. As task difficulty increases, the distribution shifts toward higher-order features.
  • Knowledge Distillation Mechanics: Distilled models (smaller models trained to mimic larger ones) often learn more reliable low-complexity features and fewer over-fitted high-complexity features compared to models trained from scratch.
  • Performance Prediction: Final testing accuracy can be predicted by analyzing the distribution of feature complexity components using a simple linear regressor.
  • Visualization: Low-complexity features typically represent general "blobs" or shapes, while high-complexity components capture fine textures or high-frequency noise.

Conclusion

This work provides a generic mathematical tool to analyze the success of network architectures and training strategies. By replacing raw features with specific disentangled components—specifically those with high reliability and effectiveness—the authors demonstrated improved classification accuracy in pre-trained models. Future work will look to extend this analysis to generative models and Natural Language Processing architectures.

Summary updated

Abstract

This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN. We propose a generic definition for the feature complexity. Given the feature of a certain layer in the DNN, our method disentangles feature components of different complexity orders from the feature. We further design a set of metrics to evaluate the reliability, the effectiveness, and the significance of over-fitting of these feature components. Furthermore, we successfully discover a close relationship between the feature complexity and the performance of DNNs. As a generic mathematical tool, the feature complexity and the proposed metrics can also be used to analyze the success of network compression and knowledge distillation.
BibTeX
@inproceedings{ren2021interpreting,
  author = {Jie Ren and Mingjie Li and Zexu Liu and Quanshi Zhang},
  title = {{Interpreting and Disentangling Feature Components of Various Complexity from DNNs}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=kZaFH7YQ04},
}

Loading related content...

Figure