Tracking the Change of Knowledge Through Layers in Neural Networks

Xu Cheng, Lei Cheng, Zhaoran Peng, Quanshi ZhangSubmission to ICLR 2024 · Withdrawn

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

This paper aims to explain how a deep neural network (DNN) gradually extracts new knowledge and forgets noisy features through layers in forward propagation. Up to now, although how to define knowledge encoded by the DNN has not reached a consensus so far, previous studies have derived a series of mathematical evidences to take interactions as symbolic primitive inference patterns encoded by a DNN. We extend the definition of interactions and, for the first time, extract interactions encoded by intermediate layers. We quantify and track the newly emerged interactions and the forgotten interactions in each layer during the forward propagation, which shed new light on the learning behavior of DNNs. The layer-wise change of interactions also reveals the change of the generalization capacity and instability of feature representations of a DNN.
BibTeX
@inproceedings{cheng2024tracking,
  author = {Xu Cheng and Lei Cheng and Zhaoran Peng and Quanshi Zhang},
  title = {{Tracking the Change of Knowledge Through Layers in Neural Networks}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=7zEoinErzQ},
}
Figure