Unsupervised Learning on Neural Network Outputs: with Application in Zero-shot Learning

Yao Lu

Abstract

The outputs of a trained neural network contain much richer information than just a one-hot classifier. For example, a neural network might give an image of a dog a probability of one in a million of being a cat, but this is still much larger than the probability of it being a car. To reveal the hidden structure in these outputs, we apply two unsupervised learning algorithms, PCA and ICA, to the outputs of a deep Convolutional Neural Network (CNN) trained on the ImageNet of 1000 classes. The PCA/ICA embedding of the object classes reveals their visual similarity, and the components can be interpreted as common visual features shared by similar classes. We propose a new zero-shot learning method using these visual features, achieving state-of-the-art results on the ImageNet dataset of over 20,000 classes.

Introduction: Extracting "Dark Knowledge"

When a deep neural network classifies an image, we usually only care about the top prediction. However, the full probability distribution across all classes—the softmax output—contains "dark knowledge" reflecting the visual similarities the network has learned. The challenge is how to systematically extract and utilize this hidden structure. Traditional zero-shot learning (ZSL) often relies on manually annotated attributes or external word embeddings. This paper asks: Can we perform zero-shot learning by looking at the visual relationships the network has already discovered on its own?

Methodology

The proposed pipeline treats the outputs of a pre-trained CNN as random variables for unsupervised learning.

Logit Normalization

Standard softmax functions tend to suppress small values, making the output distribution extremely sparse. To "magnify" the dark knowledge, the author uses normalized logits, which preserve the relative differences between low-probability classes while maintaining a probability distribution.

Comparison of Softmax (a) and Normalized Logits (b) outputs for a single image. Softmax is sparse, while normalized logits reveal the underlying distribution of non-target classes.

Unsupervised Feature Extraction (PCA and ICA)

The author applies Principal Component Analysis (PCA) and Independent Component Analysis (ICA) to the CNN outputs. While PCA finds directions of maximum variance, ICA looks for statistically independent components. A key observation is that CNN outputs exhibit high kurtosis (highly non-Gaussian), making them ideal candidates for ICA, which relies on non-Gaussianity to separate signals.

Kurtosis of neuron outputs for Softmax (a) and Normalized Logits (b). The high values (well above the Gaussian baseline of 3) justify the use of ICA for feature extraction.

Zero-Shot Learning via CCA

To bridge the gap between "seen" classes (used for training) and "unseen" classes, the author uses Canonical Correlation Analysis (CCA). This algorithm finds a common projection space between the visual features (extracted via PCA/ICA) and semantic features (derived from WordNet distances using Multi-dimensional Scaling).

Key Insights and Results

  • ICA Interpretability: While PCA and ICA yield similar accuracy in ZSL, ICA components are much easier for humans to interpret. A single ICA component might represent "dog-ness" or "car-ness," showing high values only for related categories, whereas PCA components tend to be more distributed.
  • Visual vs. Semantic Disconnect: The paper highlights that visual similarity and semantic similarity don't always align. For example, a "soccer ball" is visually similar to a "football helmet" but semantically distant.
  • State-of-the-Art ZSL: The proposed method achieved state-of-the-art results on the massive ImageNet-21k benchmark. On the "unseen" set of 20,842 classes, it reached a Top-1 accuracy of 1.8% and a Top-20 accuracy of 9.7%. These results are significant given the extreme difficulty of a 20,000-way classification task with zero training examples.

Conclusion

This work demonstrates that the "dark knowledge" in trained neural networks is a powerful resource for understanding visual relationships. By applying unsupervised learning to these outputs, we can discover latent visual features that enable effective large-scale zero-shot learning without the need for manual attribute annotation.

Summary updated

Abstract

The outputs of a trained neural network contain much richer information than just an one-hot classifier. For example, a neural network might give an image of a dog the probability of one in a million of being a cat but it is still much larger than the probability of being a car. To reveal the hidden structure in them, we apply two unsupervised learning algorithms, PCA and ICA, to the outputs of a deep Convolutional Neural Network trained on the ImageNet of 1000 classes. The PCA/ICA embedding of the object classes reveals their visual similarity and the PCA/ICA components can be interpreted as common visual features shared by similar object classes. For an application, we proposed a new zero-shot learning method, in which the visual features learned by PCA/ICA are employed. Our zero-shot learning method achieves the state-of-the-art results on the ImageNet of over 20000 classes.
BibTeX
@inproceedings{lu2016unsupervised,
  author = {Yao Lu},
  title = {{Unsupervised Learning on Neural Network Outputs: with Application in Zero-shot Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
  url = {https://openreview.net/forum?id=BJb1Brfu-H},
}
Figure