Top-Down Deep Clustering with Multi-Generator GANs

Daniel P. M. de Mello, Renato M. Assunção, Fabricio MuraiAAAI 2022 · DOI

TL;DR

HC-MGAN is a novel deep clustering method that adopts a top-down hierarchical approach by leveraging the "specialization" tendency of Multi-Generator GANs (MGANs). By training a classifier to distinguish between different generators and iteratively refining these splits, the model organizes high-dimensional data into a semantically coherent tree without requiring human labels.

What Problem Does This Paper Address?

Clustering high-dimensional data like images is notoriously difficult because standard geometric distances (like Euclidean distance) fail to capture semantic similarity in raw pixel space. While "Deep Clustering" (DC) uses neural networks to learn better representations, most existing methods are "flat"—they require the user to specify the exact number of clusters ($K$) upfront and provide no structural relationship between those clusters.

Furthermore, while Generative Adversarial Networks (GANs) are famous for learning powerful data representations, using them for clustering has typically relied on a single generator with a latent categorical variable (like InfoGAN). These often struggle with representation capacity. The authors address these gaps by introducing a hierarchical framework that uses multiple generators to "carve up" the data distribution organically.

Methods

The core of the paper is HC-MGAN (Hierarchical Clustering MGAN). Instead of clustering all data at once, it builds a binary tree from the top down through a recursive splitting process.

The Raw Split

For any given node in the tree, the authors train an MGAN with two generators ($G\alpha, G\beta$), one discriminator ($D$), and one classifier ($C$). Because generators naturally tend to specialize in different sub-regions of data to "fool" the discriminator, $G\alpha$ might start generating "3s" while $G\beta$ generates "5s."

The Raw Split mechanism: training multiple generators to specialize and using a classifier to partition the data distribution.

As shown in Figure 4, the classifier $C$ is trained to identify which generator produced a specific fake image. Once trained, $C$ can then be used to cluster real data by assigning probabilities based on which generator's distribution the real image most closely resembles.

The Refinement Phase

The "Raw Split" is often noisy. To fix this, the authors introduce a refinement step where two independent GANs are trained on the resulting soft-clusters. By giving each discriminator a "focused" view of a specific sub-region, the generators are forced to become even more realistic and distinct, sharpening the cluster boundaries.

Hierarchical Growth

This process repeats recursively. The algorithm selects the leaf node with the largest "mass" (most samples) and splits it again, creating a taxonomy of the dataset.

Top-down hierarchical clustering of MNIST digits into a semantic taxonomy.

Figure 1 illustrates this recursive process on the MNIST dataset, where the root node $S_0$ is progressively divided into more specific clusters, such as separating "1s" from other digits in the final leaf nodes.

Datasets

The authors evaluated HC-MGAN on three standard image benchmarks:

  • MNIST: 70,000 handwritten digits.
  • Fashion MNIST (FMNIST): 70,000 images of clothing items (more complex than digits).
  • Stanford Online Products (SOP): A challenging dataset of retail products featuring high intra-class variance.

Key Insights and Ideas

  • Organic Specialization: The "aha moment" is realizing that we don't need to force a GAN to cluster; if you give a GAN multiple generators, they want to specialize to minimize the classification loss.
  • Fake-to-Real Transfer: The classifier $C$ is trained entirely on "fake" images generated by the model, yet it generalizes remarkably well to real data. This allows for unsupervised labeling based on the model's own internal logic.
  • Semantic Taxonomy: Unlike flat clustering, the hierarchical approach reveals how classes relate. For example, in FMNIST, the model might first separate "footwear" from "clothing" before further splitting "sneakers" from "sandals."

Key Results

  • Competitive Accuracy: On MNIST, HC-MGAN achieved a Clustering Accuracy (ACC) of 0.943, outperforming other GAN-based methods like InfoGAN (0.890).
  • Strong FMNIST Performance: It reached an ACC of 0.721, beating almost all non-augmented deep clustering methods.
  • Qualitative Coherence: Visualizations of the FMNIST tree show that the model's errors are "sensible"—it mostly struggles to distinguish between visually similar items like shirts and pullovers, while perfectly separating footwear and trousers.

Limitations and Future Work

  • Data Augmentation: The current version does not use data augmentation (like rotations or cropping). Adding this would likely significantly boost performance.
  • Computational Complexity: Training multiple GANs in a tree structure is more intensive than a single-pass flat clustering algorithm.
  • Future Direction: The authors suggest exploring ways to automatically determine when to stop splitting the tree, allowing the model to discover the "natural" number of clusters in a dataset.

Summary updated

BibTeX
@inproceedings{mello2022top,
  author = {Daniel P. M. de Mello and Renato M. Assunção and Fabricio Murai},
  title = {{Top-Down Deep Clustering with Multi-Generator GANs}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i7.20745},
}
Figure