Abstract
Recent advances in deep learning have shown their ability to learn strong feature representations for images. The task of image clustering naturally requires good feature representations to capture the distribution of the data and subsequently differentiate data points from one another. Often these two aspects are dealt with independently and thus traditional feature learning alone does not suffice in partitioning the data meaningfully. Variational Autoencoders (VAEs) naturally lend themselves to learning data distributions in a latent space. Since we wish to efficiently discriminate between different clusters in the data, we propose a method based on VAEs where we use a Gaussian Mixture prior to help cluster the images accurately. We jointly learn the parameters of both the prior and the posterior distributions. Our method represents a true Gaussian Mixture VAE. This way, our method simultaneously learns a prior that captures the latent distribution of the images and a posterior to help discriminate well between data points. We also propose a novel reparametrization of the latent space consisting of a mixture of discrete and continuous variables. One key takeaway is that our method generalizes better across different datasets without using any pre-training or learnt models, unlike existing methods, allowing it to be trained from scratch in an end-to-end manner. We verify our efficacy and generalizability experimentally by achieving state-of-the-art results among unsupervised methods on a variety of datasets. To the best of our knowledge, we are the first to pursue image clustering using VAEs in a purely unsupervised manner on real image datasets.
Introduction
Clustering images presents a fundamental "chicken and egg" challenge: effective clustering requires robust feature representations, yet learning these features in an unsupervised manner often necessitates structural guidance, such as pre-defined clusters. Traditional techniques like K-means frequently struggle with raw pixel data as they fail to capture semantic complexity. While deep learning has advanced feature extraction, many "Deep Clustering" methods either treat feature learning and clustering as disjoint steps or rely heavily on pre-trained models (like ResNet trained on ImageNet).
This paper introduces a "true" Gaussian Mixture Variational Autoencoder (VAE) for image clustering. Unlike existing generative models like GMVAE and VaDE, which often suffer from complex stochastic priors or require "warm-up" pre-training phases, our proposed framework learns to group data and extract features simultaneously from scratch.
Methods
The Variational Clustering framework integrates a Gaussian Mixture Model (GMM) directly into the latent space of a VAE.
Key technical components include:
- Deterministic GMM Prior: In contrast to models where the prior might depend on a noise variable, this model utilizes a deterministic prior where each cluster is represented by its own learned mean and variance.
- Direct Inference: The model predicts cluster assignment probabilities $q(k|z)$ directly from the input data, making the inference process more straightforward and principled.
- Joint Learning: The parameters for the prior (the cluster shapes) and the posterior (the data encoding) are learned jointly in an end-to-end fashion. No pre-training is required.
- Augmentation Loss: To enhance robustness, the authors introduce a loss function that compares an image with its augmented version (e.g., rotated or brightened). The model minimizes the $L2$ distance between cluster predictions and the 2-Wasserstein distance between their posterior distributions, ensuring that semantically similar images end up in the same cluster.
Experimental Evaluation
The model was evaluated on a wide range of benchmarks to test its efficacy and generalizability:
- Toy Data: A 2D dataset of 5 nested circles to test latent space separation.
- MNIST & Fashion-MNIST: Standard benchmarks for handwritten digits and clothing items.
- STL-10 & CIFAR-10/100: Challenging datasets containing real-world objects (animals, vehicles) at various resolutions.
- FRGCv2: A face image dataset used to test clustering with a higher number of classes ($k=20$).
Key Insights and Results
The research highlights several critical findings regarding unsupervised clustering:
- Simplicity and End-to-End Training: Transitioning to a deterministic GMM prior makes the model more intuitive and easier to train. Learning everything at once proves superior to the "initialize then refine" approach used by models like VaDE, preventing the model from getting stuck in poor local minima created during separate pre-training phases.
- Self-Supervision via Augmentation: The inclusion of the augmentation loss is a "game changer." It forces the network to ignore noise (like slight rotations) and focus on the semantic content of the image, leading to much stronger intra-cluster relations.
- State-of-the-Art Performance: Among purely unsupervised methods, this model consistently outperformed others. For example, on CIFAR-10, it achieved 44.5% accuracy, significantly higher than GMVAE (24.74%).
- Robustness and Generalizability: The model maintained high accuracy even with imbalanced data (e.g., 89.6% on MNIST with a 10:1 imbalance ratio). Furthermore, it demonstrated impressive transfer learning; a model trained on CIFAR-10 could cluster the 9 overlapping classes in STL-10 with 41.03% accuracy without seeing the STL-10 images during training.
Conclusion
While the proposed method sets a new bar for unsupervised image clustering, a significant gap remains compared to supervised models or those using massive pre-trained backbones. Future work includes exploring VAE-GAN hybrids to produce even more realistic image samples and implementing more explicit inference constraints on the prior to further sharpen the boundaries between clusters in the latent space.
Summary updated