TL;DR
This survey provides a structured roadmap of "Deep Clustering"—the fusion of deep learning’s representational power with unsupervised clustering objectives. By categorizing methods based on data sources (single-view, multi-view, semi-supervised, and transfer learning), the authors move beyond simple network architectures to address how deep learning can solve complex, real-world grouping problems.
What Problem Does This Paper Address?
Clustering is a fundamental task in data mining, but traditional "shallow" methods (like k-means or spectral clustering) often fail when dealing with high-dimensional, non-linear data like images or complex social networks. These older methods rely on manual feature engineering, which is difficult and often suboptimal.
While "Deep Clustering" has emerged to solve this by learning "clustering-friendly" representations automatically, previous surveys focused almost exclusively on single-view data or specific network architectures (like Autoencoders). This paper addresses the gap by providing a comprehensive taxonomy that accounts for the variety of data sources and initial conditions found in the real world, such as having partial labels, multiple data streams, or related tasks to learn from.
Methods
The authors organize the vast landscape of deep clustering into four primary pillars:
- Deep Single-View Clustering: The most common form, where data comes from one source. The paper distinguishes between "Separate Analysis" (learning features first, then clustering) and "Joint Analysis" (optimizing the feature extractor and the clustering labels simultaneously). They further categorize these by network type:
- DAE-based: Deep Autoencoders minimizing reconstruction loss.
- VAE-based: Variational Autoencoders using probabilistic latent spaces.
- GAN-based: Generative Adversarial Networks using a discriminator to ensure latent distributions are "realistic."
- GNN-based: Graph Neural Networks that capture the structural relationships between data points.
- Semi-Supervised Deep Clustering: Methods that incorporate a small amount of prior knowledge, usually in the form of "must-link" (two points belong together) or "cannot-link" (two points must be separate) constraints.
- Deep Multi-View Clustering: Designed for data with multiple representations (e.g., a video with both audio and visual tracks). These methods aim to find a common latent space that utilizes complementary information from all views.
- Deep Transfer Clustering: Leverages knowledge from a labeled "source" domain to cluster an unlabeled "target" domain, often referred to as Unsupervised Domain Adaptation (UDA).
Datasets
As a comprehensive survey, this paper does not introduce a new dataset. Instead, it reviews hundreds of papers that utilize standard benchmarks across various domains:
- Image Datasets: MNIST, CIFAR-10/100, ImageNet, and the OFFICE dataset (for transfer learning).
- Graph Datasets: Social networks and citation networks.
- Multi-modal Datasets: Data containing text, images, and audio features.
Key Insights and Ideas
- The "Joint Analysis" Advantage: The authors highlight that joint optimization—where the clustering loss actually guides the feature learning—is generally superior to two-step processes. It prevents the network from learning features that are irrelevant to the final grouping task.
- The Rise of "Extra Tasks": Modern deep clustering often uses auxiliary tasks (like predicting image rotations or data augmentation) to "force" the network to learn more robust, semantic features without needing labels.
- Structural Awareness: There is a significant shift toward GNN-based clustering. By treating data points as nodes in a graph, models can capture not just what a data point is, but how it relates to its neighbors, leading to much more accurate community discovery.
- Generative Power: VAEs and GANs are increasingly popular because they allow the model to understand the underlying distribution of the data, making the resulting clusters more mathematically sound and "interpretable" in the latent space.
Key Results
The paper serves as a taxonomic guide rather than a benchmark report, but it summarizes several critical trends:
- Taxonomy Tables: The authors provide exhaustive tables (Tables 2-6) that categorize dozens of major algorithms by their network type, loss functions, and specific characteristics (e.g., whether they require the number of clusters k to be known in advance).
- Loss Function Evolution: The survey tracks the evolution of loss functions from simple reconstruction errors to complex combinations of KL-divergence, adversarial loss, and contrastive loss.
- Multi-View Synergy: The results discussed indicate that multi-view methods consistently outperform single-view counterparts by "filling in the gaps" where one data source might be noisy or incomplete.
Limitations and Future Work
The authors identify several "open frontiers" for the research community:
- Theoretical Exploration: While these models work well in practice, there is a lack of rigorous theoretical analysis explaining why certain combinations of deep architectures and clustering losses converge to optimal solutions.
- Model Efficiency: Deep clustering requires massive amounts of data and compute. Developing "lightweight" versions that don't overfit on small datasets is a major challenge.
- Massive & Complex Data: Most current models are "bespoke" for specific data types. Creating generalized deep clustering frameworks that can handle heterogeneous, massive-scale data (like real-time web streams) remains an unsolved problem.
- Noise in Multi-View: Better methods are needed to filter out "noise" when one view of the data contradicts another.
Summary updated