Abstract
Cluster analysis plays an indispensable role in machine learning and data mining. Learning a good data representation is crucial for clustering algorithms. Recently, deep clustering, which can learn clustering-friendly representations using deep neural networks, has been broadly applied in a wide range of clustering tasks. Existing surveys for deep clustering mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering. To address this issue, in this paper we provide a comprehensive survey for deep clustering in views of data sources. With different data sources and initial conditions, we systematically distinguish the clustering methods in terms of methodology, prior knowledge, and architecture. Concretely, deep clustering methods are introduced according to four categories, i.e., traditional single-view deep clustering, semi-supervised deep clustering, deep multi-view clustering, and deep transfer clustering. Finally, we discuss the open challenges and potential future opportunities in different fields of deep clustering.
Introduction
Clustering is a cornerstone of unsupervised learning, yet traditional "shallow" methods—such as k-means or spectral clustering—often struggle with the high-dimensional, non-linear nature of modern datasets like images and social networks. These traditional approaches rely heavily on manual feature engineering, which is both labor-intensive and frequently suboptimal for capturing complex data structures.
Deep clustering has emerged as a powerful alternative, leveraging deep neural networks to automatically learn "clustering-friendly" representations. While previous research has focused primarily on network architectures (e.g., Autoencoders), this survey provides a broader taxonomy based on data sources and initial conditions, addressing real-world complexities such as partial labels and multi-modal data streams.
Methodology: A Taxonomy of Data Sources
The landscape of deep clustering can be organized into four primary pillars based on the structure of the input data and the available prior knowledge.
- Deep Single-View Clustering: The most prevalent form, utilizing data from a single source. It is further divided into Separate Analysis (sequential feature learning and clustering) and Joint Analysis (simultaneous optimization). Common architectures include DAE-based (Autoencoders), VAE-based (Variational), GAN-based (Generative), and GNN-based (Graph) networks.
- Semi-Supervised Deep Clustering: These methods integrate limited prior knowledge, such as "must-link" or "cannot-link" constraints, to guide the clustering process.
- Deep Multi-View Clustering: Designed for heterogeneous data (e.g., video with audio), these models seek a shared latent space that captures complementary information across all views.
- Deep Transfer Clustering: This approach applies knowledge from a labeled source domain to an unlabeled target domain, often categorized under Unsupervised Domain Adaptation (UDA).
Key Insights: Frameworks and Loss Functions
A critical finding in recent research is the superiority of Joint Analysis. By optimizing the feature extractor and clustering objectives simultaneously, the network is prevented from learning features that are irrelevant to the final grouping task.
Modern frameworks often incorporate "Extra Tasks"—auxiliary objectives like predicting image rotations—to force the network to learn robust semantic features without explicit labels. Furthermore, the evolution of loss functions has moved beyond simple reconstruction error to include complex combinations of Kullback-Leibler (KL) divergence, adversarial loss, and contrastive loss to refine the latent space $Z$.
Datasets and Trends
Deep clustering methods are typically evaluated on standard benchmarks across various domains:
- Images: MNIST, CIFAR-10/100, and ImageNet.
- Graphs: Citation and social networks.
- Transfer Learning: The OFFICE dataset.
- Multi-modal: Datasets combining text, image, and audio features.
Trends indicate a significant shift toward Graph Neural Networks (GNNs), which capture structural relationships between data points, and Generative Models (VAEs/GANs), which provide a more mathematically sound understanding of the underlying data distribution.
Future Directions and Open Challenges
Despite rapid progress, several challenges remain:
- Theoretical Foundations: There is a need for more rigorous analysis explaining the convergence of deep architectures with clustering losses.
- Efficiency: Developing lightweight models that can handle massive-scale data without overfitting on smaller datasets is essential.
- Heterogeneous Data: Creating generalized frameworks capable of handling complex, real-time web streams remains an unsolved problem.
- Noise Robustness: Improving multi-view methods to better handle contradictory or noisy information between different data streams.
Summary updated