In traditional machine learning, clustering is often treated as a static task: you take a set of features and group them into $K$ piles. But in the era of deep learning, this creates a "chicken and egg" problem. To learn the high-level features (like "dog" vs. "cat") that make clustering possible, a neural network needs to know which points belong together. Conversely, to know which points belong together, the network needs good features.
As Maziar Moradi Fard, Thibaut Thonet, and Eric Gaussier argue in Deep k-Means: Jointly Clustering with k-Means and Learning Representations, the solution is to make clustering differentiable, allowing the representation and the clusters to evolve together. However, a major bottleneck remains: almost all these models require the user to pre-specify the number of clusters ($K$). If you guess wrong, the model becomes a "geometric vandal," as Lirong Wu and colleagues describe in Deep Clustering and Representation Learning with Geometric Structure Preservation, crushing the natural manifold of the data to fit an arbitrary number of circular blobs.
Recent research is shifting toward "K-free" deep clustering—models that autonomously discover the natural granularity of data.
The Parametric Bottleneck
Why is fixing $K$ so damaging? When a model is forced to squeeze data into too few clusters, it merges distinct semantic categories, destroying the discriminative power of the learned features. If it is forced into too many, it fractures natural groups. Meitar Ronen, Shahaf E. Finder, and Oren Freifeld note in DeepDPM: Deep Clustering With an Unknown Number of Clusters that this creates a circular dependency: you need to know the structure to find the clusters, but the clusters are what define the structure.
To break this loop, researchers are developing "plug-and-play" modules and non-parametric frameworks that treat $K$ as a dynamic variable rather than a constant.
Mechanisms for Discovering K
How does a neural network "decide" how many clusters exist? Three primary strategies have emerged across the literature:
1. Split and Merge Dynamics
Instead of starting with $K$ clusters, models can start with a single group and iteratively refine it. Hanting Chen and the team at Huawei Noah’s Ark Lab proposed a Deep Plug-and-Play Clustering module that uses Jensen-Shannon divergence to decide when a cluster is too diverse (triggering a split) or when two clusters are too similar (triggering a merge). Similarly, DeepECT by Claudia Plant and Christian Böhm uses a top-down divisive approach to build a cluster tree, discovering the hierarchy and the count simultaneously.
2. Bayesian Non-Parametrics
A more probabilistic approach involves Dirichlet Process Mixture Models (DPMMs). In DIVA: A Dirichlet Process Based Incremental Deep Clustering Algorithm via Variational Auto-Encoder, Yuan Meng and colleagues use the "Chinese Restaurant Process" intuition. As new data arrives, the model can either assign it to an existing "table" (cluster) or "birth" a new one if the data is sufficiently novel. This allows the model to handle data streams where the number of categories grows over time.
3. Information-Theoretic Compression
Some researchers look at clustering as a communication problem. Christopher Blöcker and Ingo Scholtes introduced Neuromap, which turns the "map equation" into a differentiable loss. The intuition here is that the optimal number of clusters is the one that provides the most efficient compression of the data's flow (or relationships). By minimizing this description length, the network naturally converges on the "correct" $K$.
Preserving the Geometry
A recurring theme in this research direction is that clustering should not come at the expense of representation quality. Lirong Wu emphasizes that "good" clustering should preserve the local isometry of the data. If two points are close on the original high-dimensional manifold, they should remain close in the latent space, even as they are pulled toward cluster centers.
Methods like SpherePair by Shaojie Zhang and Ke Chen achieve this by moving away from Euclidean space entirely. By mapping data onto a hypersphere and using angular constraints, they eliminate the need for fixed "anchors" or centroids, allowing the clusters to emerge from the natural density of the data.
Why It Works: The Sandbox Effect
As Collin Leiber and Lukas Miklautz describe in their survey of autoencoder-based clustering, these models act as "sandboxes." By jointly optimizing the reconstruction loss (which preserves data details) and the clustering loss (which enforces structure), the model finds a middle ground.
When $K$ is allowed to be flexible, the clustering loss no longer "fights" the data. Instead of forcing a square peg into a round hole, the model can simply add more holes. This synergy allows the network to learn features that are both semantically meaningful and structurally organized, leading to the massive performance gains (sometimes up to 70% in NMI) seen in frameworks like Uniform Deep Graph Clustering.