Bayesian nonparametric (BNP) models, such as the Dirichlet Process Mixture Model (DPMM), are often celebrated for their ability to "let the data speak." Unlike K-means or finite mixture models, they do not require a pre-specified number of clusters. Instead, they use a flexible prior—like the Chinese Restaurant Process—to theoretically infer the correct number of latent components as data arrives.
However, a growing body of research has revealed a fundamental flaw: these models are mathematically inconsistent for discovering the true number of clusters. As you collect more data, instead of settling on a stable count, these models tend to "over-cluster," spawning a long tail of tiny, redundant, and spurious components.
The Gap Between Density and Discovery
The core of the problem lies in a misunderstanding of what these models optimize for. As Jeffrey W. Miller and Matthew T. Harrison demonstrate in A simple example of Dirichlet process mixture inconsistency for the number of components, a model can be excellent at density estimation (fitting the overall shape of the data) while being a failure at clustering (identifying the discrete parts).
In a DPMM, the model prioritizes a perfect fit of the distribution. If the true data-generating distribution is even slightly different from the assumed cluster shape (e.g., using Gaussians to fit data with heavier tails), the model will not simply accept a "good enough" fit. Instead, it will use its nonparametric flexibility to "patch" the discrepancies by adding more and more tiny clusters. Diana Cai, Trevor Campbell, and Tamara Broderick prove that under such model misspecification, the estimated number of clusters will actually diverge to infinity as the sample size grows.
Why More Data Makes It Worse
Intuitively, one might expect that more data would help a model "correct" its mistakes. In BNP clustering, the opposite is often true. Chiao-Yu Yang, Nhat Ho, and Michael I. Jordan show that in Posterior Distribution for the Number of Clusters in Dirichlet Process Mixture Models, the posterior probability ratio for adding new clusters increases without bound as the sample size $N$ increases.
This happens because the standard priors (like the Dirichlet Process) do not penalize the creation of new clusters aggressively enough relative to the amount of new information. The model interprets the "noise" or "outliers" in a large dataset not as anomalies, but as evidence for the existence of very small, rare subpopulations. This leads to what Avi Matza and Yuval Bistritz describe as a "self-reinforcing feedback loop" where the model becomes increasingly biased toward complexity.
Mechanisms for Parsimony
To combat this "hallucination" of clusters, researchers have proposed several mechanisms to enforce parsimony—the idea that a simpler model with fewer clusters is preferable.
1. Repulsive Priors
Standard models assume cluster parameters are independent, which allows them to overlap and split a single natural group. Francesca Petralia, Vinayak A. Rao, and David B. Dunson introduced repulsive priors in Repulsive Mixtures. By mathematically pushing cluster centers away from each other, these priors prevent redundancy and force the model to consolidate data into fewer, more distinct components. Fangzheng Xie and Yanxun Xu further extended this using Determinantal Point Processes to ensure clusters are biologically or statistically interpretable.
2. Powering the Partition
Another approach is to modify the growth rate of the clusters. Jun Lu proposed the powered Chinese Restaurant Process (pCRP), which raises the size of existing clusters to a power $r > 1$. This simple tweak effectively increases the "gravity" of large clusters, making it much harder for a new, tiny cluster to survive unless it is supported by a significant amount of data.
3. Post-Processing and Pruning
Recognizing that the over-clustering might be baked into the Bayesian machinery, some researchers focus on cleaning the results after inference. Julyan Arbel and colleagues have championed the Merge-Truncate-Merge (MTM) algorithm. As detailed in Bayesian mixture models (in)consistency for the number of clusters, MTM acts as a "pruning" step that merges redundant components and removes low-weight clusters that are likely artifacts of the prior's inconsistency.
Summary of the Direction
The shift in the field is away from the naive assumption that "infinite" models automatically find the "truth." Instead, the focus has moved toward identifying why these models fail to converge on a finite number of components and developing structural fixes—such as repulsive interactions, modified growth processes, and robust post-processing—to ensure that the discovered clusters correspond to meaningful, stable patterns in the real world.