In supervised learning, ensembling is straightforward: if three models predict "cat," the consensus is "cat." However, in unsupervised learning, models do not share a common vocabulary. One clustering model might group images of cats under "Cluster 1," while another model groups them under "Cluster 5." Because these identifiers are arbitrary, simply averaging the outputs of multiple unsupervised models results in a meaningless "label soup."
To harness the power of ensembles for unsupervised tasks, researchers have developed methods to resolve this label correspondence problem, allowing multiple independent models to be synchronized into a single, high-performance consensus.
The Label Switching Problem
The fundamental hurdle in aggregating unsupervised models is the label-switching problem. As noted by Kevin Gimpel and Mohit Bansal in Weakly-Supervised Learning with Cost-Augmented Contrastive Estimation, different training runs assign different arbitrary identifiers to the same underlying latent categories.
Imagine two models, $M_A$ and $M_B$, both trying to partition a dataset into three groups. Even if both models perfectly identify the same three clusters, $M_A$ might label them $\{0, 1, 2\}$ while $M_B$ labels them $\{2, 0, 1\}$. Without a way to map $M_A$'s "0" to $M_B$'s "2," we cannot perform ensemble voting or aggregate their insights. This lack of a universal mapping makes it difficult to determine if two models are actually referring to the same underlying concept.
Alignment via Bipartite Matching
To solve this, researchers use bipartite label alignment. The goal is to find a permutation of labels that maximizes the agreement between two models. Kevin Gimpel and Mohit Bansal demonstrate that by using maximum-weight bipartite matching, one can synchronize labels across different runs.
This process treats the labels from two models as nodes in a bipartite graph, where the weight of an edge between "Label $i$" in Model A and "Label $j$" in Model B represents how many data points they share. By solving for the optimal matching, we can "re-index" all models to use a shared set of identifiers, enabling effective ensemble voting.
From Alignment to Consensus
Once labels are aligned, the next challenge is finding the "best" shared representation. Fei Wang, Tao Li, and their colleagues proposed a unified framework for this in Generalized Cluster Aggregation.
Instead of just voting, they frame aggregation as an optimization problem using Bregman divergences. This approach allows the system to:
- Weight models differently: If one input clustering is more consistent or reliable, the framework can automatically give it more influence.
- Incorporate prior knowledge: It can integrate semi-supervised constraints (e.g., "these two points must be in the same cluster") directly into the aggregation process.
Iterative Refinement and Pseudo-Labels
Modern approaches have moved beyond aggregating static models toward iterative "pseudo-semi-supervised" learning. In the Kingdra framework, Divam Gupta and colleagues show how to use an ensemble of deep networks to autonomously generate high-precision "pseudo-labels."
As detailed in Unsupervised Clustering using Pseudo-semi-supervised Learning, the system identifies data points where the ensemble has high confidence and high agreement. These points are treated as "ground truth" to train the models in a semi-supervised fashion. By resolving the correspondence problem across the ensemble, Kingdra can iteratively bootstrap its way to higher accuracy, outperforming single-model unsupervised methods by significant margins on both image and text benchmarks.
By treating the arbitrary nature of unsupervised labels as a synchronization problem rather than a fundamental barrier, this research direction enables the same "wisdom of the crowd" benefits that have long been standard in supervised machine learning.