Adaptive Regularized Semi-Supervised Clustering Ensemble

Rui Luo, Zhiwen Yu, Wenming Cao, Cheng Liu, Hau-San Wong, C. L. Philip Chen

Clustering is a fundamental unsupervised learning task used to group data points without pre-existing labels. However, standard clustering algorithms often struggle with high-dimensional data—such as gene expression profiles—where the "curse of dimensionality" makes it difficult to distinguish relevant features from noise. Furthermore, clustering results can be highly sensitive to small parameter adjustments.

To address these challenges, researchers have developed semi-supervised clustering, which incorporates a small amount of prior knowledge in the form of pairwise constraints: must-link (points that should be grouped together) and cannot-link (points that should not). This paper introduces ARSCE (Adaptive Regularized Semi-Supervised Clustering Ensemble), a framework designed to optimize the use of these constraints while filtering out uninformative data subspaces.

The ARSCE Framework

ARSCE operates through a sophisticated three-stage pipeline designed to maximize diversity and accuracy within the clustering ensemble.

The three-stage ARSCE framework: subspace generation, constraint-driven projection, and adaptive fusion.
  1. Diverse Subspace Generation: The algorithm generates multiple "views" of the dataset by randomly sampling subsets of features without replacement. This ensures that different ensemble members focus on distinct aspects of the data, reducing redundancy.
  2. Constraint-Driven Projection: For each generated subspace, ARSCE performs a mathematical transformation to create a "clustering-friendly" environment. It optimizes an objective function that minimizes the distance between must-link points and maximizes the distance between cannot-link points, effectively reshaping the feature space to align with human-provided hints.
  3. Adaptive Fusion via Diffusion: Rather than using a simple majority vote, ARSCE employs a "fusion of diffusion" strategy. It calculates similarity matrices for each ensemble member and merges them into a unified similarity graph. Crucially, the model adaptively weights each member, automatically downplaying noisy or low-quality subspaces while emphasizing the most informative ones.

Experimental Evaluation

The authors evaluated ARSCE on 15 real-world benchmark datasets, primarily focusing on high-dimensional biological and image data. These included 13 datasets from the GENE and ASU repositories (e.g., Alizadeh-2000-v3, Lymphoma, lung_discrete) and image recognition datasets like the Yale face database.

Key Insights and Performance

A critical finding of the study was the existence of a "Goldilocks" sampling rate. As shown in the performance metrics for the lung_discrete dataset, increasing the number of features in a subspace improves performance up to a specific point, after which additional features introduce noise that degrades the clustering quality.

Impact of sampling rate on NMI for the lung_discrete dataset, showing the optimal 'Goldilocks' zone.

ARSCE demonstrated superior accuracy compared to seven other state-of-the-art ensemble methods (such as NGCE and RSKE). On the Alizadeh-2000-v3 dataset, ARSCE achieved a Normalized Mutual Information (NMI) score of 0.793, significantly outperforming the next best method, ISSCE, which scored 0.721. The results also confirmed that ARSCE becomes increasingly effective as the percentage of available pairwise constraints grows, proving its efficiency in leveraging limited supervised information.

Conclusion and Future Directions

ARSCE provides a robust solution for high-dimensional clustering by treating pairwise constraints as a "compass" to transform feature spaces rather than just a post-processing check. While the current model relies on linear transformations and manual parameter tuning, future work will explore the integration of Deep Neural Networks to learn non-linear feature representations and improve scalability for massive computer vision datasets.

Summary updated

Abstract

Although semi-supervised clustering ensemble methods have achieved satisfactory performance, they fail to effectively utilize the constrained knowledge such as cannot-link and must-link when generating diverse ensemble members. In addition, they ignore negative effects brought about by redundancies and noisy data. To address the above shortcomings, in this paper we propose an approach to combine multiple semi-supervised clustering solutions via adaptively regularizing the weights of clustering ensemble members, which is referred to as ARSCE. First, we generate a series of feature subspaces by randomly selecting feature without replacement to avoid the scenario where there are two identical feature subspaces. Second, we conduct feature transformation on the above obtained feature subspaces while considering the pairwise constraints to find new clustering-friendly spaces, where clustering methods are exploited to generate various clustering solutions. Finally, we design a novel fusion strategy to integrate multiple clustering solutions into a unified clustering partition, where weights are designated for each clustering ensemble member. Extensive experiments are conducted on multiple real-world benchmarks, and experimental results demonstrate the effectiveness and superiority of our proposed method ARSCE over other counterparts.
BibTeX
@article{luo2020adaptive,
  author = {Rui Luo and Zhiwen Yu and Wenming Cao and Cheng Liu and Hau-San Wong and C. L. Philip Chen},
  title = {{Adaptive Regularized Semi-Supervised Clustering Ensemble}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2019.2963306},
}
Figure