Deep Adversarial Inconsistent Cognitive Sampling for Multi-view Progressive Subspace Clustering

Renhao Sun, Yang Wang, Zhao Zhang, Richang Hong, Meng Wang

Abstract

Deep multi-view clustering methods have achieved remarkable performance. However, all of them failed to consider the difficulty labels (uncertainty of ground-truth for training samples) over multi-view samples, which may result into a nonideal clustering network for getting stuck into poor local optima during training process; worse still, the difficulty labels from multi-view samples are always inconsistent, such fact makes it even more challenging to handle. In this paper, we propose a novel Deep Adversarial Inconsistent Cognitive Sampling (DAICS) method for multi-view progressive subspace clustering. A multiview binary classification (easy or difficult) loss and a feature similarity loss are proposed to jointly learn a binary classifier and a deep consistent feature embedding network, throughout an adversarial minimax game over difficulty labels of multiview consistent samples. We develop a multi-view cognitive sampling strategy to select the input samples from easy to difficult for multi-view clustering network training. However, the distributions of easy and difficult samples are mixed together, hence not trivial to achieve the goal. To resolve it, we define a sampling probability with theoretical guarantee. Based on that, a golden section mechanism is further designed to generate a sample set boundary to progressively select the samples with varied difficulty labels via a gate unit, which is utilized to jointly learn a multi-view common progressive subspace and clustering network for more efficient clustering. Experimental results on four real-world datasets demonstrate the superiority of DAICS over the state-of-the-art methods.

Introduction

In the realm of multi-view clustering, the objective is to group data points by leveraging information from diverse sources or "views"—for instance, combining color histograms with edge features in image analysis. While deep learning has significantly advanced this field, a critical oversight remains: the varying difficulty of training samples.

Real-world data is inherently noisy. Some samples are "easy" and clearly belong to a specific cluster, while others are "difficult" due to ambiguity or proximity to cluster boundaries. A significant challenge arises when a sample's difficulty is inconsistent across views—appearing easy in one view but difficult in another. Learning from these inconsistent, difficult samples prematurely can cause models to converge to poor local optima.

The DAICS Framework

To address these challenges, we introduce Deep Adversarial Inconsistent Cognitive Sampling (DAICS). This framework intelligently manages the training process by reconciling inconsistent difficulty labels and implementing a curriculum-based learning strategy.

The overall architecture of the DAICS framework, featuring the Adversarial Inconsistent Samples module, Cognitive Sampling, and the Multi-view Progressive Clustering Network.

Adversarial Inconsistent Samples (AIS) Module

The AIS module resolves the "identity crisis" of samples with conflicting difficulty labels across views. It employs an adversarial minimax game: a binary classifier attempts to distinguish between easy and difficult labels, while a feature network learns consistent representations that minimize this discriminability. This process forces the model to find a latent space where sample difficulty is perceived consistently across all views.

Cognitive Sampling (CS) Strategy

Inspired by human cognition, the CS strategy implements a "curriculum" that transitions from easy to difficult samples. The "easiness" of a sample is quantified based on its distance to its nearest neighbors and a randomly selected anchor object. This ensures the model establishes a stable foundation on certain data before tackling ambiguous cases.

Golden Section (GS) Mechanism

The GS mechanism acts as a gate unit to control the progression of training. It defines specific sampling regions to manage the transition from easy to difficult data points.

Illustration of the sampling regions and the Golden Section mechanism used to progressively select samples based on difficulty.
  • Initial Phase: The model focuses exclusively on "easy" samples to build robust individual view representations.
  • Progressive Phase: Once the sample count reaches the "Golden Section" ratio (approximately 0.618 of the total data), the gate opens. The model then begins learning a common progressive subspace, integrating both easy and difficult samples across all views.

Experimental Evaluation

DAICS was evaluated on four real-world datasets: Handwritten Numerals (HW), Caltech101-20, NUS-WIDE-OBJ, and MNIST.

The results demonstrate that DAICS consistently outperforms state-of-the-art methods such as DCCA, DAMC, and EAMC across key metrics including Accuracy, Normalized Mutual Information (NMI), and Purity. For instance, on the HW dataset, DAICS achieved a peak accuracy of 97.4%. On the large-scale MNIST dataset, it reached 69.8% accuracy, surpassing the next best deep model by over 3%.

Conclusion

DAICS introduces a robust approach to multi-view clustering by explicitly modeling sample difficulty and inconsistency. By combining adversarial learning with a progressive cognitive sampling strategy, the framework avoids local optima and achieves superior clustering performance. Future work will focus on reducing hyperparameter sensitivity and optimizing computational efficiency for even larger datasets.

Summary updated

Abstract

Deep multi-view clustering methods have achieved remarkable performance. However, all of them failed to consider the difficulty labels (uncertainty of ground-truth for training samples) over multi-view samples, which may result into a nonideal clustering network for getting stuck into poor local optima during training process; worse still, the difficulty labels from multi-view samples are always inconsistent, such fact makes it even more challenging to handle. In this paper, we propose a novel Deep Adversarial Inconsistent Cognitive Sampling (DAICS) method for multi-view progressive subspace clustering. A multiview binary classification (easy or difficult) loss and a feature similarity loss are proposed to jointly learn a binary classifier and a deep consistent feature embedding network, throughout an adversarial minimax game over difficulty labels of multiview consistent samples. We develop a multi-view cognitive sampling strategy to select the input samples from easy to difficult for multi-view clustering network training. However, the distributions of easy and difficult samples are mixed together, hence not trivial to achieve the goal. To resolve it, we define a sampling probability with theoretical guarantee. Based on that, a golden section mechanism is further designed to generate a sample set boundary to progressively select the samples with varied difficulty labels via a gate unit, which is utilized to jointly learn a multi-view common progressive subspace and clustering network for more efficient clustering. Experimental results on four real-world datasets demonstrate the superiority of DAICS over the state-of-the-art methods.
BibTeX
@article{sun2021deep,
  author = {Renhao Sun and Yang Wang and Zhao Zhang and Richang Hong and Meng Wang},
  title = {{Deep Adversarial Inconsistent Cognitive Sampling for Multi-view Progressive Subspace Clustering}},
  journal = {IEEE Transactions on Neural Networks and Learning Systems},
  year = {2021},
  doi = {10.1109/tnnls.2021.3093419},
}
Figure