Seeking commonness and inconsistencies: A jointly smoothed approach to multi-view subspace clustering

Xiaosha Cai, Dong Huang, Guang-Yu Zhang, Chang-Dong Wang

TL;DR

Researchers have developed a new multi-view subspace clustering approach called JSMC that simultaneously captures what different data sources have in common and where they disagree. By leveraging a "view-consensus grouping effect" to preserve local data structures across all views, the method achieves state-of-the-art performance on diverse real-world datasets, from facial recognition to news categorization.

What Problem Does This Paper Address?

In the world of big data, we often have "multi-view" information—the same object described in different ways. Think of a news story reported in English, French, and Spanish, or a webpage containing text, images, and video. Multi-view subspace clustering aims to find the underlying structure of this data by assuming it lies in a union of low-dimensional subspaces.

However, existing algorithms usually trip over two hurdles:

  1. The Consistency Trap: Most methods obsess over finding what is consistent across views but ignore the inconsistencies (the unique "flavor" or noise specific to one view). Ignoring these differences can muddy the final clustering result.
  2. Ignoring the Neighborhood: Many methods fail to account for local structures. If two data points are "neighbors" in several views, they should likely stay close in the final representation. Standard methods often overlook this "grouping effect."

Methods

The authors introduce Jointly Smoothed Multi-view Subspace Clustering (JSMC). Its core innovation lies in how it represents the data.

1. Commonness vs. Inconsistency

Instead of just looking for a single representation, JSMC decomposes the subspace representation of each view ($Z^{(v)}$) into two parts:

  • $C$ (The Commonness Matrix): A single, shared representation that captures the core structure present in all views.
  • $E^{(v)}$ (The Inconsistency Matrix): A view-specific part that captures noise or unique characteristics of that particular view.

By explicitly modeling $E^{(v)}$, the algorithm "cleans" the common representation $C$, making it much more robust.

2. View-Consensus Grouping Effect

To handle local structures, the authors use K-Nearest Neighbor (K-NN) graphs from every view. They enforce a "jointly smoothed" regularization: if two points are close in the original views, they must be close in the common representation $C$. This ensures that the local geometry of the data is preserved during the learning process.

3. The Unified Objective

The authors wrap these ideas into a single optimization problem that balances:

  • Self-expressive loss (how well the representation reconstructs the data).
  • Low-rank constraints (using the nuclear norm to ensure the cluster structure is clear).
  • Smoothness (the grouping effect).
  • Sparsity (handling the inconsistencies).

To solve this complex math, they use an alternating minimization strategy, which involves solving a Sylvester equation—a standard but powerful tool in control theory and matrix analysis.

Datasets

The authors evaluated JSMC on eight diverse, real-world benchmark datasets:

  • 3Sources: News articles from BBC, Reuters, and Guardian.
  • Notting-Hill: Face images from the movie Notting Hill.
  • ORL & Yale: Standard facial recognition datasets.
  • WebKB-Texas: University webpage data (citations and content).
  • Reuters: Documents in five different languages.
  • COIL-20: Images of objects taken from different angles.
  • Caltech-7: A subset of the famous Caltech-101 image dataset.

Key Insights and Ideas

  • Commonness is Stricter than Consistency: While "consistency" usually means representations should be similar, "commonness" means they share an identical core. This stricter requirement, paradoxically, leads to better results when paired with an explicit model for inconsistencies.
  • Inconsistency is Not Just Noise: The paper argues that inconsistencies can be "view-specific characteristics." By separating them, the model doesn't just discard them as random noise but acknowledges them as part of the view's unique identity.
  • The Power of the Neighborhood: The "view-consensus grouping effect" acts like a social network for data. If View A says points 1 and 2 are friends, and View B says points 2 and 3 are friends, JSMC uses that collective knowledge to keep the whole group together in the final clustering.

Key Results

JSMC consistently outperformed seven other state-of-the-art multi-view clustering methods (like RMSC and MVGL).

  • Accuracy and NMI: On the Notting-Hill dataset, JSMC achieved an NMI (Normalized Mutual Information) of 96.92%, while the next best method only reached 85.51%.
  • Ranking: Across all eight datasets and four different metrics (NMI, ARI, Accuracy, and Purity), JSMC achieved an average rank of 1.13. In the competitive world of ML benchmarks, a rank so close to 1.0 is exceptional.
  • Convergence: Despite the complex objective function, the algorithm typically converges within 30 iterations, making it computationally practical for medium-sized datasets.

Limitations and Future Work

  • Scalability: The current approach relies on Singular Value Decomposition (SVD) and solving Sylvester equations, which have a computational complexity of $O(n^3)$. This makes it difficult to apply to massive datasets with millions of samples.
  • Graph Formulation: The authors suggest that moving from a general graph to a bipartite graph formulation could be a path toward making the algorithm "ultra-scalable" for large-scale applications.
  • Ensemble Potential: The researchers also noted that this approach could be extended into an ensemble clustering framework, where multiple clustering results are combined for even greater robustness.

Summary updated

Abstract

Multi-view subspace clustering aims to discover the hidden subspace structures from multiple views for robust clustering, and has been attracting considerable attention in recent years. Despite significant progress, most of the previous multi-view subspace clustering algorithms are still faced with two limitations. First, they usually focus on the consistency (or commonness) of multiple views, yet often lack the ability to capture the cross-view inconsistencies in subspace representations. Second, many of them overlook the local structures of multiple views and cannot jointly leverage multiple local structures to enhance the subspace representation learning. To address these two limitations, in this paper, we propose a jointly smoothed multi-view subspace clustering (JSMC) approach. Specifically, we simultaneously incorporate the cross-view commonness and inconsistencies into the subspace representation learning. The view-consensus grouping effect is presented to jointly exploit the local structures of multiple views to regularize the view-commonness representation, which is further associated with the low-rank constraint via the nuclear norm to strengthen its cluster structure. Thus the cross-view commonness and inconsistencies, the view-consensus grouping effect, and the low-rank representation are seamlessly incorporated into a unified objective function, upon which an alternating optimization algorithm is performed to achieve a robust subspace representation for clustering. Experimental results on a variety of real-world multi-view datasets confirm the superiority of our approach. Code available: https://github.com/huangdonghere/JSMC.
BibTeX
@article{cai2023seeking,
  author = {Xiaosha Cai and Dong Huang and Guang-Yu Zhang and Chang-Dong Wang},
  title = {{Seeking commonness and inconsistencies: A jointly smoothed approach to multi-view subspace clustering}},
  journal = {Information Fusion},
  year = {2023},
  doi = {10.1016/j.inffus.2022.10.020},
}

Loading related content...

Figure