Seeking Commonness and Inconsistencies: A Jointly Smoothed Approach to Multi-view Subspace Clustering

Xiaosha Cai, Dong Huang, Guang-Yu Zhang, Chang-Dong Wang

Abstract: Multi-view subspace clustering aims to discover the hidden subspace structures from multiple views for robust clustering, and has been attracting considerable attention in recent years. Despite significant progress, most of the previous multi-view subspace clustering algorithms are still faced with two limitations. First, they usually focus on the consistency (or commonness) of multiple views, yet often lack the ability to capture the cross-view inconsistencies in subspace representations. Second, many of them overlook the local structures of multiple views and cannot jointly leverage multiple local structures to enhance the subspace representation learning. To address these two limitations, in this paper, we propose a jointly smoothed multi-view subspace clustering (JSMC) approach. Specifically, we simultaneously incorporate the cross-view commonness and inconsistencies into the subspace representation learning. The view-consensus grouping effect is presented to jointly exploit the local structures of multiple views to regularize the view-commonness representation, which is further associated with the low-rank constraint via the nuclear norm to strengthen its cluster structure. Thus the cross-view commonness and inconsistencies, the view-consensus grouping effect, and the low-rank representation are seamlessly incorporated into a unified objective function, upon which an alternating optimization algorithm is performed to achieve a robust subspace representation for clustering. Experimental results on a variety of real-world multi-view datasets confirm the superiority of our approach. Code available: https://github.com/huangdonghere/JSMC.

Introduction

In the era of big data, information is often "multi-view" in nature—the same object or event is described through different modalities or sources. For instance, a news story might be reported in multiple languages, or a webpage might contain a mix of text, images, and video. Multi-view subspace clustering aims to find the underlying structure of such data by assuming it lies in a union of low-dimensional subspaces.

However, existing algorithms frequently encounter two major hurdles:

  1. The Consistency Trap: Most methods focus exclusively on finding what is consistent across views while ignoring inconsistencies—the unique characteristics or noise specific to a single view. Neglecting these differences can degrade the final clustering quality.
  2. Overlooking Local Structures: Many methods fail to account for the local geometry of the data. If two data points are "neighbors" across several views, they should ideally remain close in the final representation. Standard methods often overlook this "grouping effect."

The JSMC Framework

To address these challenges, the researchers introduced Jointly Smoothed Multi-view Subspace Clustering (JSMC). The core innovation of JSMC lies in its ability to simultaneously model shared information and view-specific nuances while preserving the local data geometry.

Overview of the Jointly Smoothed Multi-view Subspace Clustering (JSMC) framework.

1. Commonness vs. Inconsistency

Instead of seeking a single uniform representation, JSMC decomposes the subspace representation of each view ($Z^{(v)}$) into two distinct components:

  • $C$ (The Commonness Matrix): A shared representation capturing the core structure present across all views.
  • $E^{(v)}$ (The Inconsistency Matrix): A view-specific component that captures noise or unique characteristics of that particular view.

By explicitly modeling $E^{(v)}$, the algorithm effectively "cleans" the common representation $C$, making the resulting cluster structure significantly more robust.

2. View-Consensus Grouping Effect

To preserve local structures, JSMC utilizes K-Nearest Neighbor (K-NN) graphs from every view. It enforces a "jointly smoothed" regularization: if two points are neighbors in the original views, they are encouraged to remain close in the common representation $C$. This ensures that the local geometry of the data is leveraged to enhance the subspace learning process.

3. Unified Optimization

The framework integrates these ideas into a single objective function that balances self-expressive loss, low-rank constraints (via the nuclear norm), smoothness (the grouping effect), and sparsity (to handle inconsistencies). The authors employ an alternating minimization strategy, utilizing the Sylvester equation to solve the resulting complex matrix problems efficiently.

Experimental Evaluation

JSMC was evaluated against seven state-of-the-art multi-view clustering methods (such as RMSC and MVGL) across eight diverse real-world datasets, including news articles (3Sources), facial images (Notting-Hill, ORL, Yale), and object images (COIL-20, Caltech-7).

Key Results

  • Superior Performance: JSMC consistently outperformed competitors. On the Notting-Hill dataset, it achieved a Normalized Mutual Information (NMI) of 96.92%, compared to the next best method's 85.51%.
  • Exceptional Ranking: Across all datasets and metrics (NMI, ARI, Accuracy, and Purity), JSMC achieved an average rank of 1.13, indicating it was the top performer in nearly every test case.
  • Practical Convergence: Despite its mathematical complexity, the algorithm typically converges within 30 iterations, making it viable for practical applications.

Discussion and Insights

  • Commonness vs. Consistency: While "consistency" usually implies that representations should be similar, "commonness" enforces an identical core. This stricter requirement, when paired with an explicit model for inconsistencies, leads to superior results.
  • Inconsistency as Identity: The paper posits that inconsistencies are not merely random noise but represent "view-specific characteristics." Separating them allows the model to acknowledge each view's unique identity without letting it interfere with the shared structure.
  • The Power of the Neighborhood: The "view-consensus grouping effect" acts as a collective intelligence mechanism. By aggregating neighbor information from all views, JSMC ensures that clusters are formed based on the most reliable local relationships.

Conclusion and Future Directions

JSMC represents a significant step forward in multi-view subspace clustering by elegantly balancing commonality and inconsistency. While highly effective, the current approach relies on $O(n^3)$ operations like SVD, which limits its scalability for massive datasets. Future work may explore bipartite graph formulations to achieve "ultra-scalable" performance or extend the method into an ensemble clustering framework for even greater robustness.

Summary updated

Abstract

Multi-view subspace clustering aims to discover the hidden subspace structures from multiple views for robust clustering, and has been attracting considerable attention in recent years. Despite significant progress, most of the previous multi-view subspace clustering algorithms are still faced with two limitations. First, they usually focus on the consistency (or commonness) of multiple views, yet often lack the ability to capture the cross-view inconsistencies in subspace representations. Second, many of them overlook the local structures of multiple views and cannot jointly leverage multiple local structures to enhance the subspace representation learning. To address these two limitations, in this paper, we propose a jointly smoothed multi-view subspace clustering (JSMC) approach. Specifically, we simultaneously incorporate the cross-view commonness and inconsistencies into the subspace representation learning. The view-consensus grouping effect is presented to jointly exploit the local structures of multiple views to regularize the view-commonness representation, which is further associated with the low-rank constraint via the nuclear norm to strengthen its cluster structure. Thus the cross-view commonness and inconsistencies, the view-consensus grouping effect, and the low-rank representation are seamlessly incorporated into a unified objective function, upon which an alternating optimization algorithm is performed to achieve a robust subspace representation for clustering. Experimental results on a variety of real-world multi-view datasets confirm the superiority of our approach. Code available: https://github.com/huangdonghere/JSMC.
BibTeX
@article{cai2023seeking,
  author = {Xiaosha Cai and Dong Huang and Guang-Yu Zhang and Chang-Dong Wang},
  title = {{Seeking Commonness and Inconsistencies: A Jointly Smoothed Approach to Multi-view Subspace Clustering}},
  journal = {Information Fusion},
  year = {2023},
  doi = {10.1016/j.inffus.2022.10.020},
}
Figure