Discriminatively Fuzzy Multi-View K-means Clustering with Local Structure Preserving

Jun Yin, Shiliang Sun, Lai Wei, Pei Wang

Abstract

Multi-view clustering aims to integrate information from diverse sources to improve data partitioning. Traditional Multi-view K-means (MVKM) often neglects cluster separability and the local geometric structure of data. We present DFMKLS (Discriminatively Fuzzy Multi-View K-means Clustering with Local Structure Preserving), a novel framework that optimizes for cluster compactness, separability, and local data geometry simultaneously. By incorporating fuzzy logic and manifold learning, DFMKLS achieves superior performance on benchmark datasets compared to state-of-the-art methods.

Introduction

Clustering multi-view data—where objects are described by multiple feature sets (e.g., images and their textual tags)—is a critical task in unsupervised learning. Standard MVKM approaches typically focus on global compactness, minimizing within-cluster variance. However, they often fail to ensure cluster separability, leading to overlapping clusters, and ignore local relationships where neighboring points in a specific view should likely share cluster assignments.

Methodology

The DFMKLS framework addresses these limitations through three core components:

  1. Fuzzy Multi-View K-means: Unlike hard assignments, fuzzy clustering provides a probabilistic membership for each data point, effectively handling data ambiguity.
  2. Discriminative Property: The objective function explicitly maximizes the distance between pairwise cluster centers. This "push-pull" mechanism pulls points toward their centers while pushing centers apart to enhance separability.
  3. Local Structure Preservation: By constructing a K-nearest-neighbor (KNN) graph for each view, the model incorporates manifold learning. A penalty is applied if neighboring points in the feature space are assigned to different clusters.

The resulting non-convex optimization problem is solved using an efficient iterative update algorithm with a multiplicative update rule.

Experimental Evaluation

The model was tested on four benchmark datasets: 3Sources (news stories), BBC (segmented documents), WebKB (university webpages), and NUS_WIDE (web images).

Parameter Sensitivity

The performance of DFMKLS is influenced by the number of neighbors $K$ and the trade-off parameter $\alpha$. As shown in Figure 1, the algorithm demonstrates relative stability across a range of $K$ values for various datasets, though optimal performance typically occurs around $K=10$.

Performance metrics (NMI, F-score, Precision, Accuracy) as a function of the number of neighbors K across four benchmark datasets.

Convergence Analysis

Efficiency is a key advantage of the proposed iterative solver. Empirical results indicate that the objective value stabilizes quickly, typically within 100 iterations, making it computationally viable for medium-to-large datasets.

Convergence curves showing the objective value decreasing over iterations for the 3Sources, BBC, WebKB, and NUS_WIDE datasets.

Key Results and Discussion

DFMKLS consistently outperformed 12 state-of-the-art methods (including RMKMC, GMC, and standard Multi-view K-means). Notable highlights include:

  • BBC Dataset: Achieved an accuracy of 0.8672, significantly higher than the second-best method (0.8068) and standard multi-view K-means (0.5547).
  • Scalability: On the large-scale NUS_WIDE dataset, it maintained superior accuracy (0.2453) and NMI, outperforming graph-based alternatives.
  • Robustness: The integration of fuzzy logic and discriminative terms allows for clearer cluster boundaries in high-dimensional spaces.

Conclusion and Future Work

DFMKLS provides a robust framework for multi-view clustering by balancing global and local data structures. While the $O(Cn^2)$ complexity is manageable for medium datasets, future work will explore anchor-based methods to scale to "Big Data." Additionally, automated hyper-parameter tuning and handling incomplete multi-view data remain open areas for investigation.

Summary updated

Abstract

Multi-view K-means clustering successfully generalizes K-means from single-view to multi-view, and obtains excellent clustering performance. In every view, it makes each data point close to the center of the corresponding cluster. However, multi-view K-means only considers the compactness of each cluster, but ignores the separability of different clusters, which is of great importance to producing a good clustering result. In this paper, we propose Discriminatively Fuzzy Multi-view K-means clustering with Local Structure Preserving (DFMKLS). On the basis of minimizing the distance between each data point and the center of the corresponding cluster, DFMKLS separates clusters by maximizing the distance between the centers of pairwise clusters. DFMKLS also relaxes its objective by introducing the idea of fuzzy clustering, which calculates the probability that a data point belongs to each cluster. Considering multi-view K-means mainly focuses on the global information of the data, to efficiently use the local information, we integrate the local structure preserving into the framework of DFMKLS. The effectiveness of DFMKLS is evaluated on benchmark multi-view datasets. It obtains superior performances than state-of-the-art multi-view clustering methods, including multi-view K-means.
BibTeX
@inproceedings{yin2024discriminatively,
  author = {Jun Yin and Shiliang Sun and Lai Wei and Pei Wang},
  title = {{Discriminatively Fuzzy Multi-View K-means Clustering with Local Structure Preserving}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i15.29585},
}
Figure