Multi-Task Multi-View Clustering for Non-Negative Data

Xianchao Zhang, Xiaotong Zhang, Han Liu

Introduction

In modern machine learning, data complexity often manifests in two distinct dimensions: multiple sources of information (multi-view) and multiple related objectives (multi-task). For instance, a webpage can be represented by its text content, title, and hyperlinks (multi-view), while clustering pages across different university websites represents related but distinct tasks (multi-task). Traditionally, these challenges have been addressed in isolation. This paper introduces MTMVC, a unified framework designed to solve both multi-view and multi-task clustering simultaneously, specifically optimized for non-negative data.

The Multi-Task Multi-View Challenge

Most existing clustering algorithms operate on a single dataset from a single perspective. However, real-world data is frequently:

  1. Multi-View: Objects are described by multiple feature sets (e.g., video audio vs. pixels).
  2. Multi-Task: Related datasets exist where knowledge from one can inform another (e.g., spam detection across different users).

While multi-view algorithms focus on consistency between views, and multi-task algorithms focus on knowledge transfer between tasks, neither typically accounts for both. MTMVC fills this gap, particularly for non-negative data like word counts, where traditional geometric methods often struggle.

The MTMVC Framework

The Multi-Task Multi-View Clustering (MTMVC) framework integrates three core components into a single objective function:

  1. Within-View-Task Clustering: Utilizing Bipartite Graph Co-clustering (BiCo), the model simultaneously clusters samples (documents) and features (words). This two-way relationship is crucial for high-dimensional text data.
  2. Multi-View Relationship Learning: This ensures consistency across different views of the same task, minimizing disagreement between representations.
  3. Multi-Task Relationship Learning: This component facilitates knowledge transfer by learning a shared feature subspace across tasks.
Conceptual framework of MTMVC showing shared features across tasks and views.

The optimization of this complex objective function is achieved through gradient ascent on a Stiefel manifold, ensuring the algorithm adheres to necessary mathematical constraints while searching for optimal cluster assignments.

Experimental Evaluation

The researchers evaluated MTMVC against several baselines using three real-world text datasets:

  • WebKB: Webpages from four universities (4 tasks) with three views: text, hyperlinks, and titles.
  • 20NewsGroups (NG): A text dataset organized into 4 tasks, distinguishing between common and task-specific vocabulary.
  • Email: Spam detection for 3 users (3 tasks) using common and user-specific word views.

Results and Discussion

MTMVC consistently outperformed standard clustering methods (k-means, spectral clustering) and specialized multi-view or multi-task algorithms.

Clustering performance (Accuracy and NMI) on the WebKB dataset.

Key findings include:

  • Accuracy Gains: On the WebKB dataset, MTMVC reached accuracies of approximately 83-87% across tasks. In comparison, standard k-means hovered around 52-61%, and even specialized multi-view algorithms (like CoRe) only reached about 67-75%.
  • Mutual Boosting: The integration of multi-view and multi-task information creates a synergistic effect. Improved clustering in one view aids the multi-task learner in finding a better shared subspace, which subsequently benefits other views.
  • Robustness: Even when the most informative view is unknown, the MTMVC-CV variant (concatenating views) maintains superior performance over baseline methods.

Conclusion

MTMVC represents a significant advancement in clustering by bridging the gap between multi-view and multi-task learning for non-negative data. By leveraging bipartite graph co-clustering and manifold optimization, it achieves state-of-the-art results on complex text datasets. Future work aims to extend this framework to general data types beyond non-negative matrices, broadening its applicability across diverse scientific domains.

Summary updated

Abstract

Multi-task clustering and multi-view clustering have severally found wide applications and received much attention in recent years. Nevertheless, there are many clustering problems that involve both multi-task clustering and multiview clustering, i.e., the tasks are closely related and each task can be analyzed from multiple views. In this paper, for non-negative data (e.g., documents), we introduce a multi-task multi-view clustering (MTMVC) framework which integrates within-view-task clustering, multi-view relationship learning and multi-task relationship learning. We then propose a specific algorithm to optimize the MT-MVC framework. Experimental results show the superiority of the proposed algorithm over either multi-task clustering algorithms or multi-view clustering algorithms for multi-task clustering of multiview data.
BibTeX
@inproceedings{zhang2015multi,
  author = {Xianchao Zhang and Xiaotong Zhang and Han Liu},
  title = {{Multi-Task Multi-View Clustering for Non-Negative Data}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure