Introduction
In modern machine learning, data complexity often manifests in two distinct dimensions: multiple sources of information (multi-view) and multiple related objectives (multi-task). For instance, a webpage can be represented by its text content, title, and hyperlinks (multi-view), while clustering pages across different university websites represents related but distinct tasks (multi-task). Traditionally, these challenges have been addressed in isolation. This paper introduces MTMVC, a unified framework designed to solve both multi-view and multi-task clustering simultaneously, specifically optimized for non-negative data.
The Multi-Task Multi-View Challenge
Most existing clustering algorithms operate on a single dataset from a single perspective. However, real-world data is frequently:
- Multi-View: Objects are described by multiple feature sets (e.g., video audio vs. pixels).
- Multi-Task: Related datasets exist where knowledge from one can inform another (e.g., spam detection across different users).
While multi-view algorithms focus on consistency between views, and multi-task algorithms focus on knowledge transfer between tasks, neither typically accounts for both. MTMVC fills this gap, particularly for non-negative data like word counts, where traditional geometric methods often struggle.
The MTMVC Framework
The Multi-Task Multi-View Clustering (MTMVC) framework integrates three core components into a single objective function:
- Within-View-Task Clustering: Utilizing Bipartite Graph Co-clustering (BiCo), the model simultaneously clusters samples (documents) and features (words). This two-way relationship is crucial for high-dimensional text data.
- Multi-View Relationship Learning: This ensures consistency across different views of the same task, minimizing disagreement between representations.
- Multi-Task Relationship Learning: This component facilitates knowledge transfer by learning a shared feature subspace across tasks.
The optimization of this complex objective function is achieved through gradient ascent on a Stiefel manifold, ensuring the algorithm adheres to necessary mathematical constraints while searching for optimal cluster assignments.
Experimental Evaluation
The researchers evaluated MTMVC against several baselines using three real-world text datasets:
- WebKB: Webpages from four universities (4 tasks) with three views: text, hyperlinks, and titles.
- 20NewsGroups (NG): A text dataset organized into 4 tasks, distinguishing between common and task-specific vocabulary.
- Email: Spam detection for 3 users (3 tasks) using common and user-specific word views.
Results and Discussion
MTMVC consistently outperformed standard clustering methods (k-means, spectral clustering) and specialized multi-view or multi-task algorithms.
Key findings include:
- Accuracy Gains: On the WebKB dataset, MTMVC reached accuracies of approximately 83-87% across tasks. In comparison, standard k-means hovered around 52-61%, and even specialized multi-view algorithms (like CoRe) only reached about 67-75%.
- Mutual Boosting: The integration of multi-view and multi-task information creates a synergistic effect. Improved clustering in one view aids the multi-task learner in finding a better shared subspace, which subsequently benefits other views.
- Robustness: Even when the most informative view is unknown, the MTMVC-CV variant (concatenating views) maintains superior performance over baseline methods.
Conclusion
MTMVC represents a significant advancement in clustering by bridging the gap between multi-view and multi-task learning for non-negative data. By leveraging bipartite graph co-clustering and manifold optimization, it achieves state-of-the-art results on complex text datasets. Future work aims to extend this framework to general data types beyond non-negative matrices, broadening its applicability across diverse scientific domains.
Summary updated