Abstract
Multi-Task Multi-Label (MTML) learning faces significant challenges due to data sparsity and complex, non-uniform task correlations. Traditional methods often treat tasks independently or apply global grouping, failing to capture label-specific relationships. This paper introduces LABTAG (LABel-sensitive TAsk Grouping), a Bayesian nonparametric framework that automatically discovers task clusters on a per-label basis. By sharing knowledge across similar tasks for specific labels, LABTAG improves performance over state-of-the-art baselines. Experimental results on real-world datasets demonstrate its superior accuracy and ability to reveal hidden task structures.
Introduction
In many real-world scenarios, we encounter Multi-Task Multi-Label (MTML) learning problems. For instance, in a movie recommendation system, each user represents a "task," while the various contexts in which they might watch a movie (e.g., at home, on a weekend, with a spouse) serve as "labels."
This domain presents two primary challenges:
- Data Sparsity: Individual tasks (users) often have very few data points, making it difficult to train robust models in isolation.
- Complex Correlations: Tasks are often related, but these relationships are rarely uniform across all labels. Two users might share identical preferences for "weekend" movies but have completely divergent tastes for "weekday" movies.
Existing models typically treat tasks as independent or group them globally. LABTAG addresses these limitations through label-sensitive grouping, where task similarity is evaluated independently for every label, allowing for more nuanced and effective knowledge sharing.
Methodology
LABTAG is built on Bayesian nonparametric principles, designed to handle uncertainty and discover latent structures without pre-defined parameters. The core of the framework involves a two-level modeling approach:
- Level 1: Feature-Label Correlations: The model captures patterns of how specific features relate to specific labels.
- Level 2: Label-Sensitive Task Grouping: For every label $c$, the model clusters tasks that exhibit similar behavior for that specific label.
To manage the complexity of these relationships, the model utilizes Dirichlet Process (DP) Priors. This allows the system to "learn" the appropriate number of feature-label patterns and task clusters directly from the data, avoiding the need for manual hyperparameter tuning. Furthermore, the authors employ Variational Inference to approximate the posterior distribution, ensuring the learning process is computationally efficient and deterministic compared to traditional MCMC sampling.
Experimental Setup
The model was evaluated on three public datasets representing diverse MTML challenges:
- TripAdvisor: A hotel rating dataset where tasks are users and labels are trip types.
- LDOS-CoMoDa: A movie recommendation dataset with labels including time, location, and companion.
- Enron Email Corpus: A natural language processing dataset for email categorization.
These datasets vary significantly in scale, with tasks ranging from 10 to 46, labels from 5 to 58, and features from 93 to 460, providing a rigorous test for the model's robustness.
Key Results and Insights
LABTAG was compared against several strong baselines, including Binary Relevance (BR), ML-kNN, and Bayesian Nonparametric Multi-label Classification (BNMC).
Performance Metrics
LABTAG achieved the highest accuracy across all three datasets. Notably, on the Enron corpus, it reached 69.06% accuracy, outperforming the next best baseline (BR at 66.20%). It consistently led in five key evaluation metrics: Accuracy, F1-score, Hamming loss, One-error, and Rank loss.
Visualizing Task Clusters
The effectiveness of label-sensitive grouping is best illustrated through Hinton diagrams. These visualizations confirm that tasks are indeed grouped differently depending on the label, validating the core hypothesis that task similarity is not a global constant.
Conclusion and Future Work
LABTAG demonstrates that label-sensitive task grouping significantly enhances MTML performance, especially when training data is sparse. By "borrowing" knowledge from similar tasks within a group, it prevents the over-fitting common in single-task models.
Future research directions include reducing the computational overhead of complex matrix operations to better handle "Big Data" scales. Additionally, integrating this grouping logic with Deep Learning architectures and addressing the "cold-start" problem for new tasks remain promising areas for exploration.
Summary updated