Label-Sensitive Task Grouping by Bayesian Nonparametric Approach for Multi-Task Multi-Label Learning

Xiao Zhang, Wenzhong Li, Vu Nguyen, Fuzhen Zhuang, Hui Xiong, Sanglu Lu

Abstract

Multi-Task Multi-Label (MTML) learning faces significant challenges due to data sparsity and complex, non-uniform task correlations. Traditional methods often treat tasks independently or apply global grouping, failing to capture label-specific relationships. This paper introduces LABTAG (LABel-sensitive TAsk Grouping), a Bayesian nonparametric framework that automatically discovers task clusters on a per-label basis. By sharing knowledge across similar tasks for specific labels, LABTAG improves performance over state-of-the-art baselines. Experimental results on real-world datasets demonstrate its superior accuracy and ability to reveal hidden task structures.

Introduction

In many real-world scenarios, we encounter Multi-Task Multi-Label (MTML) learning problems. For instance, in a movie recommendation system, each user represents a "task," while the various contexts in which they might watch a movie (e.g., at home, on a weekend, with a spouse) serve as "labels."

This domain presents two primary challenges:

  1. Data Sparsity: Individual tasks (users) often have very few data points, making it difficult to train robust models in isolation.
  2. Complex Correlations: Tasks are often related, but these relationships are rarely uniform across all labels. Two users might share identical preferences for "weekend" movies but have completely divergent tastes for "weekday" movies.

Existing models typically treat tasks as independent or group them globally. LABTAG addresses these limitations through label-sensitive grouping, where task similarity is evaluated independently for every label, allowing for more nuanced and effective knowledge sharing.

Methodology

LABTAG is built on Bayesian nonparametric principles, designed to handle uncertainty and discover latent structures without pre-defined parameters. The core of the framework involves a two-level modeling approach:

  • Level 1: Feature-Label Correlations: The model captures patterns of how specific features relate to specific labels.
  • Level 2: Label-Sensitive Task Grouping: For every label $c$, the model clusters tasks that exhibit similar behavior for that specific label.
The probabilistic graphical model of LABTAG, illustrating the hierarchical Bayesian structure and the use of Dirichlet Processes for task grouping.

To manage the complexity of these relationships, the model utilizes Dirichlet Process (DP) Priors. This allows the system to "learn" the appropriate number of feature-label patterns and task clusters directly from the data, avoiding the need for manual hyperparameter tuning. Furthermore, the authors employ Variational Inference to approximate the posterior distribution, ensuring the learning process is computationally efficient and deterministic compared to traditional MCMC sampling.

Experimental Setup

The model was evaluated on three public datasets representing diverse MTML challenges:

  1. TripAdvisor: A hotel rating dataset where tasks are users and labels are trip types.
  2. LDOS-CoMoDa: A movie recommendation dataset with labels including time, location, and companion.
  3. Enron Email Corpus: A natural language processing dataset for email categorization.

These datasets vary significantly in scale, with tasks ranging from 10 to 46, labels from 5 to 58, and features from 93 to 460, providing a rigorous test for the model's robustness.

Key Results and Insights

LABTAG was compared against several strong baselines, including Binary Relevance (BR), ML-kNN, and Bayesian Nonparametric Multi-label Classification (BNMC).

Performance Metrics

LABTAG achieved the highest accuracy across all three datasets. Notably, on the Enron corpus, it reached 69.06% accuracy, outperforming the next best baseline (BR at 66.20%). It consistently led in five key evaluation metrics: Accuracy, F1-score, Hamming loss, One-error, and Rank loss.

Visualizing Task Clusters

The effectiveness of label-sensitive grouping is best illustrated through Hinton diagrams. These visualizations confirm that tasks are indeed grouped differently depending on the label, validating the core hypothesis that task similarity is not a global constant.

A Hinton diagram for Label 3, visualizing how tasks are grouped. The varying block sizes represent the strength of task correlations for this specific label.

Conclusion and Future Work

LABTAG demonstrates that label-sensitive task grouping significantly enhances MTML performance, especially when training data is sparse. By "borrowing" knowledge from similar tasks within a group, it prevents the over-fitting common in single-task models.

Future research directions include reducing the computational overhead of complex matrix operations to better handle "Big Data" scales. Additionally, integrating this grouping logic with Deep Learning architectures and addressing the "cold-start" problem for new tasks remain promising areas for exploration.

Summary updated

Abstract

Multi-label learning is widely applied in many real-world applications, such as image and gene annotation. While most of the existing multi-label learning models focus on the single-task learning problem, there are always some tasks that share some commonalities, which can help each other to improve the learning performances if the knowledge in the similar tasks can be smartly shared. In this paper, we propose a LABel-sensitive TAsk Grouping framework, named LABTAG, based on Bayesian nonparametric approach for multi-task multi-label classification. The proposed framework explores the label correlations to capture feature-label patterns, and clusters similar tasks into groups with shared knowledge, which are learned jointly to produce a strengthened multi-task multi-label model. We evaluate the model performance on three public multi-task multi-label data sets, and the results show that LABTAG outperforms the compared baselines with a significant margin.
BibTeX
@inproceedings{zhang2018label,
  author = {Xiao Zhang and Wenzhong Li and Vu Nguyen and Fuzhen Zhuang and Hui Xiong and Sanglu Lu},
  title = {{Label-Sensitive Task Grouping by Bayesian Nonparametric Approach for Multi-Task Multi-Label Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/434},
}
Figure