Multi-Group Transfer Learning on Multiple Latent Spaces for Text Classification

Jianhan Pan, Teng Cui, Thuc Duy Le, Xiaomei Li, Jing ZhangIEEE Access 2020 · DOI

Abstract Transfer learning (TL) leverages knowledge from a source domain to improve performance in a target domain. While latent feature spaces are more effective than raw features for cross-domain text classification, existing multi-latent space methods often suffer from "negative transfer" due to low relevance between spaces and exponential computational complexity. This paper introduces Multiple Groups Transfer Learning (MGTL), which organizes latent spaces into overlapping groups and utilizes an indirect-promotion strategy via a shared classifier. Experimental results on 20-Newsgroups and sentiment datasets demonstrate that MGTL achieves superior accuracy, eliminates negative transfer, and maintains linear computational complexity.

Introduction

In cross-domain text classification, the "source" and "target" domains often share high-level concepts (latent information) even when their raw vocabularies differ. For instance, "CPU" and "Android" both relate to technology but belong to different sub-domains (hardware vs. OS). Previous attempts to capture this information by constructing multiple independent latent spaces faced two critical issues. First, forcing interactions between unrelated spaces can lead to negative transfer, where irrelevant data degrades the model. Second, the standard "direct-promotion" strategy—where every space influences every other space—results in computational costs that grow exponentially as the number of spaces increases.

Methodology

The MGTL framework addresses these challenges through two primary innovations: latent space grouping and an indirect-promotion strategy.

Latent Space Grouping and Overlapping Relevance

Instead of treating each latent space as an isolated entity, MGTL clusters adjacent spaces with similar dimensions into groups. To ensure these groups remain relevant to one another, the framework enforces an "overlapping" rule: adjacent groups must share at least one latent space. This creates a conceptual bridge across the entire model, ensuring that knowledge transfer is grounded in shared context.

Comparison between independent multi-latent spaces (a) and the proposed multi-latent space groups (b).

Indirect-Promotion Strategy

To solve the complexity explosion, MGTL moves away from the "all-to-all" direct-promotion model. Instead, it introduces a central "Classifier" that acts as a hub. Each latent space group promotes the learning of this shared classifier, which in turn informs the other groups. This "indirect-promotion" ensures that the computational complexity scales linearly with the number of groups, rather than exponentially.

Direct-promotion strategy (a) with exponential complexity vs. the MGTL indirect-promotion strategy (b) with linear complexity.

Experimental Results

The model was rigorously tested on the 20-Newsgroups and multi-domain sentiment datasets.

  • Accuracy: MGTL achieved a peak accuracy of 98.02% on traditional tasks, significantly outperforming standard baselines like Logistic Regression (65.57%).
  • Robustness: In high-risk scenarios where the target domain contained entirely new categories, MGTL was the only method to consistently avoid negative transfer (NumNT = 0).
  • Efficiency: Empirical tests confirmed that as the number of latent spaces increased, MGTL's runtime remained manageable and linear, whereas direct-promotion methods quickly became computationally prohibitive.

Conclusion

MGTL provides a scalable and robust framework for cross-domain text classification. By bridging latent spaces through overlapping groups and centralizing knowledge transfer via a shared classifier, it effectively mitigates the risks of negative transfer and computational inefficiency. Future work will focus on automating parameter selection to make the framework more adaptive to diverse NLP tasks.

Summary updated

BibTeX
@article{pan2020multi,
  author = {Jianhan Pan and Teng Cui and Thuc Duy Le and Xiaomei Li and Jing Zhang},
  title = {{Multi-Group Transfer Learning on Multiple Latent Spaces for Text Classification}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2984571},
}
Figure