Abstract
Multi-label text classification (MLTC) is a complex task where documents are associated with multiple, often semantically related, labels. Traditional supervised learning approaches often fail to capture intricate label correlations or effectively manage documents with partial label overlaps. We present the Unified Contextual and Label-Aware Framework (UCLAF), which integrates a Label Attention Aware Network (LAN) with a novel Correlation Aware Contrastive Learning (CACL) strategy. LAN utilizes an additive attention mechanism to align textual features with label semantics, while CACL refines feature representations by modeling sample-level relationships based on proportional label overlap. Our evaluation on the AAPD and EURLex benchmarks demonstrates that UCLAF achieves state-of-the-art performance, particularly in large-scale and sparse label environments.
Introduction
Multi-label text classification (MLTC) is significantly more challenging than standard single-label classification. In MLTC, a single document—such as a research abstract or a legal document—can simultaneously belong to multiple categories. This inherent complexity presents two primary challenges:
- Intricate Label Correlations: Labels are rarely independent; for example, "Environment" and "Policy" often co-occur. Most existing models fail to deeply capture these semantic dependencies.
- Partial Label Overlaps: Standard contrastive learning typically employs a binary view of similarity (positive vs. negative). In the multi-label domain, two documents may share a subset of labels, making this binary approach too reductive for optimal feature learning.
The UCLAF Framework
UCLAF addresses these challenges through a dual-module architecture designed to model both intra-sample and inter-sample relationships.
Label Attention Aware Network (LAN)
The LAN module embeds both text and labels into a shared semantic space. By employing an additive attention mechanism, the model identifies which parts of the text are most relevant to specific labels. This ensures that the model "understands" the semantic meaning of each label, rather than treating them as isolated identifiers.
Correlation Aware Contrastive Learning (CACL)
CACL represents the core innovation of the UCLAF framework. While standard Supervised Contrastive Learning (SupCon) requires exact label matches to identify positive pairs, CACL relaxes this constraint. It models sample-level relationships based on proportional overlap, treating a sample as a distinct instance for each label it carries.
As illustrated in Figure 2, CACL effectively generalizes beyond the "ALL" strategy (requiring exact matches) and the "ANY" strategy (requiring only one shared label). This approach prevents "representation collapse"—a common issue where models oversimplify complex data—by preserving the nuances of secondary labels.
Experimental Results
The performance of UCLAF was evaluated on two widely recognized MLTC benchmarks: AAPD (ArXiv Academic Paper Dataset) and EURLex (European Union legal documents).
UCLAF consistently outperformed several state-of-the-art baselines, including AttentionXML and MLGN. On the EURLex dataset, which features a massive and sparse label space, UCLAF improved P@1 (Precision at top 1) by 1.01% and P@3 by 2.62%. Furthermore, significant gains in Normalized Discounted Cumulative Gain (N@k) indicate that the model ranks relevant labels more accurately than its predecessors. Ablation studies confirmed that both the LAN and CACL modules are essential components of the framework's success.
Conclusion
UCLAF establishes a new state-of-the-art for MLTC by explicitly modeling label correlations and partial overlaps. Future research directions include the development of dynamic label embeddings that evolve during training and improving the computational efficiency of the CACL module to facilitate scaling to even larger datasets. Additionally, the authors suggest extending the UCLAF architecture to multi-modal tasks, such as joint image-text classification.
Summary updated