Abstract
In multi-label text classification, each textual document can be assigned with one or more labels. Due to this nature, the multi-label text classification task is often considered to be more challenging compared to the binary or multi-class text classification problems. As an important task with broad applications in biomedicine such as assigning diagnosis codes, a number of different computational methods (e.g. training and combining binary classifiers for each label) have been proposed in recent years. However, many suffered from modest accuracy and efficiency, with only limited success in practical use. We propose ML-Net, a novel deep learning framework, for multi-label classification of biomedical texts. As an end-to-end system, ML-Net combines a label prediction network with an automated label count prediction mechanism to output an optimal set of labels by leveraging both predicted confidence score of each label and the contextual information in the target document. We evaluate ML-Net on three independent, publicly-available corpora in two kinds of text genres: biomedical literature and clinical notes. Our benchmarking results show that ML-Net compares favorably to the state-of-the-art methods in multi-label classification of biomedical texts. ML-NET is also shown to be robust when evaluated on different text genres in biomedicine. Unlike traditional machine learning methods, ML-Net does not require human efforts in feature engineering and is highly efficient and scalable approach to tasks with a large set of labels. Finally, ML-NET is able to dynamically estimate the label count based on the document context in a more systematic and accurate manner.
Introduction
Multi-label classification is a critical task in biomedical informatics, where documents such as clinical notes or research abstracts frequently belong to multiple categories simultaneously. For instance, a single clinical report might document several distinct diagnoses, or a research paper might discuss multiple biological hallmarks.
Traditional approaches often rely on "binary relevance," which involves training independent classifiers for every possible label. This method is computationally expensive and fails to capture dependencies between labels. Modern deep learning models often output confidence scores for all labels, but they typically rely on a fixed global threshold (e.g., 0.5) to decide which labels to assign. This static approach ignores the document's specific context; some documents are inherently more complex and require more labels than others. ML-Net addresses this by introducing a dynamic label count prediction mechanism.
Methodology
ML-Net is an end-to-end deep learning framework designed to process hierarchical text structures and output a variable number of labels based on document content. The architecture consists of three primary levels: the Word Level, the Sentence Level, and the Label Decision Level.
Hierarchical Attention Network (HAN)
To capture the structure of biomedical documents, ML-Net employs a hierarchical approach. At the Word Level, tokens are passed through an embedding layer and a Bi-directional RNN (LSTM) to capture local context. An attention mechanism then identifies the most informative words to form sentence representations. At the Sentence Level, these representations are processed by another Bi-RNN and attention layer to produce a final Document Vector. This mirrors human reading patterns, where specific sentences and words carry more weight in determining the overall topic.
Label Prediction and Count Estimation
The Document Vector serves as the input for two parallel paths in the Label Decision Level:
- Label Prediction Network: This network outputs confidence scores for all possible labels. It utilizes a Log-Sum-Exp Pairwise (LSEP) loss function, which focuses on the relative ranking of labels rather than treating them as independent binary tasks. This ensures that relevant labels are ranked higher than irrelevant ones.
- Label Count Prediction Network: This is a multi-layer perceptron (MLP) that predicts the integer number of labels to be assigned. By using a "stop gradient" operation, the model ensures that the count prediction is informed by the document vector without interfering with the training of the hierarchical feature extractors. This allows the system to "know when to stop" based on the document's specific context.
Datasets and Evaluation
The performance of ML-Net was evaluated on three distinct biomedical datasets:
- Hallmarks of Cancers (HoC): 1,580 PubMed abstracts with 10 labels.
- Chemical Exposure Assessments: 3,661 PubMed abstracts with 32 labels.
- MIMIC II (Clinical Notes): A large-scale dataset of 22,815 clinical discharge summaries with over 7,000 unique ICD-9 diagnosis codes, representing an "Extreme Multi-label" challenge.
Evaluation metrics included example-based precision, recall, and F-measure to assess the model's ability to correctly identify the set of labels for each document.
Results and Discussion
ML-Net demonstrated superior performance compared to traditional baselines like SVMs and standard deep learning models. On the Hallmarks of Cancer dataset, ML-Net achieved an F-score of 0.815, a significant improvement over the SVM baseline (0.714).
A key finding was the effectiveness of the dynamic label count network. In literature-based tasks (HoC and Chemical Exposure), the version of ML-Net with automated count prediction outperformed versions using fixed thresholds. This confirms that context-aware thresholding is more effective for varying document densities.
However, results on the MIMIC II dataset were more complex. A threshold-based version of ML-Net (F-score 0.416) outperformed the label count version (0.346). This discrepancy is likely due to the strict hierarchical nature of ICD-9 codes, which the current count network does not explicitly model.
Conclusion
ML-Net represents a significant advancement in biomedical multi-label classification by eliminating the need for manual feature engineering and providing a systematic way to determine label counts. While highly effective for literature classification, future work will focus on incorporating label hierarchies and exploring more advanced sentence encoders to further enhance performance on complex clinical datasets.
Summary updated