Abstract
This paper introduces a three-stage pipeline for multi-label text classification that simplifies the task by decoupling category prediction from label frequency estimation. Utilizing Byte-Pair Encoding (BPE) and Linear Support Vector Machines (SVM), the approach achieves state-of-the-art results on the GermEval 2019 German book classification task. The proposed method provides a computationally efficient alternative to deep learning models, particularly effective for morphologically rich languages like German.
Introduction
Multi-label multi-class (MLMC) text classification remains a significant challenge in Natural Language Processing, particularly when labels are organized hierarchically. In the context of the GermEval 2019 Task 1, researchers were tasked with classifying German book descriptions (“blurbs”) into 8 broad categories and 343 specific sub-categories.
The authors identify three primary hurdles in this domain:
- Class Imbalance: Significant disparities exist between frequent and rare categories.
- Multi-label Complexity: Standard classifiers often struggle with instances belonging to multiple categories simultaneously.
- Interpretability and Efficiency: While deep learning is popular, its “black box” nature and high computational overhead make traditional machine learning models attractive alternatives if they can achieve competitive performance.
Methodology
The core innovation is a "Label Frequency Transformation" pipeline that avoids the complexity of native multi-label algorithms by decomposing the problem into three distinct stages:
- Data Transformation: The multi-label training data is “flattened.” For example, a book with three labels is treated as three separate training instances, each with a single label.
- Category Prediction: A Linear Support Vector Classification (LinearSVC) model is trained on the flattened data to generate category scores.
- Label Count Prediction: A separate Class Count Predictor determines the number of labels (k) an input should receive.
- Model Fusion: The final prediction is made by selecting the top k labels based on the scores from the Category Prediction Model.
Technical Implementation
- Tokenization: The model employs Byte-Pair Encoding (BPE), which is particularly effective for German compound words by breaking them into sub-word units.
- Feature Engineering: Beyond standard tf-idf vectors, the authors incorporated metadata such as ISBN-based Group and Publisher IDs.
- Optimization: Grid search was used for hyperparameter tuning, and specific class weights were applied to the SVM to mitigate the impact of rare categories.
Dataset and Experimental Setup
The study utilizes the GermEval 2019 Task 1 dataset, comprising German book data (titles, descriptions, authors, ISBNs) crawled from randomhouse.de. The hierarchy includes 343 total categories across three levels, divided into Subtask (a) (8 generic categories) and Subtask (b) (343 specific categories).
Results and Key Insights
The system, submitted as Team Raghavan, demonstrated strong performance:
- Subtask (a): Achieved an F1-micro score of 0.878, securing 1st place in the post-evaluation phase.
- Subtask (b): Achieved an F1-micro score of 0.395.
Key findings include the power of metadata; adding ISBN-based features provided a +0.02 boost in F1-score, suggesting that publisher identity is a strong signal for book categorization. Furthermore, the efficiency of the pipeline is notable, with the entire suite retraining in less than two minutes.
Limitations and Future Work
The model currently struggles with deep hierarchies (Subtask b) because it treats labels as flat during initial prediction, failing to fully exploit parent-child relationships. Additionally, the class count model is limited by data sparsity; as shown in the distribution of labels, instances with four or more labels are extremely rare, making them difficult to model accurately.
Future work will focus on incorporating hierarchical dependencies directly into the loss function and improving the prediction of label frequencies for rare multi-label configurations.
Summary updated