Introduction
The rapid advancement of Natural Language Processing (NLP) has been fueled by large-scale datasets. However, most existing datasets are "siloed," designed for a single task like question answering (QA), summarization, or classification. This specialization creates a bottleneck for Multi-Task Learning (MTL). When a model is trained on multiple disparate datasets, it is difficult to determine if performance gains stem from better general representations or simply from exposure to more diverse data distributions.
To bridge this gap, researchers have introduced MATINF, the first large-scale Chinese dataset jointly labeled for three major NLP tasks. By providing a single source of data for multiple labels, MATINF offers a "fair stage" for evaluating MTL models without the confounding variables of differing data distributions.
The MATINF Dataset
MATINF is a massive corpus of 1.07 million unique question-answer pairs crawled from a popular Chinese maternity and childcare website. Each entry consists of a question (Q), a detailed situational description (D), human-labeled categories (C), and the "best answer" (A) chosen by the user.
Unlike general news datasets, MATINF-C focuses on a single domain, making classification challenging due to subtle semantic differences between categories like "infant nutrition" and "infant feeding." Furthermore, the summarization task (MATINF-SUMM) utilizes noisy, user-generated content, reflecting real-world social media dynamics rather than formal academic or news text.
Methods: The MTF-S2S Baseline
The authors propose MTF-S2S (Multi-task Field-shared Sequence to Sequence), a baseline model designed to leverage MATINF's multi-field structure. Unlike traditional MTL models that primarily share initial layers, MTF-S2S implements sharing at both the layer and module levels.
Key technical features include:
- Field-Sharing: The model uses shared encoders for question descriptions and shared encoder/decoder modules for the questions themselves.
- Synchronized Training: To handle varying sub-dataset sizes, batch sizes are adjusted proportionally, ensuring balanced training progress across tasks.
- Task-Specific Heads: Specialized fully connected layers handle the final output for distinct classification tasks, such as age-grouping versus topic categorization.
Results and Key Insights
Benchmarking with state-of-the-art models like BERT and ERNIE revealed significant advantages for the multi-task approach:
- Performance Gains: The MTF-S2S model outperformed its single-task counterparts in both QA (+0.74 ROUGE-L) and summarization (+4.73 ROUGE-L). This suggests that learning to classify and summarize provides auxiliary signals that improve answer generation.
- Efficiency: Testing with BERT-of-Theseus (a compressed 6-layer model) showed that high performance can be maintained with significantly reduced computational overhead compared to the full 12-layer BERT.
- Domain Hardness: The fine-grained nature of the maternity domain proved difficult for traditional statistical methods, highlighting the need for deep semantic understanding.
Conclusion
MATINF addresses a critical need in the NLP community for cross-task datasets. By providing a large-scale, jointly labeled corpus, it enables more rigorous research into multi-task learning and general language representation. While the web-crawled nature of the data introduces some noise, MATINF serves as a robust playground for developing models that can handle the informal, domain-specific language typical of social media.
Summary updated