Abstract
Large language models (LLMs) often struggle with domain shift, particularly in subjective NLP tasks like emotion and empathy detection where labeled data is scarce and human annotations are frequently noisy. This paper introduces a novel Semi-Supervised Domain Adaptation (SSDA) framework that addresses these challenges through task decomposition and data cartography-based regularization. By training two models—an "Explorer" for unsupervised domain adaptation and a "Specialist" for semi-supervised learning—and using training dynamics to identify reliable samples, the framework significantly outperforms existing baselines in few-shot scenarios.
Introduction
While models like BERT have revolutionized NLP, they remain sensitive to domain shifts. A model trained on Reddit comments may not generalize well to clinical transcripts. In many real-world applications, we possess abundant labeled data in a source domain but very little in the target domain. This scenario, known as Semi-Supervised Domain Adaptation (SSDA), is further complicated in subjective tasks where labels are often inconsistent due to human disagreement. Standard approaches often suffer from source-domain bias or overfit to noisy target labels.
Methods
The proposed framework employs a two-pronged strategy to mitigate these issues:
1. Task Decomposition and Co-training
The system decomposes the learning process into two distinct models:
- The Explorer ($\theta^{uda}$): Focused on Unsupervised Domain Adaptation, it learns from labeled source data and unlabeled target data.
- The Specialist ($\theta^{ssl}$): Focused on Semi-Supervised Learning, it utilizes the limited labeled target samples and unlabeled target data.
These models engage in co-training, exchanging high-confidence predictions on unlabeled data to mutually expand their training sets with pseudo-labels.
2. Data Cartography-based Regularization
To handle label noise, the authors utilize "data cartography" to track training dynamics—specifically, the confidence and variability of a model's predictions for each sample over multiple epochs.
- Easy-to-learn samples (high confidence, low variability) are identified as reliable "gold standards."
- A Mixup strategy then interpolates these reliable samples with generated pseudo-labels, effectively denoising the training process and preventing overfitting to incorrect labels.
Datasets and Experimental Setup
The framework was evaluated across three diverse text classification tasks involving significant domain shifts:
- Empathy Detection: News stories (NewsEmp) to health-related tweets (TwitEmp).
- Emotion Detection: Reddit comments (GoEmotions) to TV show dialogues (MELD).
- Sentiment Analysis: Business reviews (Yelp) to movie reviews (IMDB).
Experiments were conducted in "few-shot" settings with $K \in \{4, 8, 16\}$ labeled examples per class in the target domain.
Key Results
The proposed method consistently outperformed strong baselines, including standard SSL and other co-training variants, across all datasets and shot counts.
In the Empathy Detection task (News to Health) with only 4 labels per class ($K=4$), the model achieved 69.33% accuracy, a substantial improvement over the Target-Only (42.39%) and standard SSL (44.59%) baselines. The performance remained robust as $K$ increased, and ablation studies confirmed that the data cartography-based denoising ("Co-training + Ours") provided a superior regularization effect compared to standard Mixup.
Discussion and Conclusion
By separating adaptation from target-specific learning and leveraging the model's own learning history to denoise labels, this framework provides a robust solution for low-resource, subjective NLP tasks. The primary trade-off is the increased computational cost required to estimate training dynamics over several epochs. Future research will focus on characterizing training examples "on the fly" to improve efficiency and scaling the approach to tasks with higher class cardinality.
Summary updated