Abstract
Training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality-versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account, we could get the best of both worlds. To this end, we introduce “fidelity-weighted learning” (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network, trained on the task we care about on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher, who has access to limited samples with high-quality labels.
Introduction
In modern deep learning, the performance of models is often bottlenecked by the availability of high-quality labeled data. Researchers typically face a choice between two suboptimal paths: the "Strong Path," utilizing a small set of accurate expert labels that risk overfitting, or the "Weak Path," utilizing massive but noisy datasets generated via heuristics or crowd-sourcing. Standard techniques like pre-training on weak data followed by fine-tuning on strong data often fail to account for the inherent variance in label reliability. Fidelity-Weighted Learning (FWL) is proposed to resolve this dilemma by enabling the model to "know" which weak labels are likely erroneous and down-weighting them during training.
Methodology: Fidelity-Weighted Learning
FWL implements a three-step semi-supervised framework designed to leverage the strengths of both large-scale weak data and small-scale expert data.
- Student Pre-training: A deep neural network (the student) is first trained on a large, weakly-labeled dataset $\mathcal{D}_w$. This phase focuses on learning robust data representations (embeddings) $\psi(x_i)$.
- Teacher Training: A Bayesian model, specifically a Gaussian Process (GP), acts as the teacher. It is trained on a small set of high-quality labels $\mathcal{D}_s$. Crucially, the teacher operates on the student's learned representations $\psi^*(x_j)$, allowing it to estimate label quality through the student's "lens."
- Fidelity-Weighted Fine-tuning: The student is fine-tuned using soft labels generated by the teacher. The core innovation is the modulation of the learning rate $\eta$ on a per-sample basis. If the teacher expresses high uncertainty (variance) regarding a sample, the student's parameter update for that sample is minimized.
The learning rate modulation is governed by the function $\eta_2(x_t) = \exp[-\beta\Sigma(x_t)]$, where $\Sigma$ represents the teacher's posterior uncertainty. The hyperparameter $\beta$ allows for tuning the bias-variance trade-off between the teacher's guidance and the student's initial knowledge.
Application to Document Ranking
The authors evaluated FWL on the task of Document Ranking using the Robust04 and ClueWeb (Category B) benchmarks. For these experiments, the student network utilized a specific neural ranking architecture that processes query-document pairs through embedding and compositionality layers before ranking.
Weak labels were generated using the BM25 algorithm, while strong labels consisted of official TREC query judgments. This setup allowed the researchers to test how well FWL could refine a model initially trained on simple heuristic-based rankings using a small set of expert relevance judgments.
Key Results and Insights
FWL demonstrated significant performance gains over traditional supervised learning and standard fine-tuning baselines. On the Robust04 dataset, FWL achieved a Mean Average Precision (MAP) of 0.3124, a substantial improvement over the 0.2810 achieved by standard fine-tuning ($NN_{W \to S}$) and the 0.1790 from training solely on strong labels ($NN_S$).
Key findings include:
- Robustness: FWL consistently improved results even when the initial weak labels were of lower quality.
- Efficiency: By filtering out unreliable samples via the learning rate, the framework effectively speeds up the convergence toward high-performing parameters.
- Representational Synergy: The success of the teacher is tied to the quality of the student's internal representations, highlighting a symbiotic relationship between the two components.
Conclusion
Fidelity-Weighted Learning provides a principled Bayesian approach to handling training data of varying quality. By treating the learning rate as a per-sample fidelity filter, FWL allows deep learning models to benefit from the scale of weak supervision without being misled by its noise. Future work may explore scaling this approach to other domains, such as computer vision, where automated or crowd-sourced labels are prevalent but imperfect.
Summary updated