When an AI evaluates a piece of content—whether a translation, an essay, or a video—it typically produces a single scalar value. However, a score like "0.75" is fundamentally opaque. It tells us the output is "good," but it doesn't explain why. Recent research has converged on a powerful solution: Hierarchical Multi-task Learning (MTL). By training models to simultaneously identify local, fine-grained errors and predict global quality scores, researchers are creating systems that are not only more accurate but also more explainable.
The Granularity Gap
The core challenge in Quality Estimation (QE) is the "granularity gap." A global score (sentence-level) is an abstraction of many local features (word-level). Historically, these were treated as separate problems. You either built a sequence labeler to find typos or a regressor to predict a grade.
As Hyun Kim and colleagues argue in QE BERT, this separation is artificial. A sentence's quality is the aggregate of its parts. By using a shared Transformer backbone to predict both word-level error tags and sentence-level scores, the model learns a more robust internal representation. The local task acts as a "grounding" mechanism for the global task.
Why Joint Training Works: Grounding and Regularization
The intuition behind this approach is twofold: grounding and regularization.
- Grounding: When a model like CometKiwi, developed by Marcos Vinicius Treviso and Ricardo Rei, predicts a low score for a translation, the multi-task objective forces it to "justify" that score by identifying specific word-level errors. This prevents the model from relying on "hallucinated" features or surface-level statistics that don't actually correlate with quality.
- Regularization: Predicting word-level tags serves as a powerful regularizer. In xCOMET, Nuno M. Guerreiro and André F. T. Martins demonstrate that forcing the encoder to detect error spans makes the global quality regression more stable, especially in low-resource scenarios where data is sparse.
This synergy is visible in the QE Brain system by Jiayi Wang and Kai Fan, which won the WMT18 QE task. Their model showed that fine-grained token information directly improves the global assessment by providing a "bottom-up" signal that complements the "top-down" semantic understanding.
Beyond Machine Translation
While much of this work originated in translation, the principle of hierarchical MTL is spreading across domains:
- Automated Essay Scoring: Youmna Farag and Helen Yannakoudakis showed in Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input that models can be fooled by "word salads"—grammatically correct but nonsensical sentences. By adding a task that evaluates local coherence between adjacent sentences, the model learns to distinguish between genuine quality and sophisticated nonsense.
- Speech Assessment: In the MultiPA system, Julia Hirschberg and Yu-Wen Chen use word-level tasks like phone accuracy and stress to inform sentence-level metrics like fluency and prosody.
- Video Quality: The LMM-VQA framework by Qihang Ge and Wei Sun combines continuous score regression with discrete quality classification, creating a "synergy where qualitative labels provide a structured framework" for quantitative scores.
The Synergy of Correction and Estimation
Perhaps the most advanced iteration of this idea is the unification of evaluation and correction. If a model can fix an error, it clearly understands that the error exists.
In Unified Automated Essay Scoring and Grammatical Error Correction, Seungwoo Song and colleagues found that a model's ability to "think through" grammatical corrections significantly boosts its accuracy in assigning proficiency scores. Similarly, Diptesh Kanojia and Pushpak Bhattacharyya demonstrated in Quality Estimation-Assisted Automatic Post-Editing that treating QE and post-editing as a joint "bargaining game" (Nash-MTL) reduces over-correction and improves the refinement of machine-translated text.
Conclusion
The move toward hierarchical multi-task learning represents a shift from "black-box" scoring to "evidence-based" evaluation. By forcing models to bridge the gap between local defects and global quality, researchers are building systems that are not just more robust, but also provide the "why" behind the "what." As Chrysoula Zerva and Ricardo Rei noted in the WMT 2023 Shared Task on Quality Estimation, this multi-task approach is the key to making AI evaluation truly explainable.