Multi-task learning (MTL) operates on a compelling premise: by forcing a model to learn multiple related tasks simultaneously, it develops a richer, more generalized internal representation. However, this premise assumes a level of symmetry between tasks that rarely exists in practice.
In reality, tasks often diverge significantly in their semantic granularity—the resolution and complexity of their output spaces. When a fine-grained task (such as predicting continuous similarity scores or identifying specific semantic relations) is paired with a coarse-grained task (such as binary classification), standard multi-task architectures frequently degrade rather than improve performance. Understanding why this mismatch occurs requires looking at the asymmetry of information flow and the optimization dynamics of shared neural networks.
The Asymmetry of Information Flow
Intuition suggests that if two tasks are semantically related, knowledge should flow freely between them. Yet, empirical evidence reveals a stark asymmetry: high-resolution, fine-grained supervision signals easily generalize downward to coarse tasks, but low-resolution, coarse signals fail to scale upward.
[Fine-Grained Task] ---> (Rich Semantic Nuances) ---> [Coarse-Grained Task] (Success)
[Coarse-Grained Task] ---> (Insufficient Detail) -x-> [Fine-Grained Task] (Degradation)
For instance, in natural language processing, Boyd-Graber et al. observed that models trained on fine-grained, real-valued text similarity scores transfer effectively to coarse binary classification tasks. The reverse, however, causes severe performance degradation because binary labels lack the precision to map complex semantic nuances.
This pattern repeats across diverse domains:
- Structural Analysis: Lauscher et al. found that low-level argumentative unit identification acts as a useful prerequisite for high-level rhetorical classification, but high-level labels provide no beneficial signal back to the low-level task.
- Information Extraction: Guo et al. demonstrated that deep relation classification data boosts shallow semantic role labeling, whereas shallow roles fail to assist fine-grained classification.
- Ontological Mapping: When transferring knowledge between domain ontologies, Kim et al. showed that high-granularity data refines simpler schemas, but coarse-grained datasets introduce noise and overfitting when applied to complex, high-precision targets.
Coarser task signals simply lack the necessary detail to help a model refine the intricate decision boundaries required for high-resolution tasks.
Why Joint Training Fails: Gradient Dominance and Confusion
If fine-grained data contains a superset of the information needed for coarse tasks, why does standard joint optimization fail? The answer lies in optimization dynamics and feature space pollution. When a single neural network shares parameters across tasks of varying granularities, it encounters two destructive phenomena: Gradient Dominance and Feature Confusion.
Gradient Dominance
Coarse-grained distinctions (e.g., distinguishing a dog from a car) are mathematically "easier" to learn than fine-grained distinctions (e.g., distinguishing a Siberian Husky from an Alaskan Malamute). Consequently, the strong, stable gradients generated by a broad classification task overwhelm the subtle, delicate gradients required for fine-grained differentiation.
As highlighted by He et al. and Wang et al., this gradient imbalance prevents shared mapping functions from capturing the highly nuanced attributes necessary to resolve close decision boundaries. The network effectively prioritizes the loud, coarse signal at the expense of the quiet, fine-grained one.
Feature Confusion
In standard hard-parameter sharing, the model attempts to force disparate label spaces into a single representation. Lu et al. point out that the fine-grained labels of an auxiliary task can actively confuse a model trying to learn a coarser target category, as the subtle distinctions introduce irrelevant noise to the broader objective.
Conversely, trying to combine binary detection with fine-grained vector classification in a single loss function creates fundamental incompatibilities, a challenge explored by Kim in the context of hierarchical classification.
Bypassing the Mismatch via Decoupling
To unlock the benefits of multi-task learning without suffering from granularity mismatch, architectures must move away from naive parameter sharing toward structured isolation.
Hierarchical Information Extraction
One approach is to explicitly mirror the problem's hierarchy within the network's architecture. For example, Zhou et al. designed a framework that extracts broad scenario-level characteristics before attempting to optimize task-specific objectives. By separating these layers into a multi-stage hierarchy, they prevent the noise of a coarse scenario from bleeding into and corrupting a fine-grained task space.
[Input Features] ---> [Scenario-Level Extraction (Coarse)] ---> [Task-Specific Objectives (Fine)]
Neural Representation Matching
Another elegant solution is to decouple the label spaces entirely and transfer knowledge through the feature space instead. Instead of joint training via a shared classification head, Lu et al. utilized a technique where separate models are trained for the fine-grained and coarse-grained tasks, but their internal layers are incentivized to produce similar feature geometries.
This allows the coarse-grained model to inherit the rich semantic nuances of the fine-grained space without ever being exposed to the disruptive, high-resolution label gradients that would otherwise derail its optimization.
Key Foundational Work
A Unified Architecture for Semantic Role Labeling and Relation Classification (2016) — contributed 3 concept elements to this direction.
Bayesian Supervised Domain Adaptation for Short Text Similarity (2016) — contributed 3 concept elements to this direction.
Extending the evaluation of Genia Event task toward knowledge base construction and comparison to Gene Regulation Ontology task (2015) — contributed 2 concept elements to this direction.