Introduction
Multi-Task Learning (MTL) aims to improve the performance of multiple related tasks by learning them simultaneously. The standard approach, Hard Parameter Sharing (HPS), utilizes a shared backbone to extract features followed by task-specific heads. However, HPS often suffers from "task competition," where gradients from different tasks conflict, leading to suboptimal performance. While existing methods attempt to balance losses or manipulate gradients, they rarely address the fundamental way task-specific features are derived from the shared representation.
The NORMAL Framework
The NORMAL (Neural Ordinal diffeRential equation based Multi-tAsk Learning) method proposes a paradigm shift: modeling task-specific transformations as a continuous evolution of features. Instead of discrete layers, NORMAL inserts a Time-Aware Neural ODE block between the shared feature extractor and the task-specific heads.
At the heart of NORMAL is the concept of Task Positions ($p_i$). Each task $i$ is assigned a learnable "time" point along a continuous dynamic flow. To generate features for a specific task, the shared base feature $z(0)$ is evolved through the ODE until it reaches the task's unique position $p_i$. This allows the model to learn the optimal "depth" or transformation required for each task. To ensure stability and training efficiency, the authors employ a Heavy Ball Neural ODE (HBNODE), a second-order ODE that incorporates momentum-like dynamics.
Experimental Evaluation
The effectiveness of NORMAL was evaluated across four diverse MTL benchmarks:
- Office-31 & Office-Home: Image classification across multiple domains.
- NYUv2: A multi-modal vision dataset requiring semantic segmentation, depth estimation, and surface normal prediction.
- CelebA: Large-scale facial attribute classification involving 40 tasks.
Results and Interpretability
NORMAL consistently outperformed state-of-the-art baselines, including specialized gradient manipulation techniques like PCGrad and Nash-MTL. For instance, on the Office-31 dataset, NORMAL achieved 94.80% average accuracy, surpassing the Equal Weighting baseline of 93.82%.
Beyond performance, NORMAL offers unique interpretability through its learned task positions. As shown in the evolution of $p_i$ during training, tasks that are semantically similar tend to converge toward similar positions in the continuous flow.
This implicit task grouping suggests that the model naturally discovers relationships between tasks. In the CelebA dataset, attributes related to hair or facial structure often clustered together, validating that "time" in the ODE serves as a proxy for task specificity.
Conclusion
By reimagining MTL as a continuous dynamic flow, the NORMAL method provides a flexible and interpretable way to handle task-specific transformations. While it introduces some computational overhead due to the ODE solver, the gains in accuracy and the ability to visualize task relationships make it a promising direction for complex multi-task systems. Future work may involve extending this "task position" concept to other architectures and domains like Natural Language Processing.
Summary updated