When we train a single machine learning model on a large, diverse population, we often fall into the trap of aggregation bias. As Harini Suresh explains in Understanding Potential Sources of Harm throughout the Machine Learning Life Cycle, this occurs when a model converges on a solution that works well for the majority but fails to account for the distinct underlying distributions of minority subgroups.
A growing body of research suggests that the solution is not to ignore these differences, but to embrace them through Multi-Task Learning (MTL). By treating each demographic subgroup as a distinct "task," researchers are building models that share global knowledge while specializing in local nuances.
The Intuition: Shared Foundations, Specialized Heads
The core mechanism of this approach is a structural split in the model's architecture. Instead of a monolithic function $f(x) = y$, the model is divided into:
- Shared Parameters: A backbone that learns universal patterns applicable to all groups.
- Task-Specific Parameters: Specialized "heads" or deviation vectors that capture the unique characteristics of specific demographic groups.
Luca Oneto, Michele Donini, and Massimiliano Pontil demonstrate this in Taking Advantage of Multitask Learning for Fair Classification. They frame demographic groups as related tasks, allowing minority groups to leverage the statistical power of the majority population through shared parameters. This prevents the overfitting that usually happens when you try to train a separate model for a small minority group, while still allowing the model to adjust its decision boundary for that group’s specific needs.
Solving the Fairness-Accuracy Trade-off
Traditional fairness interventions often rely on "blindness"—trying to strip sensitive information (like gender or race) from the data. However, Raghuveer Peri and Krishna Somandepalli argue in To train or not to train adversarially that this can actually degrade performance. In speaker recognition, for example, gender-neutralizing a model makes it less accurate.
By using MTL to make the model "demographic-aware" rather than "demographic-blind," the system can account for acoustic nuances specific to a group without using that information to discriminate. This allows the model to maintain high utility while narrowing the fairness gap.
Applications Across Domains
This "groups-as-tasks" framing is proving effective in high-stakes fields where data heterogeneity is a given:
- Clinical Predictions: In Developing Fairness-Aware Task Decomposition to Improve Equity in Post-Spinal Fusion Complication Prediction, Wenqi Shi and May D. Wang use specialized neural network heads to route patient subgroups. This allows the model to learn group-specific risk profiles (e.g., how age affects recovery) while maintaining a shared foundation of clinical knowledge.
- Healthcare Heterogeneity: Dongxiao Zhu and Lu Wang apply a similar logic to disease staging in Tackling Multiple Ordinal Regression Problems. They show that treating demographic subgroups as separate tasks outperforms global models that ignore heterogeneity.
- Interpretable Tabular Learning: Even in interpretable models like Generalized Additive Models (GAMs), MTL provides a path to fairness. Guangzhi Xiong introduces ProtoNAM, which uses MTL to learn group-specific patterns without requiring sensitive features as direct inputs during inference, maintaining both accuracy and equity.
Why It Works
The success of this direction stems from how it handles statistical power. In a standard model, the minority group is "noise" to be averaged out. In a split-model approach, the minority group is a "task" to be solved. Because the tasks share a backbone, the model doesn't need a massive amount of minority-group data to learn general features (like "what does a healthy spine look like?"). It only needs enough data to learn the differences (the "deviation") from the global norm. This structural inductive bias makes MTL a powerful tool for building equitable AI in a heterogeneous world.