Abstract
This paper introduces Dual-Label Learning (DLL), a framework designed to handle multi-task learning scenarios where labels are missing irregularly—a common challenge in scientific research. By treating two correlated labels as a bivariate system of equations and employing a novel "dual-tower" architecture, the authors demonstrate that partially labeled data can significantly improve model accuracy, even when up to 60% of the labels are missing.
The Problem: Irregularly Present Labels
In ideal multi-task datasets, every data point possesses a complete set of labels. However, in fields like drug discovery or materials science, labels are often "irregularly present." For instance, a drug molecule might be tested for one type of toxicity but not another due to budget or time constraints.
Traditional machine learning models typically handle this by either discarding samples with missing labels—which wastes valuable data—or using "shared-bottom" architectures that fail to explicitly model label correlations. The central challenge is maximizing the utility of these "semi-labeled" samples to improve predictions across all tasks.
The Dual-Label Learning (DLL) Framework
The authors propose shifting the perspective from viewing labels as independent targets to seeing them as a dual-function system. The DLL framework is built upon three core components:
- Dual-Tower Architecture: The model consists of two "towers." Tower 1 predicts label $y_1$ using features $x$ and label $y_2$ as an input. Tower 2 performs the inverse, predicting $y_2$ using $x$ and $y_1$.
- Three-Pronged Training:
- Standard Supervision: Learning from fully labeled samples.
- Structural Duality: If only $y_1$ is known, the model predicts $y_2$, then uses that prediction to "reconstruct" $y_1$. This ensures the towers remain consistent with one another.
- Probabilistic Duality: A regularization term based on Bayes' Theorem aligns the joint probability distributions of the two towers.
- Alternate Inference: For samples where both labels are missing, the model performs an iterative process. It makes an initial guess for $y_1$, uses it to predict $y_2$, then uses that $y_2$ to refine $y_1$, repeating until the values converge.
Experimental Evaluation
The authors evaluated DLL on three distinct datasets:
- Tox21: A real-world toxicity dataset (binary classification) with a 26% missing-label rate.
- HIGGS: A synthetic high-energy physics dataset (regression) used to simulate varying missing-label rates.
- MOF: A real-world materials science dataset (regression) focused on the synthesis conditions of metal-organic frameworks.
Key Results
- Performance Gains: DLL outperformed baseline methods (such as Independent Decomposition and Label Stacking) by up to 9.6% in F1-score for classification and reduced error (MAPE) by up to 10.2% in regression tasks.
- Extreme Robustness: DLL maintained high performance even when 60% of labels were missing. In several instances, DLL with 60% missing data outperformed baseline models that had access to 90% complete data.
- Efficiency: Despite the iterative nature of alternate inference, the model typically reached equilibrium in fewer than 10 iterations, ensuring computational efficiency during deployment.
Insights and Future Work
The core innovation of DLL is treating labels as informative features for other labels. By formulating inference as solving a system of two equations ($y_1 = g(x, y_2)$ and $y_2 = f(x, y_1)$), the model can "triangulate" correct labels even from an unlabeled starting point.
While currently optimized for two labels, the authors suggest the framework can be extended to multi-label cases. Future research will focus on scaling to N-labels through generalized multi-tower architectures and providing formal mathematical proofs for convergence across arbitrary neural network configurations.
Summary updated