Contextualizing Multiple Tasks via Learning to Decompose

Han-Jia Ye, Da-Wei Zhou, Lanqing Hong, Zhenguo Li, Xiu-Shen Wei, De-Chuan Zhan

Abstract

Meta-learning typically assumes a consistent mapping between visual features and labels across tasks. However, real-world data is often "polysemous," where objects possess multiple latent attributes (e.g., category, color, or material) that can serve as the basis for classification depending on the context. We present the LEArning to Decompose Network (LEADNET), a framework that addresses this ambiguity by decomposing a unified representation into multiple attribute-specific embedding spaces. By utilizing the support set as a disambiguating context, LEADNET identifies the relevant attributes for each task, significantly enhancing few-shot learning performance in ambiguous and biased environments.

Introduction

Standard meta-learning algorithms often struggle with data ambiguity. For instance, a pair of "red boots" might be categorized by one user as "boots" (category-based) and by another as "red" (color-based). If a meta-learner is trained on a mixture of such tasks without accounting for these shifts, the resulting embedding space becomes "confused," attempting to group visually similar items while the labels demand separation based on different latent rules.

Conceptual overview of Single Attribute per Task (SAT) and Mixed Attributes per Task (MAT) scenarios.

This paper addresses the problem of polysemous data by proposing that the support set should serve as a "context" to disambiguate the task's underlying logic. We distinguish between two primary scenarios: Single Attribute per Task (SAT), where an entire task follows one latent rule, and Mixed Attributes per Task (MAT), where different instances within the same task may be labeled based on different attributes.

Methodology: The LEADNET Framework

The core of LEADNET is the decomposition of a single embedding function $\phi$ into $C$ distinct attribute-specific embedding spaces using linear maps. The model then employs a Meta Disambiguation module to select or weight these spaces based on the support set.

Attribute Decomposition

Instead of learning a single global strategy, LEADNET learns a fixed set of "views" or embedding spaces. This approach is more computationally efficient than full weight adaptation for every new task, as it relies on selecting pre-learned representations.

Meta Disambiguation

The disambiguation process varies depending on the task structure:

  1. Single Attribute per Task (SAT): LEADNET uses a triplet-based module to analyze relationships between instances in the support set. By evaluating which triplets (e.g., two boots and one sandal) are consistent with a specific attribute space, the model infers the task's latent rule.
  2. Mixed Attributes per Task (MAT): In more complex scenarios, LEADNET utilizes "instance-label" pairs. It identifies the relevant attribute for each query instance individually by comparing it against the labeled examples in the support set.
The LEADNET architecture for SAT and MAT disambiguation.

Regularization

To ensure the learned spaces are diverse and discriminative, the authors introduce a regularization term $\Omega$. In the absence of explicit attribute labels during training, base classes are used as "synthetic attributes" to guide the decomposition process and force the model to discover latent structures.

Experimental Evaluation

LEADNET was evaluated across several datasets designed to test handling of ambiguity and bias:

  • UT-Zappos50k: A shoe dataset with six possible labeling attributes (e.g., heel height, closure).
  • Colored-Kuzushiji & Colored MNIST: Datasets where color acts as a distractor to test the model's ability to "de-bias" and focus on shape.
  • Standard Benchmarks: MiniImageNet, TieredImageNet, and CUB to verify performance on non-ambiguous few-shot tasks.

Key Results and Insights

The experimental results demonstrate that context-driven decomposition is highly effective:

  • Ambiguity Handling: On UT-Zappos50k, LEADNET achieved 39.19% (1-shot) and 48.29% (5-shot) accuracy, outperforming methods that had access to explicit attribute labels during training.
  • Robustness to Bias: In the "DrawClass" noise test on Colored-Kuzushiji, LEADNET maintained an accuracy of 84.43%, whereas standard ProtoNets saw significant performance drops. This confirms the model's ability to identify the correct latent attribute (shape) despite strong color bias.
  • Generalization: Competitive results on MiniImageNet (67.32% 1-shot) suggest that decomposition is a generally useful inductive bias for few-shot learning, even when data polysemy is not explicitly labeled.

Conclusion

LEADNET demonstrates that the support set is more than just a training set for a classifier; it is a vital context for disambiguating task-specific rules. By learning to decompose representations into attribute-specific spaces and selecting them dynamically, LEADNET provides a robust solution for meta-learning in complex, real-world environments. Future work may extend this approach to multi-modal data, where polysemy is even more prevalent.

Summary updated

BibTeX
@article{ye2024contextualizing,
  author = {Han-Jia Ye and Da-Wei Zhou and Lanqing Hong and Zhenguo Li and Xiu-Shen Wei and De-Chuan Zhan},
  title = {{Contextualizing Multiple Tasks via Learning to Decompose}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2024},
  doi = {10.1109/tpami.2023.3317425},
}
Figure