Task-Specific Preconditioner for Cross-Domain Few-Shot Learning

Suhyun Kang, Jungwon Park, Wonseok Lee, Wonjong Rhee

Introduction

Few-Shot Learning (FSL) has made significant strides in enabling models to recognize new objects from minimal examples. However, a persistent challenge remains: Cross-Domain Few-Shot Learning (CDFSL). While standard FSL excels when training and testing data share similar distributions (e.g., natural images), performance drops sharply when the model must adapt to specialized domains like satellite imagery or medical scans. Researchers have introduced Task-Specific Preconditioned gradient descent (TSP) to address this by meta-learning domain-specific optimization strategies that can be blended for new, unseen tasks.

Methodology: Task-Specific Preconditioning

The core innovation of TSP is treating the optimization process itself as a learnable, adaptive component. Instead of a fixed update rule, the method employs Preconditioned Gradient Descent (PGD). A "preconditioner" matrix $P$ is used to rescale gradients, effectively accounting for the local curvature of the loss landscape.

Conceptual overview of Task-Specific Preconditioning (TSP), where domain-specific preconditioners are combined to optimize a target task.

The TSP framework consists of three primary stages:

  1. Domain-Specific Preconditioners (DSPs): During meta-training, the model learns unique preconditioners for various training domains (e.g., Birds, Textures, Omniglot). These matrices capture the geometric nuances of their respective data distributions.
  2. Task-Coefficient Estimation: For a new task, a "Dataset Classifier" evaluates the support set to determine its similarity to known training domains, generating a set of weights.
  3. The Task-Specific Preconditioner ($P_T$): The final preconditioner is a linear combination of the DSPs, weighted by the task-coefficients.

A critical requirement for stability is that the preconditioner must be Positive Definite (PD). This ensures that the update direction always leads to a decrease in the loss function, effectively defining a valid Riemannian metric on the parameter space.

Experimental Setup

The method was rigorously evaluated on the Meta-Dataset benchmark, which encompasses 13 diverse datasets. These range from natural images (ImageNet, VGG Flower) to specialized domains (Fungi, Aircraft) and abstract representations (Omniglot, Quick Draw). This diversity is essential for testing the "cross-domain" capabilities of the proposed optimizer.

Results and Discussion

TSP was integrated into existing state-of-the-art architectures like TSA and TA²-Net, yielding significant performance improvements.

State-of-the-Art Performance

TSP achieved the highest average rank across the entire Meta-Dataset benchmark. Notable gains were observed in challenging domains when applied to TA²-Net, including a 4.6% increase on Traffic Signs and a 3.4% increase on Fungi.

The Importance of the PD Constraint

One of the key insights of the paper is the necessity of the Positive Definite constraint. Without it, learnable optimizers often fail to converge or generalize to unseen domains.

Performance comparison showing that PGD with a PD constraint significantly outperforms both standard GD and PGD without the constraint, especially on unseen domains.

As shown in the experimental results, enforcing the PD constraint allows the model to maintain high accuracy even on domains it has never encountered during training. This robustness suggests that TSP successfully interpolates optimization strategies by leveraging shared geometric traits across domains.

Conclusion

Task-Specific Preconditioning represents a significant step forward in CDFSL by moving away from "one-size-fits-all" optimization. By adapting the update rule to the specific geometry of each task, TSP achieves superior generalization. While the bi-level optimization during training is computationally demanding, the efficiency at inference time and the substantial accuracy gains make it a powerful tool for real-world few-shot applications.

Summary updated

Abstract

Cross-Domain Few-Shot Learning~(CDFSL) methods typically parameterize models with task-agnostic and task-specific parameters. To adapt task-specific parameters, recent approaches have utilized fixed optimization strategies, despite their potential sub-optimality across varying domains or target tasks. To address this issue, we propose a novel adaptation mechanism called Task-Specific Preconditioned gradient descent~(TSP). Our method first meta-learns Domain-Specific Preconditioners~(DSPs) that capture the characteristics of each meta-training domain, which are then linearly combined using task-coefficients to form the Task-Specific Preconditioner. The preconditioner is applied to gradient descent, making the optimization adaptive to the target task. We constrain our preconditioners to be positive definite, guiding the preconditioned gradient toward the direction of steepest descent. Empirical evaluations on the Meta-Dataset show that TSP achieves state-of-the-art performance across diverse experimental scenarios.
BibTeX
@inproceedings{kang2025task,
  author = {Suhyun Kang and Jungwon Park and Wonseok Lee and Wonjong Rhee},
  title = {{Task-Specific Preconditioner for Cross-Domain Few-Shot Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i17.33953},
}
Figure