Abstract
Few-shot Unsupervised Domain Adaptation (FS-UDA) is a challenging task that requires adapting models from a limited labeled source domain to an unlabeled target domain. Traditional methods often suffer from performance instability and high computational overhead due to iterative optimization. We present $E^2$MPL (Enduring and Efficient Meta Prompt Learning), a novel framework that utilizes a frozen CLIP backbone and a bilevel optimization strategy with closed-form solutions. $E^2$MPL significantly improves accuracy and stability while reducing adaptation time by approximately 70% compared to current state-of-the-art methods.
Introduction
Unsupervised Domain Adaptation (UDA) aims to generalize knowledge from a labeled source domain to an unlabeled target domain. In the few-shot setting (FS-UDA), the source domain is restricted to only a few labeled examples per category, exacerbating the domain gap. Existing FS-UDA approaches are often plagued by two issues: (1) high variance in performance across different tasks, and (2) low efficiency due to the need for iterative gradient-based updates during task adaptation. $E^2$MPL addresses these by introducing a meta-learning framework designed for both "endurance" (stability) and "efficiency."
Methodology
The $E^2$MPL framework keeps the pre-trained CLIP backbone frozen to preserve its rich general-purpose features and avoid overfitting on small few-shot datasets.
Dual-Prompt Design
The framework employs two types of prompts to capture different levels of information:
- Domain-shared prompts: These learn general meta-knowledge across various tasks to help bridge the gap between source and target domains.
- Task-specific prompts: These are generated by a lightweight prompt network that adaptively adjusts based on the specific images in the current task.
Bilevel Optimization with Closed-Form Solutions
The meta-learning process is structured as a bilevel optimization:
- Inner Loop (Task Adaptation): Instead of iterative gradient descent, $E^2$MPL uses Ridge Regression and Cross-domain Projection. These have closed-form solutions, allowing the model to calculate optimal parameters in a single step, which ensures both speed and stability.
- Outer Loop (Meta-Update): The model updates the prompt parameters and the prompt network by minimizing a meta-loss that includes classification loss, entropy minimization, and class discrimination across multiple meta-tasks.
Experimental Results
The authors evaluated $E^2$MPL on the DomainNet benchmark, testing 20 source-to-target domain pairs across 3,600 distinct tasks.
Accuracy and Stability
$E^2$MPL achieved a 15.4% improvement in accuracy for 1-shot tasks and an 8.7% improvement for 5-shot tasks over previous state-of-the-art methods. Beyond average accuracy, the model demonstrated remarkable stability across different domain shifts.
Efficiency
The use of closed-form solutions in the inner loop resulted in a 68.5% reduction in adaptation time for 1-shot tasks and a 74.1% reduction for 5-shot tasks. This efficiency allows for rapid deployment and testing across a vast number of tasks.
Conclusion
$E^2$MPL provides a robust and efficient framework for FS-UDA by combining meta-prompt learning with closed-form optimization. By achieving high accuracy with minimal variance and computational cost, it sets a new standard for enduring and efficient domain adaptation. Future research will focus on addressing class imbalance and extending the framework to more complex, specialized real-world domains.
Summary updated