Source-Free Domain Adaptation for Multi-Label Image Classification: A Framework, Baselines, and Benchmark

Salih Numan Büyükbaş, Erchan Aptoula, Mamatha Thota, Petra BosiljarXiv 2026 · arXiv

TL;DR

This paper introduces the first comprehensive benchmark for source-free domain adaptation (SFDA) in multi-label image classification and proposes MPACT, a novel framework designed to handle overlapping labels. The authors demonstrate that simply porting single-label SFDA methods to multi-label settings often fails, highlighting the critical need for adaptation strategies that respect non-exclusive class relationships.

What Problem Does This Paper Address?

Deep learning models often struggle when deployed on data that differs from their training set (the "domain shift" problem). Source-Free Domain Adaptation (SFDA) aims to fix this by adapting a pre-trained model to a new, unlabeled target domain without needing access to the original training data—a common requirement due to privacy, licensing, or storage constraints.

While SFDA has seen rapid progress, nearly all existing research assumes single-label classification, where classes are mutually exclusive (e.g., an image is either a "cat" or a "dog"). However, real-world scenarios are often multi-label: an aerial image might contain "buildings," "trees," and "pavement" simultaneously. This paper tackles the "single-label trap," showing that core SFDA mechanisms—like softmax-based confidence and nearest-centroid pseudo-labeling—break down when multiple labels can co-occur.

Methods

The authors propose MPACT (Multi-label Prototype Alignment with Contrastive Teaching), a mean-teacher framework specifically re-engineered for multi-label data.

Figure 1: Overview of MPACT. Every 𝑇epochs the teacher 𝑓̄𝜃makes a full pass over the target data under weak augmentation and caches per-class soft pseudo-labels ̃𝑦; the student 𝑓𝜃trains on mini-batches under strong augmentation and is supervised by that cache until the next refresh, through the symmetric term sbce. The multi-label information maximization term im acts on the student’s predictions 𝑞alone. The student’s pooled features 𝑧are aligned to the class prototypes 𝜇𝑐by pca, with the prototypes constructed once before adaptation and held fixed throughout. The dashed arrow is the exponential moving average update of the teacher from the student, along which no gradient flows.

Key components include:

  • Symmetric Binary Cross Entropy (SBCE): A noise-robust loss function that handles the uncertainty of the teacher's pseudo-labels by combining forward and reverse terms.
  • Threshold-Aware Prototype Initialization: Unlike standard methods that average features across a batch (often resulting in prototypes biased toward the "batch mean"), MPACT only uses high-confidence samples to anchor class prototypes in feature space.
  • Prototype Contrastive Alignment (PCA): A loss term that pulls samples toward their relevant class prototypes. Crucially, it uses a soft-attraction mechanism that allows a single image to be "positive" for multiple prototypes at once, reflecting the multi-label nature of the data.
  • Multi-label Information Maximization: Encourages the model to make confident, diverse predictions across the independent Bernoulli variables representing each class.

Datasets

The authors established a new benchmark consisting of five adaptation tasks across six datasets, covering diverse imagery types:

Table 1 Multi-label datasets used in this study. Density is the mean number of positive labels per image on the training split. Cityscapes and Foggy Cityscapes share one row because the latter re-renders the same scenes under synthetic fog and inherits their annotations unchanged.
  1. Remote Sensing: UCM $\leftrightarrow$ AID (aerial scenes).
  2. Natural to Clipart: PASCAL VOC $\leftrightarrow$ Clipart1k.
  3. Driving Scenes: Cityscapes $\rightarrow$ Foggy Cityscapes (appearance shift under identical geometry).

Key Insights and Ideas

  • The "Softmax" Assumption is Deeply Embedded: Many SFDA methods rely on classes "competing" for probability mass. In multi-label settings, this competition is non-existent. Simply swapping a softmax for sigmoids is a "valid but insufficient" extension.
  • Sparsity and Imbalance: In multi-label datasets, most (sample, class) pairs are negative. This means batch-level statistics (like centroids) are easily corrupted by the overwhelming majority of negative samples if not carefully thresholded.
  • The Prototype-Agreement Diagnostic: The authors introduced a clever AUC-based diagnostic to distinguish between genuine feature-space alignment and a model simply "agreeing with its own mistakes" (self-reinforcement). They found that prototype alignment is most beneficial when the initial domain gap is large.
  • Negative Result on Filtering: Interestingly, they found that discarding "low-confidence" samples (a common trick in single-label SFDA) actually hurts multi-label performance because an image might be "low confidence" for one class but "high confidence" for another.

Key Results

  • MPACT is the most robust: It was the only method to improve upon the unadapted source model across all five benchmark tasks.
  • Quantitative Gains: MPACT achieved the best average mean Average Precision (mAP) of 55.71%, outperforming the strongest extended baseline (NVC-LLN at 54.22%) and significantly beating the source-only baseline (51.54%).
  • Failure of Baselines: Half of the extended single-label baselines failed to improve on the source model at all on average, with some (like UCON) even collapsing completely on certain tasks.
  • Qualitative Recovery: In the VOC $\rightarrow$ Clipart task, MPACT was the only method capable of recovering complete label sets for complex images, whereas baselines often predicted only a single label or none at all.

Limitations and Future Work

  • Static Prototypes: The authors found that making prototypes learnable didn't provide a measurable benefit in their current setup, though they suggest this might change with different optimization strategies.
  • The Diversity Prior: The current information maximization term assumes a target marginal probability of 0.5 for every class. For rare classes, this is a "systematically wrong target." Future work should focus on "marginal-aware" regularization that estimates class frequencies from the target data.
  • Open/Partial Labels: The current framework assumes a closed set of labels; extending this to "open-set" scenarios where new classes appear in the target domain remains an open challenge.

Summary updated

Abstract

Source-free domain adaptation (SFDA) adapts a model trained on a labelled source domain to an unlabelled target domain without access to the source data. Almost the entire SFDA literature assumes mutually exclusive classes, yet many deployed settings are multi-label. This paper formulates SFDA for multi-label classification and establishes the first benchmark for it: five adaptation tasks over six datasets spanning remote sensing, natural, clipart-style and driving imagery, with four single-label SFDA methods extended to provide the first baselines and a published multi-label method run alongside them, over three seeds. The benchmark shows that extending a single-label method is unreliable rather than merely suboptimal: half of the extended baselines fail to improve on the unadapted source model at all, and on two of the five tasks none of them does. The failures are not incidental. Nearest-centroid pseudo-labelling, softmax-based confidence, the diversity term of information maximization and confidence-weighted prototype construction each rely on mutual exclusivity, and each fails differently without it, so replacing the softmax with per-class sigmoids is a valid but insufficient extension. Multi-label Prototype Alignment with Contrastive Teaching (MPACT) is proposed in response, a mean-teacher framework whose objective, prototype construction and alignment term are each reconsidered for overlapping label sets. It attains the best average on every reported metric and is the only method that improves on the unadapted source model on all five tasks. The analysis further introduces a prototype-agreement diagnostic separating feature-space alignment from self-reinforcing agreement with the classifier, and reports a negative result for confidence-based pseudo-label filtering.
BibTeX
@misc{buyukbas2026source,
  author = {Salih Numan Büyükbaş and Erchan Aptoula and Mamatha Thota and Petra Bosilj},
  title = {{Source-Free Domain Adaptation for Multi-Label Image Classification: A Framework, Baselines, and Benchmark}},
  year = {2026},
  eprint = {2611.21107},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21107},
}
Figure