Effortless Active Labeling for Long-Term Test-Time Adaptation

Guowei Wang, Changxing Ding

Introduction

In real-world deployments, AI models frequently encounter data that differs from their training distribution—such as a vision system navigating heavy fog or artistic renditions of objects. Test-Time Adaptation (TTA) addresses this by allowing models to update themselves on-the-fly using unlabeled online data. However, "Long-Term" TTA is notoriously difficult. Most methods rely on self-training, where the model uses its own predictions as pseudo-labels. If the model makes an initial error, it learns from that mistake, creating a feedback loop of error accumulation that eventually leads to model collapse.

Active TTA (ATTA) attempts to mitigate this by requesting labels for a few samples from an oracle (human or larger model). While effective, traditional ATTA is resource-intensive, requiring multiple labels per batch and complex memory buffers. EATTA (Effortless Active Test-Time Adaptation) proposes a more efficient alternative: achieving robust adaptation with just one carefully selected label per batch.

Comparison between traditional ATTA and the proposed EATTA framework. EATTA simplifies the process by requiring only one sample per batch and making the buffer optional.

Methodology

EATTA introduces two primary innovations to enable high-performance adaptation with minimal labeling effort: Border Sample Selection and Gradient Norm-based Debiasing.

Border Sample Selection

Standard active learning often selects the most "uncertain" samples (those with high entropy). In a TTA context, these samples are often too far from the model's current knowledge to be useful for immediate learning. EATTA instead targets "border samples"—data points situated between the source and target distributions. To identify these, the framework uses feature perturbation: a small amount of noise is added to a sample's features. If the model's prediction changes significantly, the sample is identified as being on the decision boundary and highly informative for stable adaptation.

Gradient Norm-based Debiasing

A significant challenge in "effortless" labeling is the mathematical imbalance between one perfectly labeled sample and dozens of unlabeled ones in a single batch. The gradient signal from the supervised sample is typically much stronger than the noisy gradients from unlabeled data, causing the model to overfit to that single point. EATTA solves this by calculating the magnitude (norm) of the gradients for both supervised and unsupervised losses and applying dynamic weights to balance their influence, ensuring stable optimization across the entire batch.

The EATTA architecture detailing the feature extractor, the perturbation-based selection process (Eq. 3), and the gradient norm-based debiasing mechanism (Eq. 4 & 5).

Experimental Results

The researchers evaluated EATTA across several rigorous benchmarks for distribution shifts, including ImageNet-C (synthetic corruptions), ImageNet-R/K/A (real-world shifts like sketches and adversarial examples), and PACS (domain adaptation across styles).

  • Superior Accuracy: On ImageNet-C, EATTA reduced the average error rate to 53.8%, significantly outperforming the popular TENT method (70.9%) and previous active methods like SimATTA (54.8%), despite using a smaller annotation budget.
  • Efficiency and Robustness: EATTA's basic version outperforms state-of-the-art methods that require three times as many annotations. Furthermore, the system remains stable even when the budget is reduced to one label every five batches, whereas competing methods see performance plummet under such constraints.
  • Buffer-Free Operation: Unlike previous methods that rely on memory-intensive buffers to store labeled samples, EATTA maintains high performance without one, making it ideal for deployment on resource-constrained edge devices.

Conclusion

EATTA demonstrates that the most "confusing" samples are not always the most informative. By focusing on feasible "border samples" and balancing gradient influence, EATTA provides a stable, low-effort path for long-term model adaptation. Future work may involve integrating large foundation models as automated oracles to remove the human from the loop entirely and developing more sophisticated strategies for handling long-tailed class imbalances at test time.

Summary updated

Abstract

Long-term test-time adaptation (TTA) is a challenging task due to error accumulation. Recent approaches tackle this issue by actively labeling a small proportion of samples in each batch, yet the annotation burden quickly grows as the batch number increases. In this paper, we investigate how to achieve effortless active labeling so that a maximum of one sample is selected for annotation in each batch. First, we annotate the most valuable sample in each batch based on the single-step optimization perspective in the TTA context. In this scenario, the samples that border between the source- and target-domain data distributions are considered the most feasible for the model to learn in one iteration. Then, we introduce an efficient strategy to identify these samples using feature perturbation. Second, we discover that the gradient magnitudes produced by the annotated and unannotated samples have significant variations. Therefore, we propose balancing their impact on model optimization using two dynamic weights. Extensive experiments on the popular ImageNet-C, -R, -K, -A and PACS databases demonstrate that our approach consistently outperforms state-of-the-art methods with significantly lower annotation costs. Code is available at: \href https://github.com/flash1803/EATTA https://github.com/flash1803/EATTA .
BibTeX
@inproceedings{wang2025effortless,
  author = {Guowei Wang and Changxing Ding},
  title = {{Effortless Active Labeling for Long-Term Test-Time Adaptation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02387},
}
Figure