Abstract
Fully Test-Time Adaptation (TTA) is the process of updating a pre-trained model using only a stream of unlabeled, shifted test data. While promising, existing TTA methods are often brittle when faced with realistic "in-the-wild" conditions such as small batch sizes and severe class imbalance. This paper provides a systematic dissection of TTA and identifies a "Bag of Tricks" (BoT)—including batch renormalization, class re-balancing, and confidence calibration—that establishes new state-of-the-art results and makes TTA significantly more robust for real-world deployment.
Introduction
Deep learning models frequently encounter distribution shifts in deployment, where test data differs from training data (e.g., weather effects or stylistic changes). Fully TTA aims to mitigate this on-the-fly without requiring source data or labels. However, standard TTA methods like Tent assume ideal conditions, such as large, perfectly balanced batches. In real-world scenarios, data often arrives in tiny batches or even single instances, leading to model collapse where the model performs worse after adaptation than before.
Methods
The authors evaluate and combine several orthogonal techniques to form the "Bag of Tricks" (BoT):
- Normalization Tweaks: Standard Batch Normalization (BN) fails on small batches due to high variance in statistics. The authors suggest using Batch Renormalization (BReN) or architectures using Group Normalization (GN) or Layer Normalization (LN).
- Class Re-balancing (DOT): Diversity-Oriented Test-time adaptation (DOT) estimates class frequencies on-the-fly and weights updates to prevent bias toward frequent classes in the stream.
- Reliable Sample Selection: An entropy-based filter selects only high-confidence samples for adaptation, improving stability and reducing computational overhead.
- Temperature Scaling: A simple calibration trick that adjusts output confidence to prevent the model from becoming overconfident in its own potentially incorrect predictions.
Key Insights
- The BatchNorm Trap: BN is the primary failure point for TTA at small batch sizes. When the batch size is 1 or 2, the statistics are so noisy they "poison" the model's weights. Switching to BReN or LN is a critical requirement for robust TTA.
- Architecture Robustness: Vision Transformers (ViTs) using Layer Normalization are inherently more robust to small batch sizes and class imbalance than traditional ResNets, even before applying TTA tricks.
- Synergy and Efficiency: The tricks are more than the sum of their parts. By using sample selection, the authors achieved superior results while only performing backpropagation on approximately 26% of the data, significantly lowering the power requirements for on-device adaptation.
Key Results
The BoT consistently outperformed previous state-of-the-art methods like Tent, SAR, and Delta across major benchmarks including ImageNet-C, ImageNet-R, and VisDA2017.
On ImageNet-C with a batch size of 1, a standard ResNet50 using Tent collapses to 0.14% accuracy. Applying the BoT brings that up to 20.31%. Across all batch sizes and architectures, BoT provided an average accuracy improvement of +17.08% over vanilla Tent on ResNet50-BN and +7.66% on Vision Transformers.
Conclusion
By identifying and combining simple yet effective tricks, this work makes Fully TTA practical for real-world applications where data streams are unpredictable and resource-constrained. The resulting "Bag of Tricks" establishes a new baseline for robust, efficient adaptation in the wild.
Summary updated