PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking

Zekai Shao, Yufan Hu, Bin Fan, Hongmin LiuCVPR 2025 · DOI

Abstract

RGB-T tracking leverages visible and thermal modalities to maintain target visibility under adverse conditions. However, domain shifts between training and testing environments often degrade performance. Existing test-time adaptation (TTA) methods are either computationally prohibitive for real-time applications or prone to model drift. We present PURA (Parameter Update-Recovery Adaptation), a framework that enables real-time, stable adaptation. PURA utilizes Fast Parameter Updates (FPU) via BatchNorm statistics and a Parameter Decomposition Recovery (PDR) mechanism based on Singular Value Decomposition (SVD) to maintain core tracking knowledge. Experimental results on major benchmarks like RGBT234 and LasHeR demonstrate that PURA achieves state-of-the-art performance (93.3% MPR) at 42 FPS, effectively balancing speed and stability.

Introduction

RGB-T tracking is essential for robust computer vision in challenging environments. Despite its potential, trackers often suffer from domain shift—discrepancies between the training distribution and real-world test scenarios. Traditional Test-Time Adaptation (TTA) strategies, such as entropy minimization, rely on backpropagation, which is too slow for high-speed tracking. Conversely, simpler update methods often lead to "model drift," where the tracker overfits to noisy test samples and loses its original discriminative power.

Comparison of TTA paradigms: (a) Entropy Minimization, (b) Consistency Regularization, and (c) the proposed PURA framework featuring Fast Update and Gradual Recovery.

PURA addresses these challenges by introducing a dual-path adaptation strategy. It separates immediate environmental alignment from long-term model stability, ensuring the tracker remains both fast and accurate throughout long sequences.

Methodology

The PURA framework is built on three pillars designed to optimize the trade-off between adaptation speed and stability.

Fast Parameter Update (FPU)

To achieve real-time performance, PURA avoids heavy gradient computations. Instead, it performs FPU by updating the mean and variance statistics within the model's BatchNorm (BN) layers. By aligning these internal distributions with the current test samples, the model quickly adapts to new lighting or background conditions at a negligible computational cost.

Parameter Decomposition Recovery (PDR)

Frequent updates can cause the model to "forget" its pre-trained capabilities. PDR mitigates this by treating the history of parameter changes as "pseudo-gradients." Every $K$ frames, the system applies Singular Value Decomposition (SVD) to these changes to identify the principal update direction. This allows the model to recover critical parameters and discard noise, acting as a self-correcting mechanism against catastrophic forgetting.

The detailed architecture of PURA, illustrating the flow from RGB-T input through the Transformer block to the Prediction Head, where FPU and PDR mechanisms are applied.

Adaptive Momentum Scaling (AMS)

PURA employs AMS to dynamically adjust the trust placed in updates. By analyzing the consistency of update directions, AMS increases momentum when updates are stable and scales it back during erratic periods, preventing the model from being misled by outliers.

Experiments and Results

PURA was evaluated on several standard RGB-T benchmarks, including GTOT, RGBT210, RGBT234, and LasHeR.

Performance Benchmarks

On the RGBT234 dataset, PURA achieved a Maximum Precision Rate (MPR) of 93.3% and a Maximum Success Rate (MSR) of 70.3%. These results surpass recent state-of-the-art models like MambaVT and TBSI. Notably, PURA maintains a processing speed of 42 FPS, significantly faster than gradient-based TTA methods like Tent (~18 FPS).

Robustness to Domain Shift

In scenarios involving severe data corruption (e.g., noise or blur), PURA demonstrated a 3.9% performance improvement over non-adaptive baselines. This highlights its effectiveness in real-world "wild" scenarios where environmental conditions are unpredictable.

Discussion and Conclusion

PURA demonstrates that effective test-time adaptation does not require full backpropagation. By focusing on BatchNorm statistics and utilizing SVD for parameter recovery, the framework achieves a unique balance of speed and stability.

One limitation is the reduced benefit on very short video sequences, where there is insufficient data to establish a stable update trajectory. Future research may explore extending this "update-recovery" logic to other architectural components, such as LayerNorm in Transformers. Overall, PURA provides a robust and efficient solution for keeping RGB-T trackers on target in dynamic environments.

Summary updated

BibTeX
@inproceedings{shao2025pura,
  author = {Zekai Shao and Yufan Hu and Bin Fan and Hongmin Liu},
  title = {{PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02057},
}
Figure