Abstract
Test Time Adaptation (TTA) has emerged as a practical solution to mitigate the performance degradation of Deep Neural Networks (DNNs) in the presence of corruption or noise affecting inputs. Existing approaches in TTA continuously adapt the DNN, leading to excessive resource consumption and performance degradation due to the accumulation of error stemming from a lack of supervision. In this work, we propose Domain-Aware Real-Time Dynamic Adaptation (DARDA) to address such issues. Our key approach is to proactively learn latent representations of specific corruption types, each associated with a sub-network state tailored to correctly classify inputs affected by that corruption. After deployment, DARDA adapts the DNN to previously unseen corruptions in an unsupervised fashion by estimating the latent representation of the ongoing corruption and selecting the closest pre-trained sub-network. Experiments on Raspberry Pi and NVIDIA Jetson Nano show that DARDA reduces energy consumption by 1.74x and average cache memory footprint by 2.64x compared to state-of-the-art methods, while improving accuracy across CIFAR-10, CIFAR-100, and TinyImagenet.
Introduction
Deep learning models are notoriously brittle when encountering environmental shifts such as fog, snow, or sensor noise. While Test-Time Adaptation (TTA) aims to update models "on the fly" using unlabeled data, current methods suffer from three major hurdles: high computational costs that drain edge device batteries, error accumulation over time due to unsupervised updates, and instability when faced with highly correlated (non-IID) data streams typical of real-world edge deployments. DARDA addresses these challenges by shifting from continuous, blind adaptation to a proactive, domain-aware strategy.
The DARDA Framework
DARDA introduces a library of pre-specialized model states and a mechanism to match the current environment to the most suitable state.
Core Components
- Corruption Extractor: A lightweight module that isolates "corruption features" (noise/weather) from "semantic features" (objects) by comparing differently downsampled versions of an image.
- Cross-Modal Latent Space: A shared mathematical space where corruption signatures and sub-network fingerprints are mapped. This allows the system to treat adaptation as a retrieval problem—finding the "closest" pre-trained sub-network for a new environment.
- Sub-Network Bootstrapping: The DNN is composed of specialized sub-networks. When a shift is detected, DARDA "plugs in" the most relevant weights rather than retraining from scratch.
- Label-Balanced Memory Bank: To maintain stability during non-IID data streams (e.g., a drone seeing only trees for a long period), a small buffer of diverse samples prevents the model from becoming biased toward frequently seen classes.
Experimental Evaluation
The framework was evaluated using CIFAR-10, CIFAR-100, and TinyImagenet datasets. The system was trained on 15 known corruption types and tested on 4 unseen corruptions (Gaussian blur, saturate, spatter, and speckle noise) to evaluate generalization. Hardware testing was conducted on the NVIDIA Jetson Nano and Raspberry Pi.
Results
DARDA demonstrates significant improvements in both performance and efficiency over baselines like TENT, CoTTA, and RoTTA.
- Accuracy: DARDA improved classification accuracy by 10.4% on CIFAR-10 and 5.7% on CIFAR-100 compared to the next best method.
- Efficiency: On mobile edge hardware, DARDA achieved a 1.74x reduction in energy consumption and a 2.64x smaller cache memory footprint. As shown in the resource consumption analysis, DARDA (NOTE) maintains significantly lower overhead compared to methods like CoTTA.
- Stability: The system remained robust against non-IID data, avoiding the performance collapse common in standard TTA methods.
Conclusion
DARDA provides a resource-efficient path for deploying robust DNNs on edge devices. By decoupling corruption signatures from semantic content and utilizing a latent space for sub-network selection, it avoids the pitfalls of continuous unsupervised fine-tuning. Future work will explore extending this framework to object detection and autonomous long-term library expansion.
Summary updated