DARDA: Domain-Aware Real-Time Dynamic Neural Network Adaptation

Shahriar Rifat, Jonathan Ashdown, Francesco Restuccia

Abstract

Test Time Adaptation (TTA) has emerged as a practical solution to mitigate the performance degradation of Deep Neural Networks (DNNs) in the presence of corruption or noise affecting inputs. Existing approaches in TTA continuously adapt the DNN, leading to excessive resource consumption and performance degradation due to the accumulation of error stemming from a lack of supervision. In this work, we propose Domain-Aware Real-Time Dynamic Adaptation (DARDA) to address such issues. Our key approach is to proactively learn latent representations of specific corruption types, each associated with a sub-network state tailored to correctly classify inputs affected by that corruption. After deployment, DARDA adapts the DNN to previously unseen corruptions in an unsupervised fashion by estimating the latent representation of the ongoing corruption and selecting the closest pre-trained sub-network. Experiments on Raspberry Pi and NVIDIA Jetson Nano show that DARDA reduces energy consumption by 1.74x and average cache memory footprint by 2.64x compared to state-of-the-art methods, while improving accuracy across CIFAR-10, CIFAR-100, and TinyImagenet.

Introduction

Deep learning models are notoriously brittle when encountering environmental shifts such as fog, snow, or sensor noise. While Test-Time Adaptation (TTA) aims to update models "on the fly" using unlabeled data, current methods suffer from three major hurdles: high computational costs that drain edge device batteries, error accumulation over time due to unsupervised updates, and instability when faced with highly correlated (non-IID) data streams typical of real-world edge deployments. DARDA addresses these challenges by shifting from continuous, blind adaptation to a proactive, domain-aware strategy.

The DARDA Framework

DARDA introduces a library of pre-specialized model states and a mechanism to match the current environment to the most suitable state.

Overview of the DARDA framework architecture, illustrating the flow from corruption extraction to latent space matching and sub-network adaptation.

Core Components

  1. Corruption Extractor: A lightweight module that isolates "corruption features" (noise/weather) from "semantic features" (objects) by comparing differently downsampled versions of an image.
  2. Cross-Modal Latent Space: A shared mathematical space where corruption signatures and sub-network fingerprints are mapped. This allows the system to treat adaptation as a retrieval problem—finding the "closest" pre-trained sub-network for a new environment.
  3. Sub-Network Bootstrapping: The DNN is composed of specialized sub-networks. When a shift is detected, DARDA "plugs in" the most relevant weights rather than retraining from scratch.
  4. Label-Balanced Memory Bank: To maintain stability during non-IID data streams (e.g., a drone seeing only trees for a long period), a small buffer of diverse samples prevents the model from becoming biased toward frequently seen classes.

Experimental Evaluation

The framework was evaluated using CIFAR-10, CIFAR-100, and TinyImagenet datasets. The system was trained on 15 known corruption types and tested on 4 unseen corruptions (Gaussian blur, saturate, spatter, and speckle noise) to evaluate generalization. Hardware testing was conducted on the NVIDIA Jetson Nano and Raspberry Pi.

Results

DARDA demonstrates significant improvements in both performance and efficiency over baselines like TENT, CoTTA, and RoTTA.

Resource consumption increase relative to inference-only execution (left) and adaptation latency for unseen corruptions on edge devices (right).
  • Accuracy: DARDA improved classification accuracy by 10.4% on CIFAR-10 and 5.7% on CIFAR-100 compared to the next best method.
  • Efficiency: On mobile edge hardware, DARDA achieved a 1.74x reduction in energy consumption and a 2.64x smaller cache memory footprint. As shown in the resource consumption analysis, DARDA (NOTE) maintains significantly lower overhead compared to methods like CoTTA.
  • Stability: The system remained robust against non-IID data, avoiding the performance collapse common in standard TTA methods.

Conclusion

DARDA provides a resource-efficient path for deploying robust DNNs on edge devices. By decoupling corruption signatures from semantic content and utilizing a latent space for sub-network selection, it avoids the pitfalls of continuous unsupervised fine-tuning. Future work will explore extending this framework to object detection and autonomous long-term library expansion.

Summary updated

Abstract

Test Time Adaptation (TTA) has emerged as a practical solution to mitigate the performance degradation of Deep Neural Networks (DNNs) in the presence of corruption/ noise affecting inputs. Existing approaches in TTA continuously adapt the DNN, leading to excessive resource consumption and performance degradation due to accumulation of error stemming from lack of supervision. In this work, we propose Domain-Aware Real-Time Dynamic Adaptation (DARDA) to address such issues. Our key approach is to proactively learn latent representations of some corruption types, each one associated with a sub-network state tailored to correctly classify inputs affected by that corruption. After deployment, DARDA adapts the DNN to previously unseen corruptions in an unsupervised fashion by (i) estimating the latent representation of the ongoing corruption; (ii) selecting the sub-network whose associated corruption is the closest in the latent space to the ongoing corruption; and (iii) adapting DNN state, so that its representation matches the ongoing corruption. This way, DARDA is more resource efficient and can swiftly adapt to new distributions caused by different corruptions without requiring a large variety of input data. Through experiments with two popular mobile edge devices - Raspberry Pi and NVIDIA Jetson Nano - we show that DARDA reduces energy consumption and average cache memory footprint respectively by 1.74x and 2.64x with respect to the state of the art, while increasing the performance by 10.4%, 5.7% and 4.4% on CIFAR-10, CIFAR-100 and TinyImagenet.
BibTeX
@inproceedings{rifat2025darda,
  author = {Shahriar Rifat and Jonathan Ashdown and Francesco Restuccia},
  title = {{DARDA: Domain-Aware Real-Time Dynamic Neural Network Adaptation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00194},
}
Figure