Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge

Aditya Rajagopal, Christos-Savvas Bouganis

Abstract

This paper introduces ADaPT, a framework for on-device adaptation of Convolutional Neural Networks (CNNs) on edge hardware like the NVIDIA Jetson TX2. By utilizing local data distributions to guide "data-aware" pruning, the authors achieve significant accuracy gains—up to 10.2 percentage points over standard methods—while reducing memory usage by 90% and doubling inference speed. ADaPT automates complex pruning dependencies in modern architectures, enabling models to autonomously improve their performance in specific deployment environments without cloud connectivity.

Introduction

Most contemporary AI models are developed in cloud environments, trained on massive, curated datasets, and deployed to edge devices as static entities. However, real-world deployments—such as drones in forests or sensors in urban centers—often encounter data distributions that differ significantly from the training set. Moving this edge data back to the cloud for model updates presents significant hurdles regarding user privacy and the bandwidth costs associated with high-resolution sensor data.

The challenge lies in enabling true Edge Intelligence: the ability for a model to perceive its specific environment and autonomously improve its own architecture and weights locally. This paper addresses this by proposing a framework for on-device adaptation, allowing models to fine-tune and prune themselves directly on edge hardware like the NVIDIA Jetson TX2.

The ADaPT Framework

The authors propose a methodology termed Data-aware Pruning and Retraining (DaPR), implemented in the open-source ADaPT framework. The framework focuses on structured pruning, which removes entire convolutional filters rather than individual weights. This approach ensures immediate inference speedups on standard GPUs and CPUs without requiring specialized hardware.

Architectural units like Residual Blocks and Depthwise Separable Units handled by ADaPT's Pruning Dependency Calculation.

A core innovation of ADaPT is the Pruning Dependency Calculation (PDC). In modern architectures like ResNet or MobileNet, layers are often interconnected via residual skips or depthwise separable convolutions. Pruning a filter in one layer necessitates pruning corresponding filters in connected layers to maintain architectural integrity. PDC automates this complex mapping, which was previously a manual and error-prone task. Furthermore, ADaPT employs a binary search strategy to identify the optimal pruning level—the smallest possible model that maintains a target accuracy on local data—using L1-norm ranking to identify and remove filters with the lowest "energy."

Experimental Methodology

To evaluate the framework, the researchers utilized the CIFAR-100 dataset, modified to simulate diverse edge deployment scenarios. The 100 classes were grouped into five distinct subsets: Aquatic (fish and aquatic mammals), Indoor (furniture and appliances), Outdoor (trees, people, and vehicles), Natural (flowers, insects, and reptiles), and a Random control mix. Models were initially trained on the full dataset and then "deployed" to these subsets to measure their ability to adapt to niche environments.

Results and Discussion

Experiments conducted on the NVIDIA Jetson TX2 revealed that data-aware pruning is a significant performance driver. On average, the DaPR approach yielded a 10.2 percentage point (pp) increase in accuracy over standard, data-agnostic pruning methods. In extreme cases, accuracy improvements reached as high as 42.0pp. Beyond accuracy, the framework achieved up to 2.22x faster inference and a 90% reduction in memory footprint.

Search time vs. inference time tradeoff for AlexNet on the Random subset.

The search algorithm proved highly efficient, with search times per minibatch remaining well under a minute on edge hardware. The results also highlighted varying architecture sensitivities; while older architectures like AlexNet are relatively rigid, modern topologies like MobileNetV2 show significant changes in optimal topology when fine-tuned on specific subsets. Ultimately, the paper demonstrates an "Oracle" effect: even the least effective data-aware strategy often outperforms the best data-agnostic strategy, proving that local context is more valuable than sheer model scale.

Conclusion

While ADaPT demonstrates the power of on-device adaptation, several avenues for future work remain. The current framework assumes the availability of ground-truth labels at the edge, which might require a larger "teacher" model or high-confidence initial passes. Additionally, while L1-norm ranking is computationally efficient, more sophisticated importance metrics could further optimize results for complex architectures. Future validation on larger-scale datasets like ImageNet or real-world video streams will be essential for industrial scaling.

Summary updated

BibTeX
@inproceedings{rajagopal2020now,
  author = {Aditya Rajagopal and Christos-Savvas Bouganis},
  title = {{Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00364},
}
Figure