Abstract
Deep Neural Networks (DNNs) are increasingly deployed in safety-critical edge applications, where they face the dual challenges of limited hardware resources and vulnerability to adversarial attacks. Traditional optimization methods often address hardware efficiency and adversarial robustness in isolation, frequently leading to sub-optimal trade-offs. This paper presents RoHNAS, a novel Neural Architecture Search (NAS) framework designed to simultaneously optimize for accuracy, robustness, and hardware metrics (energy, latency, and memory). By expanding the search space to include both Convolutional Neural Networks (CNNs) and brain-inspired Capsule Networks (CapsNets), RoHNAS identifies Pareto-optimal architectures tailored for deployment on specialized edge accelerators.
Introduction
The transition of deep learning models from high-performance data centers to edge devices—such as autonomous drones, IoT sensors, and medical devices—is hindered by a "double-edged sword." First, DNNs are computationally intensive, often exceeding the power and memory constraints of small-scale hardware. Second, they are notoriously fragile; infinitesimal, human-imperceptible perturbations to input data (adversarial attacks) can cause catastrophic failures in model prediction.
Historically, researchers have attempted to mitigate these issues post-design. However, improving robustness often increases computational overhead, while aggressive hardware compression can degrade a model's resilience. RoHNAS addresses this by treating robustness and efficiency as conjoint optimization goals during the architectural design phase, navigating the resulting "search space explosion" through an evolutionary-based automated framework.
Methodology
RoHNAS utilizes a multi-objective evolutionary search to discover architectures that balance conflicting performance metrics. The framework is built upon several core pillars:
1. Multi-Objective Optimization
At its core, RoHNAS employs the Non-dominated Sorting Genetic Algorithm II (NSGA-II). This allows the framework to maintain a population of candidate architectures and evolve them toward the "Pareto front"—a set of designs where no single metric (e.g., energy) can be improved without degrading another (e.g., adversarial robustness).
2. Analytical Hardware Modeling
To avoid the prohibitive time cost of deploying thousands of candidate models to physical hardware, RoHNAS utilizes high-fidelity mathematical models. These models estimate energy consumption, latency, and memory footprint based on the specific operations of the network layers and the architectural constraints of the target hardware, specifically the CapsAcc accelerator. This approach allows for rapid evaluation while maintaining a high correlation with actual hardware performance.
3. Smart Perturbation Selection
Evaluating robustness is computationally expensive. RoHNAS optimizes this by analyzing the "Accuracy vs. Noise" curve for a given dataset. It identifies the "tipping point"—the adversarial perturbation value ($\epsilon$) where the model's accuracy begins to drop most sharply. By focusing evaluation on this critical threshold, the search algorithm receives the most informative feedback regarding an architecture's inherent resilience.
4. Accelerated Evaluation Strategy
To further reduce the search time (saving thousands of GPU hours), RoHNAS employs a fast evaluation strategy. Candidate models are initially trained for a limited number of epochs (5–10). The framework operates on the observation that early-stage performance is a reliable predictor of final fully-trained performance, allowing the algorithm to prune weak candidates early in the evolutionary process.
Experimental Results
The framework was validated using standard image classification benchmarks: MNIST, Fashion-MNIST, and CIFAR-10.
Hardware-Robustness Trade-offs
RoHNAS successfully identified models that outperform human-designed counterparts like ResNet-20 and the original CapsNet. For the CIFAR-10 dataset, the framework discovered a model achieving 86.07% accuracy with an energy consumption of 38.63 mJ, a memory footprint of 11.85 MiB, and a latency of 4.47 ms.
The Role of Capsule Networks
A significant finding of the study is that Capsule Networks (CapsNets) often exhibit superior robustness compared to CNNs for a given memory footprint. By including CapsNets in the NAS search space—a rarity in current NAS literature—RoHNAS was able to exploit these structural advantages for edge deployment.
Search Efficiency
The study demonstrated that the evolutionary search strategy significantly outperforms random search. In the MNIST experiments, RoHNAS-generated models remained robust at noise levels ($\epsilon$) nearly an order of magnitude higher than those identified by previous state-of-the-art methods like NASCaps.
Discussion and Limitations
While RoHNAS represents a significant step forward in conjoint optimization, several challenges remain. The computational cost of running NAS on larger-scale datasets like ImageNet remains high, necessitating the use of proxy tasks. Additionally, while the analytical hardware models are highly effective for ranking architectures, they currently underestimate absolute energy consumption by approximately 25% due to unmodeled interconnect overheads. Finally, while the framework primarily utilized Projected Gradient Descent (PGD) attacks, future iterations will explore a broader spectrum of adversarial techniques to ensure generalized robustness.
Conclusion
RoHNAS demonstrates that adversarial robustness and hardware efficiency do not have to be mutually exclusive. By integrating hardware-aware modeling with evolutionary search across both CNN and CapsNet architectures, the framework provides a robust pipeline for developing secure, efficient deep learning models for the next generation of edge computing devices.
Summary updated