Introduction
Deep learning models are increasingly deployed across a spectrum of hardware, from high-performance servers to resource-constrained edge devices. However, a model optimized for one environment often performs poorly in another. Traditionally, Neural Architecture Search (NAS) has been the go-to solution for finding hardware-specific optimal models. Yet, the computational overhead of NAS—often requiring thousands of GPU hours—creates a significant barrier to entry for most researchers and engineers.
The Cost of Neural Architecture Search
Existing methods like Once-For-All (OFA) require training a massive "supernet" before the search process can even begin. Others, like LANA, necessitate the manual design and training of numerous alternative blocks. These approaches are computationally prohibitive and design-intensive, making hardware-aware optimization inaccessible for many.
Bespoke addresses these challenges by shifting the paradigm from designing new blocks to recycling existing, high-quality sub-networks from publicly available pretrained models. This "harvesting" approach significantly reduces the training time required to achieve competitive performance.
The Bespoke Methodology
The Bespoke framework operates through a three-stage pipeline designed to minimize both human design effort and computational cost.
1. Preprocessing and the "Model House"
Instead of a search space of arbitrary operations, Bespoke builds a "Model House" from a library of diverse, publicly available pretrained models (e.g., ResNet, MobileNet, EfficientNet). Using a Modified Depth-First Search (DFS), the framework identifies and extracts functional sub-networks from these models that can serve as alternatives for blocks in a target "teacher" model.
2. Sub-Network Training via Distillation
The harvested sub-networks are trained using Block-wise Knowledge Distillation. By learning to mimic the outputs of specific blocks in the teacher model, these alternatives converge rapidly—often requiring only a few epochs—because they already possess well-trained features from their original pretraining. A key technical innovation here is channel-wise masking, which ensures compatibility between sub-networks of different shapes and the teacher architecture.
3. Efficient Model Search
To find the optimal configuration for a specific device, Bespoke employs Simulated Annealing. It evaluates the real-world inference latency of various block combinations on the target hardware, searching for the architecture that maximizes accuracy within a defined latency budget.
Experimental Results
Bespoke was evaluated on CIFAR-100 and ImageNet-1K using a diverse "Model House" containing architectures like EfficientNet, ResNet50V2, and MobileNetV3.
- Performance: On ImageNet, Bespoke-EB2 achieved 78.61% Top-1 accuracy with a CPU latency of 17.55ms, outperforming the standard EfficientNet-B0 (77.19% accuracy at 18.63ms).
- GPU Acceleration: For GPU targets, Bespoke identified models approximately 1.6x faster than baseline EfficientNet-B2 with minimal accuracy loss.
- Computational Efficiency: While traditional NAS methods like OFA require ~1,205 training epochs, Bespoke completes its entire preprocessing and search pipeline in the equivalent of roughly 19 epochs.
Conclusion
Bespoke demonstrates that high-performance, hardware-aware models can be built efficiently by leveraging the wealth of existing pretrained architectures. By reducing the "cost of entry" for model optimization, Bespoke enables broader access to specialized neural networks for diverse deployment scenarios. Future work will explore expanding the "Model House" to support tasks like image segmentation and pose estimation.
Summary updated