Abstract
In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS). However, existing NAS methods for object detection require hundreds to thousands of GPU hours of searching, making them impractical in fast-paced research and development. In this work, we propose a novel zero-shot NAS method to address this issue. The proposed method, named MAE-DET, automatically designs efficient detection backbones via the Maximum Entropy Principle without training network parameters, reducing the architecture design cost to nearly zero yet delivering the state-of-the-art (SOTA) performance. Under the hood, MAE-DET maximizes the differential entropy of detection backbones, leading to a better feature extractor for object detection under the same computational budgets. After merely one GPU day of fully automatic design, MAE-DET innovates SOTA detection backbones on multiple detection benchmark datasets with little human intervention. Comparing to ResNet-50 backbone, MAE-DET is $+2.0\%$ better in mAP when using the same amount of FLOPs/parameters, and is $1.54$ times faster on NVIDIA V100 at the same mAP.
Introduction
In computer vision, the "backbone" of an object detection model is typically the most computationally expensive component, often accounting for over half of the total inference time. While Neural Architecture Search (NAS) has been employed to find more efficient backbones, traditional methods are prohibitively slow, requiring thousands of GPU hours to train and evaluate candidate networks.
Recent "zero-shot" NAS techniques, which rank networks without training, have successfully accelerated search for image classification. However, these methods often fail when applied to object detection. Unlike classification, which predicts a single global label, detection requires identifying multiple objects across various scales. Existing zero-shot proxies do not account for this multi-scale requirement, leading to sub-optimal architectures for detection tasks.
Methodology
MAE-DET (Maximum Entropy Detection) treats a neural network as an information processing system, leveraging the Maximum Entropy Principle. The core hypothesis is that a network's representational power is maximized when the entropy (information capacity) of its features is maximized.
Multi-Scale Entropy Prior (MSEP)
To estimate entropy efficiently, the authors use a Gaussian upper bound, allowing them to estimate differential entropy by calculating the variance of feature maps after a single forward pass of random noise through an untrained network.
Crucially for detection, MAE-DET introduces the Multi-Scale Entropy Prior (MSEP). Rather than evaluating only the final output, it calculates a weighted sum of entropies across five stages of the backbone (C1 through C5). This ensures the architecture is optimized to provide high information density for objects of all sizes (small, medium, and large).
Search Strategy
The framework employs a coarse-to-fine evolutionary search. It navigates a search space including block types, kernel sizes, depths, and widths. The search begins by broadly mutating all parameters and gradually narrows its focus to fine-tuning kernels and widths, efficiently identifying the optimal architecture within a specific FLOPs budget.
Experimental Results
The researchers evaluated MAE-DET on standard benchmarks including MS COCO, PASCAL VOC, and Cityscapes. The search process itself was remarkably efficient, taking only 14.4 GPU hours (approximately 0.6 days), compared to over 600 hours for training-based methods like DetNAS.
Performance and Efficiency
On the MS COCO dataset, the MAE-DET-M backbone achieved 46.8% mAP, representing a +2.1% improvement over a standard ResNet-50 backbone with similar FLOPs and parameters. Furthermore, at equivalent accuracy levels to ResNet-50, MAE-DET demonstrated a 1.54x speedup on an NVIDIA V100 GPU.
Transferability
The searched architectures showed strong transferability. When tested on PASCAL VOC, the MAE-DET backbone outperformed ResNet-50 by 4.1% mAP, indicating that the entropy-based search captures fundamental architectural strengths rather than over-fitting to a specific dataset.
Discussion and Conclusion
MAE-DET demonstrates that "simple is better": optimized standard convolutional blocks can outperform complex hand-crafted architectures like ResNet if their depth and width are balanced using entropy principles. It provides the first robust evidence that training-free zero-shot NAS can be successfully adapted for complex tasks like object detection by utilizing multi-scale proxies.
Future work may expand this approach to search the entire detection pipeline (including the FPN and detection heads) simultaneously and explore more sophisticated statistical models for entropy estimation beyond the Gaussian assumption.
Summary updated