MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection

Zhenhong Sun, Ming Lin, Xiuyu Sun, Zhiyu Tan, Hao Li, Rong Jin

TL;DR

MAE-DET is a "zero-shot" Neural Architecture Search (NAS) framework that designs state-of-the-art object detection backbones in less than one GPU day without training a single weight. By maximizing the Multi-Scale Entropy of a network's features, it discovers architectures that are significantly faster and more accurate than standard ResNets.

What Problem Does This Paper Address?

Designing the "backbone" (the feature extractor) for object detection is a massive bottleneck. While Neural Architecture Search (NAS) can automate this, traditional NAS is computationally ruinous, often requiring thousands of GPU hours because it must train hundreds of candidate networks to see which performs best.

Furthermore, existing "zero-shot" NAS methods—which rank networks without training—were designed primarily for image classification. These methods often fail for object detection because detection requires extracting features at multiple scales to find both tiny and massive objects. Simply picking a good classification backbone often leads to sub-optimal detection performance.

Methods

The authors introduce MAE-DET (Maximum-Entropy Detection), which relies on three core innovations:

  1. Maximum Entropy Principle: Instead of training, MAE-DET treats a neural network as an information processing system. According to information theory, a system's capacity is maximized when its entropy is maximized. The authors use the Gaussian upper bound of differential entropy, which allows them to estimate a network's potential by simply looking at the variance of its feature maps after a single forward pass with random weights.
  2. Multi-Scale Entropy Prior (MSEP): Since object detection relies on feature pyramids (stages C1 through C5), MAE-DET doesn't just look at the final layer. It calculates a weighted sum of entropies across different stages.
Multi-Scale Entropy Prior

They found that weighting the final, high-context stage (C5) more heavily—specifically using a ratio of $(0, 0, 1, 1, 6)$ for stages C1-C5—is the key to finding backbones that excel at detection.

  1. Coarse-to-Fine Evolution: They use an evolutionary algorithm that first mutates block types and depths (coarse) and then narrows down to fine-tuning kernel sizes and widths (fine) to stay within a specific computational budget (FLOPs).

Datasets

The researchers evaluated MAE-DET across several standard computer vision benchmarks:

  • MS COCO: The primary dataset for training and evaluating object detection and instance segmentation.
  • Pascal VOC: Used to test the transferability of the searched models.
  • Cityscapes: Used to evaluate performance on urban scene understanding and semantic segmentation.
  • ImageNet-1k: Used to verify the classification performance of the discovered backbones.

Key Insights and Ideas

  • Training is Optional for Ranking: You don't need to know the "correct" weights to know if an architecture has the capacity to learn. A network that produces high-entropy (high-information) features with random weights is fundamentally a better "pipe" for data.
  • The "Simple is Better" Doctrine: MAE-DET uses standard convolutional blocks. It proves that optimizing the macro-structure (how wide and deep each stage is) provides more gains than adding complex, hand-engineered "bells and whistles."
  • Detection Needs Context: The most surprising finding was that single-scale entropy (common in classification NAS) leads to "over-deep" models that ignore early-stage features. Weighting the C5 stage heavily ensures the backbone captures enough global context for complex detection tasks.

Key Results

  • Speed: The search takes only 0.6 GPU days. This is roughly 50x to 100x faster than previous training-based NAS methods like DetNAS or SP-NAS.
  • Accuracy vs. ResNet: Compared to a ResNet-50 backbone, MAE-DET achieves a +2.0% mAP (mean Average Precision) boost on COCO while using the same number of FLOPs and parameters.
  • Inference Efficiency: At the same accuracy level as ResNet-50, MAE-DET is 1.54x faster on an NVIDIA V100 GPU.
Performance vs. FLOPs and Speed
  • SOTA Performance: MAE-DET-L achieved 48.0% mAP on COCO, outperforming many manually designed and NAS-generated backbones.

Limitations and Future Work

  • Gaussian Assumption: The entropy estimation relies on the assumption that feature distributions are roughly Gaussian. While mathematically convenient, real-world feature distributions are likely more complex.
  • Heuristic Weighting: The $\alpha$ weights for the multi-scale entropy (MSEP) were determined through empirical correlation analysis. A more automated way to determine these weights for different tasks (e.g., medical imaging vs. satellite imagery) remains an open question.
  • Search Space Constraints: The study primarily focused on ResNet-like and MobileNet-like search spaces; exploring more radical architectural paradigms (like Transformers) using the Maximum Entropy Principle is a suggested next step.

Summary updated

Abstract

In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the hel...
BibTeX
@inproceedings{sun2022mae,
  author = {Zhenhong Sun and Ming Lin and Xiuyu Sun and Zhiyu Tan and Hao Li and Rong Jin},
  title = {{MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=gRaiu4I7RRS},
}
Figure