Pruning Based Training-Free Neural Architecture Search

Jiawang Zhou, Fanglin Chen, Guangming LuICME 2022 · OpenReview

Abstract

Neural Architecture Search (NAS) plays an important role in searching for high-performance neural networks. How-ever, NAS algorithms are slow and require a terrific amount of computing resources, because they need to be trained on supernet or dense candidate networks to obtain information for evaluation. If the high-performance network architecture could be selected without training, it would eliminate a signif-icant part of the computational cost. Therefore, we propose a zero-cost metric called EX-score, which can represent the ex-pressivity of the network and rank the untrained architectures. To further reduce cost, we design a pruning based zero-cost neural architecture search framework (PZ-NAS) using EX-score. PZ-NAS can prune the initialised supernet rapidly and obtains hundreds of times faster speed performance, whilst archieving comparable accuracy property on CIFAR-IO and ImageNet.
BibTeX
@inproceedings{zhou2022pruning,
  author = {Jiawang Zhou and Fanglin Chen and Guangming Lu},
  title = {{Pruning Based Training-Free Neural Architecture Search}},
  booktitle = {ICME},
  year = {2022},
  url = {https://openreview.net/forum?id=2x-sogOO_n_},
}

Loading related content...

Figure