Optimization-Based Neural Networks Compression

Younes Tahiri, Mohamed El Amine Seddik, Mohamed TamaazoustiICIP 2021 · OpenReview

Abstract

This paper presents a method for constructing a size compressed neural network with better or similar accuracy than a given dense neural network, therefore the compressed network requires less memory and computational resources. The presented method relies basically on learning successive mappings between the given dense neural network (teacher) hidden features and the size-compressed neural network (student) hidden features, where the latter is learned also to solve the initial task of the teacher network. The presented method is particularly compared to baselines where we specifically show that the additional learned mappings significantly improve the performance (accuracy and computation) of the student network.
BibTeX
@inproceedings{tahiri2021optimization,
  author = {Younes Tahiri and Mohamed El Amine Seddik and Mohamed Tamaazousti},
  title = {{Optimization-Based Neural Networks Compression}},
  booktitle = {ICIP},
  year = {2021},
  url = {https://openreview.net/forum?id=IAukpJW7Qma},
}

Loading related content...

Figure