A STUDY ON POLARITY DISTRIBUTIONS FOR NETWORK LEARNING

Aoife Igoe, Arindam Biswas, Biswajit BasuICLR 2024 · OpenReview

Abstract

Transfer learning is one of the most important techniques in modern deep learning. The knowledge gained from transferring weights helps networks to learn fast achieving high accuracy. Recent work has shown that transferring the polarity of the weights plays a fundamental role in transfer learning. In this work, we concentrate on the polarity distribution and study its effects on the learning accuracy. Our results on benchmark datasets show that only the knowledge of the polarity distribution (percentage of weights having polarity positive, negative or zero) is sufficient to achieve comparable accuracy within a short training period.
BibTeX
@inproceedings{igoe2024study,
  author = {Aoife Igoe and Arindam Biswas and BISWAJIT BASU},
  title = {{A STUDY ON POLARITY DISTRIBUTIONS FOR NETWORK LEARNING}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=0Ddq3BtHsH},
}

Loading related content...

Figure