Signs in the Lottery: Structural Similarities Between Winning Tickets
Lists
Only you see this
Abstract
Winning tickets are sparse subnetworks of a deep network that can be trained in isolation to the same performance as the full network. Winning tickets have been found in many different contexts, however their structural characteristics are not well understood. We propose that the signs of the connections in winning tickets play a crucial role. We back this claim by introducing a sign-based structural comparison metric that allows to distinguish winning tickets from other sparse networks. We further analyze typical (signed) patterns in convolutional kernels of winning tickets and find structures that resemble patterns found in trained networks.
BibTeX
@inproceedings{holler2023signs,
author = {Isabel Holler and Mats Leon Richter and Ulf Krumnack},
title = {{Signs in the Lottery: Structural Similarities Between Winning Tickets}},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://openreview.net/forum?id=3l9mLzLa0BA},
}Loading related content...