A Walkthrough for the Principle of Logit Separation

Gil Keren, Sivan Sabato, Björn W. Schuller

Abstract

We consider neural network training, in applications in which there are many possible classes, but at test-time, the task is a binary classification task of determining whether the given example belongs to a specific class. We define the Single Logit Classification (SLC) task: training the network so that at test-time, it would be possible to accurately identify whether the example belongs to a given class in a computationally efficient manner, based only on the output logit for this class. We propose a natural principle, the Principle of Logit Separation, as a guideline for choosing and designing loss functions that are suitable for SLC. We show that the Principle of Logit Separation is a crucial ingredient for success in the SLC task, and that SLC results in considerable speedups when the number of classes is large.
BibTeX
@inproceedings{keren2019walkthrough,
  author = {Gil Keren and Sivan Sabato and Björn W. Schuller},
  title = {{A Walkthrough for the Principle of Logit Separation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/861},
}

Loading related content...

Figure