Semi Supervised Autoencoders: Better Focusing Model Capacity during Feature Extraction
Lists
Only you see this
Abstract
Previous work showed that unsupervised layer-wise pre-training can be used to overcome the difficulty of training a deep architecture. In this paper, we address one of the limitations when using unsupervised models like regularized autoencoders to learn features that we hope to be useful for a subsequent supervised task, namely their blindness to that specific task. We propose to change the cost function to focus on accurate reconstruction of input features that seem more useful for the supervised task. This is achieved by using A-norm in the minimized reconstruction error instead of Euclidean-norm. Through the choice of an appropriate A-matrix, the capacity of the model can be steered towards modeling relevant information for the classification task. Comparative experiments with the proposed denoising autoencoder variant show that this way of proceeding yields extracted features that achieve better classification performance on several datasets.
BibTeX
@inproceedings{almousli2013semi,
author = {Hani Almousli and Pascal Vincent},
title = {{Semi Supervised Autoencoders: Better Focusing Model Capacity during Feature Extraction}},
booktitle = {ICONIP},
year = {2013},
url = {https://openreview.net/forum?id=EyYuXPNgtF0},
}Loading related content...