Huber-Norm Regularization for Linear Prediction Models
Lists
Only you see this
Abstract
In order to avoid overfitting, it is common practice to regularize linear prediction models using squared or absolute-value norms of the model parameters. In our article we consider a new method of regularization: Huber-norm regularization imposes a combination of $$\ell _{1}$$ and $$\ell _{2}$$ -norm regularization on the model parameters. We derive the dual optimization problem, prove an upper bound on the statistical risk of the model class by means of the Rademacher complexity and establish a simple type of oracle inequality on the optimality of the decision rule. Empirically, we observe that logistic regression with Huber-norm regularizer outperforms $$\ell _{1}$$ -norm, $$\ell _{2}$$ -norm, and elastic-net regularization for a wide range of benchmark data sets.
BibTeX
@inproceedings{zadorozhnyi2016huber,
author = {Oleksandr Zadorozhnyi and Gunthard Benecke and Stephan Mandt and Tobias Scheffer and Marius Kloft},
title = {{Huber-Norm Regularization for Linear Prediction Models}},
booktitle = {ECML/PKDD},
year = {2016},
doi = {10.1007/978-3-319-46128-1\_45},
}Loading related content...