Structured sparse support vector machine with ordered features

Kuangnan Fang, Peng Wang, Xiaochen Zhang, Qingzhao ZhangJournal of Applied Statistics 2020 · DOI

Abstract

In the application of high-dimensional data classification, several attempts have been made to achieve variable selection by replacing the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mi>ℓ</mml:mi> <mml:mrow><mml:mn>2</mml:mn></mml:mrow> </mml:msub> </mml:math> -penalty with other penalties for the support vector machine (SVM). However, these high-dimensional SVM methods usually do not take into account the special structure among covariates (features). In this article, we consider a classification problem, where the covariates are ordered in some meaningful way, and the number of covariates <i>p</i> can be much larger than the sample size <i>n</i>. We propose a structured sparse SVM to tackle this type of problems, which combines the non-convex penalty and cubic spline estimation procedure (i.e. penalizing second-order derivatives of the coefficients) to the SVM. From a theoretical point of view, the proposed method satisfies the local oracle property. Simulations show that the method works effectively both in feature selection and classification accuracy. A real application is conducted to illustrate the benefits of the method.
BibTeX
@article{fang2020structured,
  author = {Kuangnan Fang and Peng Wang and Xiaochen Zhang and Qingzhao Zhang},
  title = {{Structured sparse support vector machine with ordered features}},
  journal = {Journal of Applied Statistics},
  year = {2020},
  doi = {10.1080/02664763.2020.1849053},
}

Loading related content...

Figure