Support Vector Clustering for Outlier Detection

Hailei Wang, Wenbo Li, Bingyu Sun2012 · DOI

Abstract

In this paper a novel Support vector clustering(SVC) method for outlier detection is proposed. Outlier detection algorithms have application in several tasks such as data mining, data preprocessing, data filter-cleaner, time series analysis and so on. Traditionally outlier detection methods are mostly based on modeling data based on its statistical properties and these approaches are only preferred when large scale set is available. To solve this problem, in this paper we focus on establishing the context of support vector clustering approach for outlier detection. Compared to traditional outlier detection methods , the performance of the SVC is not sensitive to the selection of needed parameters. The experiment results proved the efficiency of our method.

Introduction

Outlier detection—identifying data points that deviate significantly from the norm—is a cornerstone of data mining and signal processing. Traditional methods often rely on statistical approaches that require a known probability distribution or massive datasets to estimate parameters accurately. Alternatively, neural networks are used but are notoriously difficult to tune and prone to local minima. This paper introduces a novel approach using Support Vector Clustering (SVC) to create a robust boundary for anomaly detection without requiring prior knowledge of the data's underlying probability distribution.

Methodology

The proposed method adapts Support Vector Clustering (SVC) for anomaly detection through several key steps:

  1. Feature Mapping: Data points are mapped into a high-dimensional feature space using a kernel function, allowing for the definition of simple geometric boundaries for complex data.
  2. Smallest Enclosing Sphere: The algorithm identifies the smallest sphere in this high-dimensional space that contains the majority of the data.
  3. Slack Variables: To handle noise, slack variables ($\zeta_i$) allow the sphere to exclude certain points, preventing outliers from distorting the boundary.
  4. Optimization: Formulated as a convex quadratic program, the method ensures a global solution, avoiding the local minima common in neural network training.
Figure 1: Visualization of outlier detection using SVC. The green boundary encloses normal data, while points A, B, and C are correctly identified as outliers.

Experimental Design

The method was evaluated using the IRIS dataset, a classic benchmark in pattern recognition. The Versicolor species served as the "normal" class, while samples from Setosa and Virginica were randomly introduced as outliers. Results were averaged over 10 experimental runs to ensure statistical significance.

Results and Analysis

The SVC algorithm naturally categorizes data into three groups: internal points, support vectors (lying on the boundary), and outliers. A key advantage of this approach is its focus on the boundary rather than the entire data density, making it efficient for high-dimensional datasets.

The researchers compared SVC against Nearest Neighbor Data Description (NND). As shown in the performance data below, SVC significantly outperformed NND in both outlier detection and target classification accuracy.

Figure 2: Performance comparison between SVC and NND. SVC maintains high detection rates as the number of outliers increases, whereas NND performance degrades.

As the number of outliers increased from 2 to 10, SVC's detection rate remained robust (reaching 8.0 for 10 outliers), while NND's performance dropped significantly (falling to 1.2). SVC also maintained higher accuracy for correctly identifying the "target" (normal) objects across all tests.

Discussion and Future Work

While SVC offers parameter stability and global optimization, its computational complexity can increase with the number of data points. Future research should explore the impact of different kernel functions (e.g., RBF vs. Polynomial) and evaluate the method on larger, more complex datasets to further validate its scalability.

Summary updated

Abstract

In this paper a novel Support vector clustering(SVC) method for outlier detection is proposed. Outlier detection algorithms have application in several tasks such as data mining, data preprocessing, data filter-cleaner, time series analysis and so on. Traditionally outlier detection methods are mostly based on modeling data based on its statistical properties and these approaches are only preferred when large scale set is available. To solve this problem, in this paper we focus on establishing the context of support vector clustering approach for outlier detection. Compared to traditional outlier detection methods , the performance of the SVC is not sensitive to the selection of needed parameters. The experiment results proved the efficiency of our method.
BibTeX
@misc{wang2012support,
  author = {Hailei Wang and Wenbo Li and Bingyu Sun},
  title = {{Support Vector Clustering for Outlier Detection}},
  year = {2012},
  doi = {10.2991/iccia.2012.83},
}
Figure