Large margin filtering for Signal Sequence Labeling

Rémi Flamary, Benjamin Labbé, Alain RakotomamonjyICASSP 2010 · DOI · OpenReview

Abstract

Signal Sequence Labeling consists in predicting a sequence of labels given an observed sequence of samples. A naive way is to filter the signal in order to reduce the noise and to apply a classification algorithm on the filtered samples. We propose in this paper to jointly learn the filter with the classifier leading to a large margin filtering for classification. This method allows to learn the optimal cutoff frequency and phase of the filter that may be different from zero. Two methods are proposed and tested on a toy dataset and on a real life BCI dataset from BCI Competition III.

Introduction

In signal sequence labeling—such as decoding brain activity (EEG) or segmenting speech—researchers face two major hurdles: noise and time-lags. Standard practice involves a two-step pipeline: first, apply a hand-tuned filter (like a low-pass or Savitzky-Golay filter) to clean the signal, then feed the result into a classifier. However, this is suboptimal. Manual filter tuning requires prior knowledge of the "correct" frequencies and often ignores the natural delay between a signal (e.g., a neuron firing) and the corresponding label (e.g., a physical movement).

This paper introduces Filter-SVM, a novel approach that seeks to make the filter "aware" of the classification task by automatically tuning the filtering stage—including its cutoff frequency and phase—as part of the machine learning process.

Methodology

The authors propose two main approaches to incorporate temporal context into Support Vector Machines (SVMs):

  1. Window-SVM (W-SVM): A baseline that considers a fixed time-window around each sample. It treats all samples in the window as features for a standard linear SVM. While it can adapt to time-lags, its complexity grows quadratically with the window size and number of channels, making it prone to overfitting.
  2. Filter-SVM: The core contribution. This method explicitly dissociates the FIR (Finite Impulse Response) filter coefficients from the classifier weights. It optimizes a cost function that jointly minimizes the SVM hinge loss while regularizing both the classifier and the filter.
Figure 6: Histograms showing the increased feature separation achieved by large margin filtering compared to no filtering.

Optimization

The joint optimization of the filter and classifier is non-convex. To solve this, the authors developed an iterative algorithm: for a fixed filter, the problem becomes a standard (convex) linear SVM; they then use gradient descent to update the filter coefficients based on the SVM's performance. This allows the model to discover the most discriminative frequencies and delays for each specific channel.

Experimental Evaluation

Datasets

  • Toy Dataset: A synthetic multi-channel signal where only a few channels were discriminative. It featured switching means, Gaussian noise, and artificial time-lags to test channel selection and delay compensation.
  • BCI Competition III Dataset: Real-world EEG data from three subjects. The task was to label brain activity as "left arm," "right arm," or "feet" movements based on 96 channels of Power Spectral Density (PSD) features.

Results

On the toy dataset, Filter-SVM demonstrated superior robustness to both noise and the inclusion of irrelevant channels. As shown in the performance plots, while standard SVM and HMM approaches degraded under high noise or high dimensionality, Filter-SVM maintained a low error rate.

Figure 5: Test error rates on the toy dataset as a function of noise level (sigma) and total number of channels (nbtot).

On the BCI Competition III dataset, Filter-SVM achieved an average test error of 0.2018. This is a significant improvement over a standard SVM (0.4123) and outperformed the top results from the original competition (0.3135) under comparable settings.

Key Insights and Discussion

  • Automatic Time-Lag Adaptation: Filter-SVM can learn a phase shift. If the brain signal precedes the label by 100ms, the learned filter shifts its weights to capture that temporal offset.
  • Implicit Channel Selection: By using a double-weighting scheme, the model effectively performs feature selection. Noisy or irrelevant channels receive near-zero weights.
  • Interpretability: The learned filters can be visualized as space-time maps, showing exactly which time-offsets and brain regions the model prioritizes.

Limitations

The current optimization is non-convex, meaning the algorithm might find a local minimum. Additionally, the implementation is currently limited to linear decision functions. Future work may involve extending this to non-linear kernels or multi-task learning frameworks.

Summary updated

Abstract

Signal Sequence Labeling consists in predicting a sequence of labels given an observed sequence of samples. A naive way is to filter the signal in order to reduce the noise and to apply a classification algorithm on the filtered samples. We propose in this paper to jointly learn the filter with the classifier leading to a large margin filtering for classification. This method allows to learn the optimal cutoff frequency and phase of the filter that may be different from zero. Two methods are proposed and tested on a toy dataset and on a real life BCI dataset from BCI Competition III.
BibTeX
@inproceedings{flamary2010large,
  author = {Rémi Flamary and Benjamin Labbé and Alain Rakotomamonjy},
  title = {{Large margin filtering for Signal Sequence Labeling}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2010},
  doi = {10.1109/icassp.2010.5495281},
}
Figure