Class-Conditional Signature Filtering for Noisy Sequence Training Sets

Training sets of time series and speech are often dirty. Some recordings carry the wrong label. Some had a sensor lead come loose partway through. Some come from automatic alignment that cut a segment in the wrong place or stitched sub-events together in the wrong order. Older speech work showed that removing a few percent of such samples before training pays off. Rejecting misaligned utterances brought recognition error back to clean-data levels, and pruning abnormal training scores lowered speaker-verification error. The same work also left two problems open. Its detectors judge frames or summary statistics, so they cannot see corruption that lives in the order of events. And how much to remove is set by hand-picked percentiles, so pushing past a few percent silently throws away valid but unusual data.

We propose a filter that addresses both problems. It describes each training sequence by its path signature, a fixed set of features computed from iterated integrals of the multi-channel trajectory. Level one of these features records net change per channel; level two records which channel moved before which. The filter scores each sequence by how well it fits the class it is labeled as, compared with the best competing class. It then removes sequences until an estimated rate of wrongly discarded clean data reaches a target the user chooses. The claim is concrete. On corruptions that preserve the values of a sequence but break their order or timing, this filter improves a fixed downstream classifier more than three alternatives: pointwise filters, the classifier's own training loss, and doing nothing. It does so while keeping the share of discarded clean samples near the chosen rate.

What exists and what is missing

Two papers already put signatures to work on anomalies. Signature Isolation Forest builds isolation trees that split on signature coordinates or a signature kernel. It shows that functional isolation forests fail on "swap" anomalies, where two sub-events trade places but the marginal values stay the same, while the signature version separates them. Novelty detection on path space trains a one-class support vector machine on the expected signature of normal data, with concentration bounds on false-alarm rates. Both papers motivate this proposal and supply candidate scorers. Both are also class-agnostic and evaluation-time only. They flag strange test sequences; they never clean a training set or measure what a downstream classifier gains. A mislabeled but otherwise ordinary sequence is not strange globally, only relative to its claimed class, so a class-agnostic detector is the wrong tool for training-set cleaning. That is the step we add.

The filtering literature supplies the protocol and the warnings. Outlier detection for acoustic model training using robust statistics finds three things that shape our design. Sequence-level rejection beats frame-level rejection. Word error is lowest at about a 3% removal rate even though distance to the clean model keeps falling until about 15%, so over-pruning costs accuracy. Restricting rejection to untrusted additional data while keeping certified clean data helps. Pruning abnormal data for better making a decision in speaker verification prunes by first-order deviation from the mean and names automatic stopping as unsolved. Investigating Text Simplification Evaluation prunes the top 2–5% of edit distances. On the medical simplification corpus, which legitimately needs large edits, the same fixed percentile removed valid pairs. Fixed percentiles do not transfer across datasets. These findings constrain us: removal must be targeted, rate-controlled, and judged by downstream metrics, not by detection scores alone.

Core mechanism

Each training sequence $x_i$ with label $y_i$ is mapped to a truncated log-signature $\phi(x_i)$, a compressed form of the signature that keeps the same information with fewer coordinates. For wide inputs such as 40-channel filterbanks, we first project to a few channels, because signature size grows as $d^M$ for $d$ channels and truncation depth $M$. For each class $c$ we fit a robust density model $D_c$ in signature space. The default is a minimum-covariance-determinant Mahalanobis model, which estimates the mean and covariance while ignoring the most outlying points; a per-class Signature Isolation Forest is the alternative. The filtering score is the margin

$$s_i = D_{y_i}(\phi(x_i)) - \min_{c \neq y_i} D_c(\phi(x_i)),$$

where $D_c(\cdot)$ is the anomaly score under class $c$'s model. A large $s_i$ means the sequence fits its own label poorly and some other class well, the signature of label noise. A sequence that fits no class well also scores high, the signature of corruption.

To set how much to remove, we follow the acoustic-modeling practice of keeping a small trusted subset. We assume a few percent of each class has been verified, and convert each score to a conformal p-value against that subset. Conformal calibration here means ranking each score among the trusted scores. We remove sample $i$ when

$$p_i = \frac{1 + \#\{j \in \text{trusted}: s_j \ge s_i\}}{1 + n_{\text{trusted}}} \le \alpha.$$

Under exchangeability of clean samples with the trusted ones, this discards at most a fraction $\alpha$ of clean data in expectation. That is the false-pruning control earlier pipelines lacked. When no trusted subset exists, we fall back on the tail bounds of Novelty detection on path space and test how far its regularity assumptions survive real sensor dropouts.

Finally, signatures ignore the speed at which a path is traversed unless a time channel is added. We therefore compute the score twice: once without time and once with a time channel scaled by $\tau$, as in Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances. A sequence flagged only by the time-augmented score is a duration or timing distortion, the kind of error targeted by Impact of durational outlier removal from unit selection catalogs. A sequence flagged by both is a change in shape or order. This split is diagnostic and gives the user control over which kind of corruption to remove.

Research Program

We first build the filter and a corruption-injection harness. The harness applies four kinds of corruption at 5% and 10% rates:

  • Label flips: the label is replaced with a wrong class.
  • Segment swaps: two sub-segments of a sequence exchange places, preserving all values.
  • Local time-warps: part of a sequence is stretched or compressed.
  • Spike or dropout bursts: short non-physiological spikes or missing stretches are inserted.

We then hold downstream classifiers fixed and vary only the training set they see. The classifiers are Rough Transformers, which feed local and global signatures of patched windows into attention, and MambaSL, a state-space classifier. MambaSL is the strongest classifier in the file on the UEA archive, a standard collection of multivariate time-series classification datasets. The analysis asks three questions: which corruption types each filter catches, whether the realized clean-removal rate tracks $\alpha$, and whether the time/no-time split correctly attributes timing corruptions.

Baselines

The naive reference is training on the full contaminated set with no filtering, together with random removal at the same fraction the signature filter removes. Random removal separates the benefit of removing the right samples from the benefit of removing some samples.

The strongest classical competitor is model-based self-filtering. The downstream classifier itself, or a likelihood model as in Method B of Outlier detection for acoustic model training using robust statistics and the iterative purging of The 2013 KIT Quaero speech-to-text system for French, scores training samples by loss after a warm-up period, and the highest-loss samples are dropped. We run it with the same architecture, the same warm-up compute as the signature filter's fitting cost plus one classifier epoch, and the same conformal stopping rule on its scores. This last point matters: any advantage then comes from the score, not the threshold.

The pointwise competitor is an isolation forest and a robust covariance model on per-sequence summary statistics (mean, variance, extrema, kurtosis per channel). This is the reviewer's "just flatten it" option.

The two existing signature detectors are the class-agnostic Signature Isolation Forest and the signature one-class SVM, both used as training filters.

Three ablations isolate the key ingredients. The first is class-agnostic versus class-conditional scoring. The second truncates at level one (net displacement only) versus level three or more, which tests whether order information matters. The third compares scoring with and without the time channel.

For reference numbers from the literature:

All filters get an equal hyperparameter-search budget on a held-out validation split that is never filtered. "Beating" the strongest competitor means two things. First, the signature filter must recover a statistically larger share of the gap between noisy-trained and clean-trained accuracy than loss-based filtering, by paired comparison across datasets and seeds, on swap and time-warp corruption. Second, it must not do worse on label flips.

Why not just…?

Why not just filter on the classifier's own loss? This is the real threat. Loss-based filtering already worked in speech recognition and needs no new machinery. For pure label flips on easy classes, we expect it to match or beat our filter, and the experiments are built to show this honestly. Our case rests on two weaknesses of loss-based filtering. An expressive classifier memorizes corrupted samples quickly, so their loss stops standing out. And a classifier whose patching or pooling partly discards order may not penalize a swapped sequence at all. If loss-based filtering with the same conformal stopping rule matches us on swap and warp corruptions too, the contribution shrinks to the stopping rule and the cost profile, and we will say so.

Why not just run an isolation forest on summary statistics? Swap and warp corruptions leave per-channel marginal statistics unchanged by construction. Signature Isolation Forest already shows that dictionary-based and pointwise detectors cannot separate swap anomalies, and frame-level rejection lost to sequence-level rejection in acoustic training. The level-one ablation tests this directly. It is nearly a summary-statistics filter, and if it performs as well as deeper truncation, order information is not doing the work.

Why not just train on everything and let a large model absorb the noise? The no-filter baseline answers this per dataset. The acoustic results, where error fell from 7.22% to 6.94% after removing only 1% of naturally contaminated data, suggest the noise is not absorbed. The answer may differ for modern sequence models, which is why no-filtering is reported alongside every result rather than assumed to lose.

Why not just use a fixed 3–5% cutoff? The text-simplification and acoustic results show the best fraction moves across datasets: it was 3% for word error in one study, and the same percentile rule wrongly removed legitimate high-edit medical pairs in another. We report every filter both at fixed percentiles and under the conformal rule, and measure how far each fixed cutoff falls from that dataset's best fraction.

Proposed Experiments

The controlled study runs on all 30 UEA datasets and on the uWave and HHAR activity-recognition datasets used by CPDA. Each dataset receives each corruption type at both rates. We report three metrics: detection AUROC for corrupted samples, downstream test accuracy relative to the clean-trained reference, and the realized clean-removal rate against $\alpha \in \{0.01, 0.03, 0.05\}$.

The natural-contamination study covers two datasets with no ground-truth corruption labels, so it can be judged only by downstream metrics:

  • PhysioNet 2019 sepsis: lead disconnections and non-physiological spikes are known to occur in the records. We filter training patients and retrain the LightGBM pipeline, judged by cross-validated utility relative to 0.430.
  • IEMOCAP: we filter training utterances under leave-one-speaker-out evaluation, judged by unweighted accuracy relative to 53.03%. This tests whether the noise the original authors blamed for their accuracy plateau is removable.

Three outcomes would settle the claim:

  1. On segment-swap corruption, the class-conditional signature filter must beat both loss-based filtering and the summary-statistics forest in downstream accuracy. Otherwise the order-sensitivity argument fails.
  2. The realized clean-removal rate must stay within a small tolerance of $\alpha$ across datasets. Otherwise the stopping rule offers no protection for valid but unusual data.
  3. Corruptions flagged only by the time-augmented score must be predominantly the injected time-warps. Otherwise the shape-versus-timing split is not usable as a diagnostic.