All for one: feature combination for highly channel-degraded speech activity detection

Martin Graciarena, Abeer Alwan, Dan Ellis, Horacio Franco, Luciana Ferrer, John H. L. Hansen, Adam Janin, Byung Suk Lee, Yun Lei, Vikramjit Mitra, Nelson Morgan, Seyed Omid Sadjadi, T. J. Tsai, Nicolas Scheffer, Lee Ngee Tan, Benjamin WilliamsInterspeech 2013 · OpenReview

Abstract

Speech Activity Detection (SAD) is a fundamental prerequisite for downstream speech processing tasks, yet its performance degrades significantly in noisy, channel-distorted environments. This paper presents a robust SAD system developed for the DARPA RATS program, which utilizes audio retransmitted through actual radio hardware. The core contribution is a feature fusion approach that combines standard Mel Frequency Cepstral Coefficients (MFCCs) with four specialized features designed to capture pitch and spectro-temporal patterns. Experimental results on the RATS corpus demonstrate that this "All for One" combination yields a 24.3% relative reduction in Equal Error Rate (EER) over a standard MFCC baseline, proving highly effective against extreme distortions like non-linear clipping and frequency shifts.

Introduction

Speech Activity Detection (SAD) serves as the critical "gatekeeper" in speech processing pipelines. Accurate detection of speech versus background noise is essential; failures at this stage propagate to downstream tasks such as translation or speaker identification. While SAD is largely considered a solved problem for clean audio, it remains a significant challenge for "channel-degraded" speech.

This research focuses on the DARPA Robust Automatic Transcription of Speech (RATS) program. The RATS dataset is uniquely challenging as it consists of audio retransmitted through physical radio hardware, introducing severe distortions including extreme band-limiting, non-linear clipping, frequency shifts, and high-energy "non-transmission bursts." Standard features like MFCCs often struggle in these environments, as rhythmic or high-energy noise can easily be mistaken for human speech.

Methodology

The proposed system utilizes a smoothed log-likelihood ratio (LLR) between two Gaussian Mixture Models (GMMs)—one representing speech and the other non-speech. The primary innovation lies in the integration of four novel, specialized features with the standard 40-dimensional MFCC baseline:

  1. GABOR: Employs 59 spectro-temporal filters to detect "ripples" in the audio, processed via a Multi-Layer Perceptron (MLP) to generate a speech confidence measure.
  2. Combo: A 1-dimensional feature merging four voicing measures (Harmonicity, Clarity, Prediction Gain, and Periodicity) with Spectral Flux.
  3. SAcC (Subband Autocorrelation Classification): A noise-robust pitch tracker using an MLP to estimate the probability of a frame being unvoiced.
  4. MBCombF0: A multiband comb-filter approach that estimates voicing degrees by analyzing harmonic structures across frequency subbands.
Visualization of a degraded audio signal (WAV), its spectrogram, and the corresponding extracted features (GABOR, CRSS/Combo, SAcC, MBCombF0) used for detection.

To capture temporal context, a Discrete Cosine Transform (DCT) is applied over a 30-frame window, reducing long-range modulation into a compact feature vector for the GMM backend.

Experimental Setup

The study utilized the DARPA RATS corpus, which includes source material from existing corpora (e.g., Fisher English, Levantine Arabic) retransmitted through eight distinct analog radio channels (A through H). These channels cover a wide frequency range from 1MHz to 2.4GHz. Performance was evaluated on two sets: Dev-1 and Dev-2, with the latter characterized by lower Signal-to-Noise Ratios (SNRs).

Results and Discussion

The primary evaluation metric is the Equal Error Rate (EER). The results highlight the power of feature fusion. While individual novel features like GABOR performed worse than the MFCC baseline in isolation (4.00% vs 2.05% EER on Dev-1), their combination with MFCCs provided substantial gains.

Equal Error Rate (EER) results for various feature combinations on the Dev-1 and Dev-2 datasets, showing the performance gain from the 'All for One' fusion approach.

The "All for One" approach (MFCC + All 4 specialized features) achieved an EER of 1.55% on Dev-1 and 2.10% on Dev-2. This represents a 24.3% relative error reduction over the MFCC baseline on Dev-1. The success of this fusion stems from complementarity: MFCCs capture the spectral envelope, while the specialized features capture harmonic and temporal patterns that radio noise rarely mimics.

Conclusion

Combining diverse acoustic features is highly effective for SAD in severely degraded channels. While the current system uses 16 channel-dependent models, future work will focus on closing the performance gap on the most severely degraded channels and developing more universal models that do not require prior knowledge of the channel type.

Summary updated

BibTeX
@inproceedings{graciarena2013all,
  author = {Martin Graciarena and Abeer Alwan and Dan Ellis and Horacio Franco and Luciana Ferrer and John H. L. Hansen and Adam Janin and Byung Suk Lee and Yun Lei and Vikramjit Mitra and Nelson Morgan and Seyed Omid Sadjadi and T. J. Tsai and Nicolas Scheffer and Lee Ngee Tan and Benjamin Williams},
  title = {{All for one: feature combination for highly channel-degraded speech activity detection}},
  booktitle = {Interspeech},
  year = {2013},
  url = {https://openreview.net/forum?id=ES4s3ZIrSZiJ},
}
Figure