TL;DR
This paper introduces a novel sample-based approach for detecting phone boundaries in speech, moving beyond traditional frame-based methods. By analyzing acoustic features at the individual sample level and employing a supervised neural network, the authors achieve state-of-the-art accuracy on the TIMIT corpus, significantly improving upon existing techniques.
What Problem Does This Paper Address?
Automatic phonetic segmentation, the process of identifying the boundaries between individual speech sounds (phones), is a fundamental challenge in speech processing. While methods exist for when the text transcription is known (phone boundary alignment), detecting these boundaries without prior text knowledge (phone boundary detection) is considerably more difficult. Existing frame-based approaches, which analyze speech features in discrete time windows, struggle to capture the rapid spectral changes crucial for precise boundary identification. This limitation leads to coarser temporal resolution and less accurate results, especially for rapid speech. This paper aims to overcome these limitations by proposing a more granular, sample-based analysis.
Methods
The proposed method consists of three main stages:
- Sample-based Acoustic Parameter Extraction: Instead of frame-based features like MFCCs, this work extracts features at the individual sample level. This includes:
- Sub-band Signal Envelopes: Six sub-band signal envelopes are extracted from the speech signal, processed through band-pass filters and Hilbert transforms.
- Sample-based KL Distance: This measures the similarity between adjacent signal segments (at sample
nandn+1) based on their normalized sub-band envelopes. It's a key indicator of spectral change. - Spectral Entropy: This measures the flatness of the speech power spectrum, also computed at the sample level.
- Boundary Candidate Pre-selection: To reduce computational load, a pre-selection step identifies potential boundary candidates. This is done using the sample-based KL distance, selecting samples where the KL distance is increasing and exceeds a certain threshold (
Th_KL). This effectively picks out points of significant acoustic change. - Supervised Neural Network for Final Detection: A Multi-Layer Perceptron (MLP) neural network is trained to make the final boundary decision from the pre-selected candidates. The MLP takes a rich feature vector as input, which includes:
- Features from the current boundary candidate (KL distance, spectral entropy, and their delta terms).
- Features from adjacent segments (average normalized sub-band envelopes).
-
Indicators for the first and last candidates in an utterance.
A novel iterative training procedure is introduced, which integrates target selection and MLP training. This process refines the target boundaries for the MLP by iteratively selecting the most probable candidate within a tolerance window around the manually labeled boundary.
Datasets
- TIMIT Speech Corpus: This is the primary dataset used for both training and testing the proposed algorithm. The paper notes the total number of samples and phone boundaries for both the training and testing portions of TIMIT.
Key Insights and Ideas
- Sample-level analysis is crucial: The core idea is that analyzing speech features at the individual sample level, rather than in fixed frames, allows for a much finer temporal resolution and better capture of rapid acoustic events, which are critical for accurate phone boundary detection.
- KL distance as a strong cue: The sample-based KL distance proves to be a highly effective feature for identifying points of significant spectral change, making it ideal for both pre-selection and as an input to the final detector.
- Iterative training for better targets: The proposed iterative target selection and MLP training method is a clever way to adapt the supervised learning process to the nuances of acoustic boundary detection, especially when manual labels might not perfectly align with objective acoustic measures.
- Bridging the gap between objective and subjective boundaries: The iterative training helps bridge the gap between the precise, objective acoustic changes detected by the algorithm and the more subjective, or potentially less consistent, manual labels.
Key Results
- Improved Accuracy: The proposed sample-based MLP detector achieved an Equal Error Rate (EER) of 13.2% for the training set and 15.1% for the test set on the TIMIT corpus. This represents a significant improvement over previous frame-based methods, with an approximate 20% EER reduction compared to a cited frame-based approach.
- High Boundary Detection Rate: At the EER operating point, 43.5% of detected boundaries were within 80 samples of the manual labels, and 88.2% were within 240 samples. This demonstrates excellent precision.
- Efficient Pre-selection: The pre-selection step significantly reduces the number of samples considered, with only about 0.21% of speech samples (1 in 476) being preselected as candidates.
- Qualitative Performance: The paper notes that the algorithm correctly detects boundaries that are often missed by frame-based methods, particularly short silences before fricatives and semivowels.
Limitations and Future Work
- Error Analysis: The paper acknowledges that some boundaries are inherently difficult to detect, especially those between sounds with slow acoustic transitions (e.g., vowel-to-vowel transitions). Miss detection rates were higher for certain phone categories like fricative-fricative and nasal-nasal boundaries.
- Future Work: While not explicitly detailed, the success of this sample-based approach suggests future work could explore:
- Applying this method to other speech tasks like speaker diarization or speech recognition.
- Investigating other sample-level acoustic features.
- Exploring different neural network architectures for the final detection stage.
- Evaluating performance on larger and more diverse speech corpora.
Summary updated