A deep neural network approach for sentence boundary detection in broadcast news

Chenglin Xu, Lei Xie, Guangpu Huang, Xiong Xiao, Engsiong Chng, Haizhou Li

Abstract

Sentence boundary detection (SBD) is a critical preprocessing step for downstream natural language processing tasks such as machine translation and summarization. This paper presents a hybrid architecture combining a Deep Neural Network (DNN) and a Conditional Random Field (CRF) to improve SBD in broadcast news. By leveraging the DNN's ability to extract complex non-linear patterns from raw prosodic features and the CRF's capacity for sequential lexical modeling, the proposed system achieves a significant reduction in NIST SU error rates. Experimental results on NIST RT-04F and RT-03F datasets demonstrate a 16.7% relative error reduction on reference transcripts and improved robustness to automatic speech recognition (ASR) errors compared to traditional decision tree-based approaches.

Introduction

In the processing of spoken language, converting a continuous stream of words into structured text with sentence boundaries is essential for readability and subsequent computational tasks. This task, Sentence Boundary Detection (SBD), relies on two primary information sources: prosodic cues (the "music" of speech, such as pitch, energy, and duration) and lexical cues (the actual words and their sequences).

Traditional state-of-the-art SBD systems often utilize Decision Trees (DT) for prosodic modeling. However, DTs are limited in capturing the complex, non-linear relationships inherent in acoustic data. Furthermore, while Hidden Markov Models (HMMs) are common for sequence modeling, they often fail to optimize for the most likely label sequence as effectively as discriminative models. This paper addresses these limitations by introducing a hybrid DNN-CRF framework.

Methodology

The proposed system employs a two-stage hybrid approach to integrate prosodic and lexical information effectively.

The Prosodic DNN

The first stage utilizes a Deep Neural Network to process 162 raw prosodic features. Unlike rigid decision trees, the DNN learns a hierarchy of abstract representations, mapping acoustic features to the probability of an inter-word boundary. Through experimentation, the authors found that a DNN with 3 hidden layers and 80 units per layer provided the optimal balance of performance and generalization.

The Linear-Chain CRF

The posterior probabilities generated by the DNN are quantized into discrete bins and combined with lexical features, including word N-grams, Part-of-Speech (POS) tags, and syntactic chunks. These combined features are fed into a linear-chain Conditional Random Field (CRF). As a discriminative model, the CRF considers the entire word sequence to ensure global consistency in boundary predictions.

The hybrid DNN-CRF architecture for sentence boundary detection, integrating lexical features and prosodic DNN outputs.

Experimental Setup

The model was evaluated using English broadcast news data from the NIST RT-04F and RT-03F MDE evaluations. The training set comprised approximately 36 hours of audio, with development and test sets of 2 hours each. To assess real-world performance, the system was tested on both human-generated Reference (REF) transcripts and Automatic Speech Recognition (ASR) transcripts, which featured a 29.5% word error rate.

Results and Discussion

The hybrid DNN-CRF model significantly outperformed the baseline DT-CRF system across both transcript types, as shown in the performance summary below.

Comparison of Sentence Boundary Detection performance (P/R/F1 and NIST error) between Decision Tree (DT) and DNN approaches on REF and ASR transcripts.

Key Findings

  • Superior Feature Extraction: The DNN's ability to learn non-linear features from "raw" data provided an 11% error reduction over prosodic Decision Trees alone.
  • Robustness to Noise: The DNN proved more resilient to ASR errors. While ASR timing errors often "smear" prosodic data, the DNN's abstract representations handled these perturbations better than the rigid splits of a DT.
  • Sequential Context: Combining lexical "skeletons" (POS tags, N-grams) with acoustic "flesh" (DNN outputs) via the CRF significantly boosted overall performance, achieving a 16.7% relative reduction in error on reference transcripts.

Conclusion

This work demonstrates that a hybrid DNN-CRF architecture effectively captures the complexities of speech prosody and lexical context for sentence boundary detection. While the system shows high precision, a "recall gap" remains on ASR transcripts (dropping to approximately 49%) due to word errors confusing the lexical model. Future research will explore end-to-end training to eliminate manual quantization and investigate temporal architectures such as Recurrent Neural Networks (RNNs) or Convolutional Neural Networks (CNNs) to further refine boundary detection in spontaneous speech.

Summary updated

BibTeX
@inproceedings{xu2014deep,
  author = {Chenglin Xu and Lei Xie and Guangpu Huang and Xiong Xiao and Engsiong Chng and Haizhou Li},
  title = {{A deep neural network approach for sentence boundary detection in broadcast news}},
  booktitle = {Interspeech},
  year = {2014},
  doi = {10.21437/interspeech.2014-599},
}
Figure