Sentence segmentation of aphasic speech

Kathleen C. Fraser, Naama Ben-David, Graeme Hirst, Naida Graham, Elizabeth RochonNAACL 2015

TL;DR

This paper investigates how automatic speech recognition (ASR) errors and the unique speech patterns of individuals with aphasia affect sentence segmentation. The authors find that while standard segmentation methods can be adapted, certain syntactic complexity metrics are more robust to segmentation noise than others, making them promising for future diagnostic tools.

What Problem Does This Paper Address?

Accurately identifying sentence boundaries in speech is crucial for analyzing syntactic complexity, a key indicator of cognitive impairments like aphasia. However, existing automatic speech recognition (ASR) systems often introduce errors (insertions, deletions, substitutions) that can disrupt sentence segmentation. Furthermore, the speech patterns of individuals with aphasia, such as altered pausing and disfluencies, may not align with the assumptions of standard segmentation algorithms trained on normal speech. This paper bridges this gap by evaluating the performance of automatic sentence segmentation on aphasic speech and assessing the impact of segmentation errors on syntactic complexity measures.

Methods

The researchers employed a conditional random field (CRF) classifier for sentence segmentation, a standard approach in the field. They experimented with various combinations of features, including:

  • Lexical features: Unlemmatized word tokens.
  • Part-of-Speech (POS) features: POS tags of the current, next, and previous words.
  • Prosodic features: Measures of pitch, energy, and duration of phonemes around word boundaries, as well as pause lengths.

To assess the impact of segmentation, they compared syntactic complexity metrics calculated on both manually and automatically segmented transcripts. These metrics included measures like Yngve depth, tree height, and various units defined by Lu's Syntactic Complexity Analyzer (SCA), such as clauses, T-units, and complex nominals.

Datasets

The study utilized two primary datasets:

  • Primary Progressive Aphasia (PPA) data: Narrative speech samples from 28 patients with PPA (11 with semantic dementia (SD) and 17 with progressive nonfluent aphasia (PNFA)) and 23 age- and education-matched healthy controls. These samples were manually transcribed and then processed to simulate ASR output by removing capitalization and punctuation.
  • Broadcast News data: A large corpus (TDT4) and a smaller subset of 20 randomly selected news stories. This data served as a benchmark for performance on prepared speech.

Key Insights and Ideas

  • Aphasic speech presents unique segmentation challenges: The study confirms that standard segmentation methods, while adaptable, perform best on prepared speech (broadcast news) and healthy controls, with performance degrading on aphasic speech.
  • Syntactic complexity metrics vary in robustness: Not all syntactic complexity measures are equally affected by noisy segmentation. Some, like T-units and clause counts, show little significant difference between manual and automatic segmentation, suggesting they are robust. Others, like Yngve depth, are more sensitive to segmentation errors.
  • Segmentation errors are systematic: While automatic segmentation introduces errors, these errors appear to be systematic. For example, the system tends to produce more sentences than manually annotated. Crucially, the direction of differences in syntactic complexity between patient groups often remains consistent between manual and automatic segmentation.
  • Feature importance shifts: POS features become more important for segmentation in PPA groups compared to controls, potentially due to the increased use of conjunctions to connect clauses in aphasic speech.

Key Results

  • Performance on PPA data is promising: Despite the challenges, the study found that standard segmentation methods could still be effective for aphasic speech, with results on PPA patients being "not much worse" than controls in some cases.
  • Robustness of metrics: Six out of eight primary syntactic units measured by Lu's SCA showed no significant difference between manual and automatic segmentation across all clinical groups.
  • Directional consistency: In 92% of cases, and in every significant difference between patient groups, the direction of the trend in syntactic complexity metrics was the same whether using manual or automatic segmentation. For example, controls consistently showed greater maximum Yngve depth than SD participants in both manual and automatic transcripts.

Limitations and Future Work

  • Simulated ASR data: The study used manually transcribed data with punctuation removed to simulate ASR output. Real-world ASR data would likely contain additional recognition errors, the impact of which is not fully explored.
  • Small PPA dataset: The PPA dataset, while carefully collected, is relatively small, which could limit the generalizability of machine learning models.
  • Future directions: The authors suggest adapting standard algorithms to improve performance on impaired speech, evaluating forced alignment on aphasic speech, and exploring new features for boundary classification. They also highlight the need to investigate inter-annotator agreement to establish a gold standard for sentence segmentation in spoken language, as the concept of a "sentence" itself can be ambiguous in speech.

Summary updated

BibTeX
@inproceedings{fraser2015sentence,
  author = {Kathleen C. Fraser and Naama Ben-David and Graeme Hirst and Naida Graham and Elizabeth Rochon},
  title = {{Sentence segmentation of aphasic speech}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
}

Loading related content...

Figure