Evaluation of semantic role labeling and dependency parsing of automatic speech recognition output

Benoît Favre, Bernd Bohnet, Dilek Hakkani-TürICASSP 2010 · OpenReview

Abstract

Semantic role labeling (SRL) is an important module of spoken language understanding systems. This work extends the standard evaluation metrics for joint dependency parsing and SRL of text in order to be able to handle speech recognition output with word errors and sentence segmentation errors. We propose metrics based on word alignments and bags of relations, and compare their results on the output of several SRL systems on broadcast news and conversations of the OntoNotes corpus. We evaluate and analyze the relation between the performance of the subtasks that lead to SRL, including ASR, part-of-speech tagging or sentence segmentation. The tools are made available to the community.
BibTeX
@inproceedings{favre2010evaluation,
  author = {Benoît Favre and Bernd Bohnet and Dilek Hakkani-Tür},
  title = {{Evaluation of semantic role labeling and dependency parsing of automatic speech recognition output}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2010},
  url = {https://openreview.net/forum?id=BuJ5k4im0YI},
}

Loading related content...

Figure