TL;DR
This paper tackles the challenge of entity linking in spoken language, which is significantly harder than in written text due to Automatic Speech Recognition (ASR) errors. They introduce novel phonetic and lattice-based features that improve entity linking accuracy on noisy ASR transcripts, demonstrating that spoken language entity linking is feasible and crucial for applications like search.
What Problem Does This Paper Address?
Entity linking, the task of identifying mentions of entities in text and linking them to a knowledge base (like Wikipedia), has been extensively studied for written text across various domains (news, blogs, social media). However, its application to spoken language, particularly transcribed speech, remains largely unexplored. Spoken language presents unique hurdles: ASR systems introduce transcription errors that can distort context and lead to misidentified or misspelled named entities. These errors make it difficult for traditional entity linking systems, which rely heavily on textual similarity, to accurately match mentions to knowledge base entries and disambiguate between similar entities. Given the increasing prevalence of audio content in information retrieval and media consumption, enabling entity linking for spoken language is a critical, yet unsolved, problem.
Methods
The authors propose new features designed to combat the specific challenges of ASR-corrupted spoken language. Their approach focuses on two main areas:
- Contextual Errors and Shortening: Spoken language transcripts can be less contextually rich than written documents, and ASR errors can further degrade this context.
- Name Mention Transcription Errors: ASR systems often struggle with named entities, leading to phonetic misspellings or out-of-vocabulary terms.
To address these, they introduce four types of features:
- Text Features: These are standard features used in written entity linking, comparing mention strings and document context to knowledge base entries.
- Phone Features: This novel approach represents words as sequences of phonemes (sounds) instead of text. They use a grapheme-to-phone (G2P) system to convert words into phonetic representations, allowing for similarity matching even when words are phonetically similar but textually different due to ASR errors.
- Metaphone Features: Building on phone features, Metaphones are used to further normalize phonetic representations, mapping similar-sounding phones to the same code. This helps to capture phonetic similarities more robustly.
- Lattice Features: These features leverage the ASR lattice, which represents multiple possible word hypotheses from the ASR system. They use expected word counts from the lattice to inform context, weighting features by term expectation.
In addition to these features, they also modify the "triage" step (candidate generation) to incorporate these new phonetic and lattice-based heuristics. Their entity linking system, Slinky, uses a learned cascade of classifiers to rank candidates.
Datasets
The authors introduce a new benchmark dataset:
- HUB4 Broadcast News Transcripts: This dataset is derived from the HUB4 broadcast news corpus, manually transcribed. They used gold named entity annotations from Parada et al. (2011) and linked these entities to the TAC KBP knowledge base (derived from Wikipedia). This corpus contains 2140 annotated person entities.
- TAC KBP Knowledge Base: A large knowledge base containing 818,741 entries derived from Wikipedia, used for linking.
The dataset was split into 60% for training and 40% for testing, ensuring that mention strings were disjoint between the sets.
Key Insights and Ideas
- Phonetic Similarity is Key: The most significant insight is that phonetic similarity is crucial for handling ASR errors in spoken language entity linking. When ASR misrecognizes a word, it often produces a phonetically similar but textually different output. Phonetic features can bridge this gap, enabling correct linking where text-based features fail.
- Metaphones Outperform Basic Phones: The Metaphone features, which provide a more aggressive phonetic normalization, generally perform better than raw phone sequences, suggesting that capturing broader phonetic similarity is more beneficial.
- Lattice Features Offer Limited Gains: While lattice features aim to improve context by considering ASR hypotheses, they showed less significant improvements. This suggests that the primary challenge lies in accurately identifying the mention string itself, rather than just refining the context.
- Triage Recall is Critical: The paper highlights that the overall accuracy of spoken language entity linking is heavily dependent on the recall of the candidate generation (triage) step. If the correct entity is not even considered as a candidate due to ASR errors in the mention string, the linker cannot succeed. Their new features significantly improve this triage recall.
- Spoken Language Entity Linking is Feasible: Despite the challenges, the results demonstrate that entity linking for spoken language, even with noisy ASR output, can achieve respectable accuracy, approaching that of written language when the system is well-tuned.
Key Results
- Significant Accuracy Improvement: The proposed features dramatically improve entity linking accuracy on ASR-corrupted transcripts. For the
tri4bASR system (51% WER), accuracy increased from 0.55 (using only text features) to 0.78 (using text + metaphone + lattice features). - Recall Boost: The primary driver of accuracy improvement is a significant boost in recall, particularly in the triage stage. This allows the ranking system to consider more correct candidates.
- Metaphone Features Lead: Metaphone features showed better performance than basic phone features.
- Robustness to WER: Even with a Word Error Rate (WER) of 51% for the
tri4bsystem, entity linking accuracy was not severely impacted, provided that the correct candidate was recalled. - Reference vs. ASR: On reference transcripts, entity linking accuracy was high (0.82-0.85). However, on ASR transcripts, accuracy dropped considerably (0.48-0.55) before the proposed features were applied.
Limitations and Future Work
- Focus on Person Entities: The current work focuses exclusively on person entities. Extending to other entity types (organizations, locations, etc.) would be valuable.
- Domain Specificity: The dataset is broadcast news. Evaluating the features on other spoken language domains (e.g., conversational speech, lectures) is a natural next step.
- Improving Triage: While recall improved, further work is needed to further enhance the candidate generation process, especially for mentions with high ASR error rates.
- Error Analysis: The authors suggest future work should focus on additional methods for identifying relevant KB candidates given inaccurate transcriptions of mention strings.
Summary updated