Abstract
Researchers at Apple have developed a "Named Entity Understanding" (NEU) framework that jointly optimizes Named Entity Recognition (NER) and Entity Linking (EL) to handle the messy reality of voice assistant queries. By using a joint reranking module and character-level neural networks, the system significantly improves accuracy on noisy spoken data, outperforming traditional pipelined approaches. (Source: Muralidharan et al., NAACL 2021 Industry Papers).
Introduction
Voice assistants like Siri or Alexa face a unique "noise" problem. Unlike written text, spoken queries are often extremely short, averaging just five tokens, which provides very little context for disambiguation. Furthermore, the input is prone to three types of errors: Automatic Speech Recognition (ASR) errors (e.g., "Bohemian raspberry" instead of "Rhapsody"), user errors (e.g., "Cristiano Nando" instead of "Ronaldo"), and the use of non-canonical forms or aliases.
In a traditional pipeline, NER identifies a "thing" (e.g., a Person), and EL then tries to link that "thing" to a specific entry in a Knowledge Base (KB). If the NER step fails due to noise, the EL step has no chance of succeeding. This paper proposes a way to let these two tasks interact to fix errors through a joint reranking architecture.
Methodology
The authors propose a Named Entity Understanding (NEU) architecture that moves away from a rigid pipeline toward a joint reranking approach.
NER Module and Candidate Retrieval
The system uses a standard biLSTM-CRF model with character and word-level (GloVe) features. Instead of passing a single best guess, it employs beam search to generate the top five NER hypotheses. For each hypothesis, a retrieval engine queries the Knowledge Base. To handle noise, the query passes through a seq-to-seq correction model and uses an inverted index that includes common aliases and n-grams.
Joint Linking and Reranking
This is the core innovation: a neural ranker evaluates the NER hypotheses and EL candidates simultaneously.
- Character-level CNNs: To be robust against misspellings or ASR errors, CNNs create embeddings of the mentions and the KB entities. Because "raspberry" and "rhapsody" look similar at a character level, the model can learn to associate them even if their word embeddings are distinct.
- Relational and Popularity Features: The model looks for relationships in the KB. If a query mentions two entities belonging to the same sports league or music album, this "relational signal" boosts confidence. It also incorporates "prior usage probability" to account for entity popularity.
Key Results
The NEU framework showed consistent improvements across high-entity domains including Music, Sports, and Movie & TV.
Overall Performance
NER accuracy improved by up to 3.13% in F1 score through reranking, while EL accuracy saw gains of up to 3.6%.
Robustness to Noise
The system's strength is most apparent in noisy conditions. On the "noisy" subset of the music domain, using the MLP-based similarity feature improved EL performance by a staggering 10.03%.
Downstream Integration
Adding NEU features improved performance in downstream tasks. Domain Classification accuracy rose (e.g., from 88.95 to 90.04), and Semantic Parsing accuracy improved across all tested domains, demonstrating the utility of the framework in a production-style environment.
Conclusion
The NEU framework demonstrates that Entity Linking should inform NER rather than just follow it. By using character-level features and KB relational data, the system achieves significant robustness to the ASR and user errors common in voice assistant interactions. Future work will focus on addressing the "cold start" problem for entities missing from the KB and further optimizing inference latency.
Summary updated