Abstract
Large-scale conversational AI systems frequently encounter "frictions" caused by Automatic Speech Recognition (ASR) errors, particularly when interpreting entity-heavy queries. We present KG-ECO (Knowledge Graph enhanced Entity COrrection), a two-stage framework designed to identify and correct corrupt entities in user utterances. By integrating textual entity descriptions with structural graph data via Graph Attention Networks (GATs), KG-ECO significantly improves query rewriting accuracy. Our approach is particularly effective for rare "few-shot" entities, nearly doubling the precision of baseline models in low-frequency scenarios while maintaining high safety standards on clean queries.
Introduction
Conversational assistants like Alexa and Siri often struggle with entity-related errors. For example, a user's request to "play bad romance" might be misheard by the ASR module as "play bad boy dance." Correcting these errors is a non-trivial task for two primary reasons: first, the lack of external context makes it difficult for a model to distinguish between a valid but obscure phrase and a phonetic error; second, the system must simultaneously identify the "corrupt span" (where the error is) and determine the correct replacement entity without relying on a separate, potentially error-prone Named Entity Recognition (NER) step.
The KG-ECO Framework
KG-ECO addresses these challenges through a two-layer architecture that leverages the rich relational data found in Knowledge Graphs (KGs).
L1: Retrieval (The Fast Layer)
The first stage utilizes a bi-encoder architecture to efficiently narrow down millions of potential entities to a manageable candidate set. One encoder processes the user's utterance using RoBERTa, while the other encodes entities from a massive index. To enrich the entity representations, KG-ECO combines RoBERTa-based textual descriptions (e.g., "Lady Gaga: American singer...") with a Graph Attention Network (GAT). The GAT captures the entity's structural context by processing its immediate neighbors in the KG, allowing the model to understand relationships like "performer" or "genre."
L2: Re-ranking and Span Detection (The Smart Layer)
The top candidates from L1 are passed to a cross-encoder in the second stage. This layer performs joint learning on two tasks: ranking the candidates based on their relevance to the utterance and predicting the start and end positions of the corrupt text within the original query. This joint approach allows the system to be more flexible and robust, as it does not require a pre-defined entity span to begin the correction process.
Knowledge Graph Integration
The authors utilized a condensed version of the Wikidata dataset, encompassing 3 million entities and 141 million triples. A key innovation in KG-ECO is the use of a modified GAT that handles specific relationship types using a non-parametric subtraction method. By looking at "one-hop" subgraphs, the model can leverage "context entities" present in the query (like an artist's name) to find the correct "target entity" (a song title), even if the target entity itself is rare or phonetically distorted.
Experimental Results
KG-ECO was evaluated using 8.5 million de-identified rephrase pairs and a specific "friction set" of 107,000 utterances containing known errors. The results demonstrate a substantial improvement over state-of-the-art utterance-level rewriters:
- Overall Accuracy: KG-ECO achieved an Entity Precision of 43.9%, compared to 33.4% for the baseline gUFS-QR model.
- Few-Shot Performance: The model's reliance on KG data proved most beneficial for rare entities. For entities appearing 10 times or fewer in the training data, KG-ECO reached 31.3% precision, nearly doubling the 23.0% achieved by a version of the system without KG features.
- System Safety: On a "clean set" of 3,000 utterances requiring no correction, the model maintained a low false trigger rate of 2.3%, ensuring that valid user queries are not inadvertently corrupted by the rewriter.
Discussion and Future Work
The success of KG-ECO highlights the importance of external knowledge as a "safety net" for conversational AI. While the system is currently limited by the coverage of Wikidata and the computational overhead of real-time GAT processing, it provides a clear path forward for more robust query rewriting. Future research may explore multi-hop reasoning to capture more distant entity relationships and further optimization to reduce the latency of the cross-encoder stage. By treating the Knowledge Graph as an external memory, KG-ECO moves conversational systems closer to human-like understanding of context and intent.
Summary updated