Knowledge-Based Word Lattice Rescoring in a Dynamic Context

Todd Shore, Friedrich Faubel, Hartmut Helmke, Dietrich KlakowInterspeech 2012 · DOI

Abstract

Automatic Speech Recognition (ASR) in specialized domains like Air Traffic Control (ATC) faces unique challenges due to sparse training data and high-stakes accuracy requirements. This paper presents a method for rescoring ASR word lattices using real-time situational data, such as radar logs. By validating commands against the physical state of the airspace, the researchers achieved an 80% reduction in Word Error Rate (WER).

Introduction

Standard ASR systems are heavily data-driven, relying on patterns found in massive datasets. However, they often struggle in specialized domains where the "correct" answer depends entirely on the current physical environment. In Air Traffic Control, a controller’s command is dictated by the "dynamic context"—the positions, speeds, and identities of aircraft currently on the radar. A traditional ASR might hear a callsign that sounds plausible but belongs to a plane that isn't even in the sector. This paper addresses the challenge of integrating this higher-level situational knowledge into the recognition process to filter out linguistically plausible but contextually impossible hypotheses.

Methodology

The authors employ a technique called Knowledge-Based Word Lattice Rescoring. The process begins with a grammar where semantic concepts, such as callsigns or altitudes, are wrapped in XML-style tags. When a controller speaks, the ASR generates a "lattice"—a directed graph representing various sequences of words with associated probability scores.

Figure 5: Example of a word lattice with semantic tags for airline identification.

As the system traverses this lattice, it extracts the semantic values and checks them against real-time radar data using three specific constraints:

  • Callsign Validity: Does the recognized callsign exist in the current airspace?
  • Speed Logic: If the command is "reduce speed," is the target speed actually lower than the plane's current speed?
  • Altitude Logic: If the command is "descend," is the target altitude lower than the plane's current altitude?

Hypotheses that violate these constraints are penalized with a higher "cost," effectively pushing the contextually correct hypothesis to the top of the list.

Experimental Setup and Datasets

Because no public ATC dataset included synchronized radar logs, the authors created a custom corpus using an Air Traffic Management simulation (4D-CARMA).

Figure 6: The 4D-CARMA simulation environment providing real-time radar data for dynamic context.
  • The ATC Corpus: 1,107 commands recorded from 16 speakers with various international accents.
  • Contextual Logs: Every 5 seconds, the state of the airspace (aircraft positions, speeds, etc.) was logged to provide ground truth for the rescoring constraints.
  • Vocabulary: The grammar covered 244,220 unique aircraft callsigns.

Key Results and Insights

The results demonstrate that context is king in specialized domains. The most significant impact came from the callsign constraint; by checking the radar, the system could prune the list of possible callsigns from over 244,000 down to the ~10 aircraft actually present in the sector. This massive reduction in perplexity drove the bulk of the accuracy gains.

  • Baseline (No Context): 2.81% Word Error Rate (WER).
  • With Callsign Constraint: WER dropped to 0.55% (an 80.4% relative reduction).
  • With All Constraints: WER reached 0.52%.
  • Mean Reciprocal Rank (MRR): Improved from 0.849 to 0.967, indicating the correct hypothesis was almost always ranked first after rescoring.

Conclusion and Future Work

While the current system uses binary constraints (valid or invalid), future iterations could employ "soft" constraints to consider the likelihood of specific flight paths. Additionally, integrating these constraints directly into the initial ASR decoder—rather than as a post-processing rescoring step—could allow for more aggressive pruning, making the system even faster and more efficient for real-time deployment.

Summary updated

BibTeX
@inproceedings{shore2012knowledge,
  author = {Todd Shore and Friedrich Faubel and Hartmut Helmke and Dietrich Klakow},
  title = {{Knowledge-Based Word Lattice Rescoring in a Dynamic Context}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-328},
}
Figure