Knowledge extraction from aeronautical messages (NOTAMs) with self-supervised language models for aircraft pilots

Alexandre Arnold, Fares Ernez, Catherine Kobus, Marion-Cécile MartinNAACL 2022 · DOI

Introduction

In modern aviation, pilots are inundated with information during pre-flight briefings. Researchers at Airbus AI Research, in a paper presented at NAACL-HLT 2022, have proposed a solution using advanced Natural Language Processing (NLP) to structure and prioritize Notice to AirMen (NOTAMs). By leveraging domain-specific language models, the team developed a pipeline that translates cryptic aeronautical messages into machine-parsable formats, significantly reducing the cognitive load on flight crews.

The Challenge: Information Overload and Linguistic Complexity

NOTAMs are essential text-based alerts regarding potential hazards, such as closed runways or temporary obstacles. However, they present two major hurdles for pilots:

  1. Volume: Long-haul flights can generate lists spanning up to 100 pages, containing hundreds of individual alerts.
  2. Linguistic Complexity: NOTAMs utilize a highly specialized "aeronautical English" characterized by non-standard abbreviations and a lack of formal grammar.
An example of a raw NOTAM message showing the cryptic aeronautical English and non-standard abbreviations.

Standard NLP models trained on general corpora like Wikipedia perform poorly on this dialect. Effective knowledge extraction requires models that understand the specific nuances of aviation communication, where the overlap with standard English is minimal.

Methodology: Domain-Specific Language Modeling

The researchers employed a self-supervised approach to overcome the scarcity of expert-labeled data.

Pre-training and Tokenization

RoBERTa and DeBERTa v2 models were pre-trained from scratch on a corpus of 1.2 million unlabeled NOTAMs. A dedicated tokenizer was developed to handle the unique vocabulary of aeronautical messages, ensuring the model could accurately represent domain-specific terms.

Downstream Tasks

  • Criticality Prediction: A regression head was added to assign priority scores (1 to 5), preserving the ordinal relationship between levels of importance.
  • Named Entity Recognition (NER): To capture nested and long-span entities (e.g., a closure affecting a specific runway), the team implemented a RoBERTa-CRF (Conditional Random Field) architecture.
  • Translation to Airlang: An encoder-decoder model was built to translate raw text into Airlang, a structured, machine-parsable language used for digital map visualizations in flight bags.

Key Results

The domain-adapted models demonstrated high precision across all tasks. The RoBERTa-CRF model achieved F1-scores of 99.3% for runways and 94.9% for taxiways. Complex entities like "closure" reached a 79.2% F1-score, benefiting significantly from the CRF layer's ability to maintain consistency over several words.

In the translation task, the RoBERTa model with a 1536 hidden size achieved a "perfect" translation rate of 84.5% after applying simple post-processing rules to normalize whitespace and capitalization.

Translation performance metrics for different encoder models, highlighting the impact of model size and post-processing.

Conclusion and Future Directions

While these models show promise in automating the extraction of actionable knowledge, the researchers emphasize their role as support tools rather than replacements for human analysis. Future work will focus on uncertainty quantification through conformal predictions—allowing the model to signal when it is unsure—and abstractive summarization to further streamline the pre-flight briefing process for pilots.

Summary updated

BibTeX
@inproceedings{arnold2022knowledge,
  author = {Alexandre Arnold and Fares Ernez and Catherine Kobus and Marion-Cécile Martin},
  title = {{Knowledge extraction from aeronautical messages (NOTAMs) with self-supervised language models for aircraft pilots}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.naacl-industry.22},
}
Figure