DRAGON-V: Detection and Recognition of Airplane Goals with Navigational Visualization

Christabel Wayllace, Sunwoo Ha, Yuchen Han, Jiaming Hu, Shayan Monadjemi, William Yeoh, Alvitta Ottley

Introduction

Runway incursions—incidents where an unauthorized aircraft or vehicle enters a runway—remain a significant safety challenge in aviation. According to the FAA, pilots are responsible for roughly 65% of these incidents, often due to miscommunication with Air Traffic Controllers (ATCs), a lack of familiarity with complex airport layouts, or failure to follow standard operating procedures.

DRAGON-V is a visualization system designed to help ATCs predict an airplane's intended runway during taxiing using probabilistic goal recognition. By analyzing real-time geolocation data against airport layouts, the system provides a visual "heads-up" on whether a pilot is following their assigned route or heading toward a potential runway incursion.

System Architecture

DRAGON-V System Architecture

The DRAGON-V architecture consists of five integrated modules designed to transform raw geolocation data into actionable intent prediction:

  1. Airport Layout Retrieval: The system represents the airport (taxiways, truck routes, and runways) as a directed graph where nodes are defined by latitude and longitude. For this study, data was pulled from the X-Plane flight simulator.
  2. Optimal Path Computation: The system identifies all possible "goals" (runway ends) and pre-computes the optimal (shortest distance) paths from every node in the airport graph to every goal.
  3. Geolocation Retrieval: Real-time coordinates are streamed from the aircraft (simulated via the X-Plane Connect toolbox) using UDP messages.
  4. Goal Recognition: This is the core "intelligence" of the system. It uses single-observation cost-based goal recognition. Instead of looking at the entire history of the flight, it compares the cost of the pilot's current position relative to the starting point against the optimal path to a specific goal. It uses Bayes' rule and a sigmoid function to transform the "cost difference" into a probability.
  5. Visualization Interface: The UI displays the airport map with the airplane’s trajectory and color-coded goals.

Experimental Setup

The researchers utilized the X-Plane flight simulator ecosystem for both their environment and data. They extracted the taxiway and runway networks for the Seattle-Tacoma (Sea-Tac) airport. The system was tested using simulated flight paths within X-Plane, including paths with varying degrees of "suboptimality" (i.e., paths where the pilot takes a wrong turn or a longer route).

Key Results and Visualization

The DRAGON-V Visualization Interface

DRAGON-V functions as a proactive tool, attempting to infer the pilot's "mental model" or goal based on their movement. Qualitative results show that the system successfully:

  • Ranks Goals Dynamically: As an aircraft moves, the color-coding of the runways updates in real-time to reflect changing probabilities. Goals are color-coded: green for high probability and purple for low probability.
  • Handles Suboptimality: Even when a pilot takes a non-optimal path, the system continues to provide a ranked list of likely goals rather than failing or becoming "certain" of a wrong goal. This accounts for "bounded rationality," recognizing that pilots generally try to take efficient routes but may be limited by poor visibility or lack of familiarity.
  • Provides Interactive UI: Controllers can hover over specific goals to see exact probability percentages or view the specific optimal path associated with that goal.

Limitations and Future Work

While a promising proof-of-concept, the authors acknowledge several areas for growth. The current version only tracks one aircraft at a time and defines "cost" solely by distance. Future iterations could incorporate more complex variables like weather conditions, current airport congestion, and specific route restrictions. Additionally, the researchers plan to move from a "post-processed" visualization to a fully online, streaming update system to better support real-world ATC environments.

Summary updated

Abstract

We introduce Detection and Recognition of Airplane GOals with Navigational Visualization (DRAGON-V), a visualization system that uses probabilistic goal recognition to infer and display the most probable airport runway that a pilot is approaching. DRAGON-V is especially useful in cases of miscommunication, low visibility, or lack of airport familiarity which may result in a pilot deviating from the assigned taxiing route. The visualization system conveys relevant information, and updates according to the airplane's current geolocation. DRAGON-V aims to assist air traffic controllers in reducing incidents of runway incursions at airports.
BibTeX
@inproceedings{wayllace2020dragon,
  author = {Christabel Wayllace and Sunwoo Ha and Yuchen Han and Jiaming Hu and Shayan Monadjemi and William Yeoh and Alvitta Ottley},
  title = {{DRAGON-V: Detection and Recognition of Airplane Goals with Navigational Visualization}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i09.7108},
}
Figure