FlightBERT++: A Non-autoregressive Multi-Horizon Flight Trajectory Prediction Framework

Dongyue Guo, Zheng Zhang, Zhen Yan, Jianwei Zhang, Yi LinAAAI 2024 · DOI

TL;DR

This paper introduces FlightBERT++, a novel framework for predicting future flight trajectories that overcomes the limitations of existing autoregressive methods. By employing a non-autoregressive approach and a new "differential prediction" paradigm, FlightBERT++ achieves higher accuracy and significantly improved computational efficiency, making it more suitable for real-time air traffic control applications.

What Problem Does This Paper Address?

Air Traffic Control (ATC) relies heavily on accurate Flight Trajectory Prediction (FTP) to manage airspace safely and efficiently. Current FTP methods often predict future flight paths step-by-step (autoregressively). This approach, while intuitive, suffers from two major drawbacks: error accumulation, where small prediction errors at each step compound over time, leading to increasingly inaccurate forecasts, and low computational efficiency, as predicting many future steps requires many sequential calculations, making it too slow for real-time decision-making. Furthermore, existing methods like FlightBERT, which use binary encoding (BE) for trajectory attributes, can produce extreme outliers when high-bit predictions are incorrect.

Methods

FlightBERT++ builds upon the binary encoding (BE) representation of FlightBERT but introduces several key innovations:

  • Non-Autoregressive Multi-Horizon Prediction: Instead of predicting one step at a time, FlightBERT++ predicts all future trajectory points simultaneously. This directly addresses the error accumulation problem.
  • Generalized Encoder-Decoder Architecture: The framework uses a standard encoder-decoder structure. The encoder processes historical flight data to learn temporal-spatial patterns. The decoder then predicts future states.
  • Horizon-Aware Context Generator (HACG): This novel module is crucial for enabling non-autoregressive prediction. It generates context representations for each future horizon by considering both the overall historical trajectory information and specific "horizon encodings" (information about which future time step is being predicted). This allows the model to "look ahead" and generate predictions for multiple horizons at once.
  • Differential-Prompted Decoder (DPD): To tackle the outlier issue from BE representation, the DPD predicts differential changes in trajectory attributes rather than absolute values. This is more robust to high-bit errors. The "prompted" aspect means it uses the differential sequence as a prompt to learn transition patterns more effectively, mitigating potential loss of geographical/kinematical information.
  • Transformer Backbone: Both the encoder and decoder utilize Transformer blocks, which are well-suited for capturing temporal dependencies in a non-autoregressive manner.

Datasets

The proposed framework is evaluated on a real-world flight trajectory dataset collected from an industrial ATC system. This dataset spans 9 days with 20-second intervals, covering specific geographical regions. The data includes timestamps, call signs, longitude, latitude, altitude, and velocity in x, y, and z directions. The dataset was split into training (7 days), validation (1 day), and testing (1 day) sets. No new benchmark datasets were introduced in this paper.

Key Insights and Ideas

  • Non-autoregression is key for efficiency and accuracy: By breaking free from step-by-step prediction, the framework significantly reduces computation time and prevents error compounding.
  • Differential prediction is a clever way to handle BE limitations: Predicting changes rather than absolute values makes the model more resilient to the extreme errors that can arise from misinterpreting binary codes.
  • Context matters for multi-horizon prediction: The HACG's ability to generate horizon-specific context is vital for the non-autoregressive decoder to make informed predictions across multiple future time steps.
  • Flight patterns are learnable: The visualization results suggest that FlightBERT++ can capture not just flight dynamics but also typical flight intents (like turns or descents), indicating a deeper understanding of flight behavior.

Key Results

  • Accuracy: FlightBERT++ consistently outperformed competitive baselines (including FlightBERT, LSTM, Transformer, Kalman Filter, and LSTM+Attention) across various metrics like Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), and Mean Distance Error (MDE). For example, it achieved lower MAE across all predicted attributes.
  • Efficiency: FlightBERT++ demonstrated substantial improvements in computational efficiency. Despite having a larger model size than some comparison models, it achieved the fastest prediction speed among deep learning-based models, with a Mean Time Cost (MTC) of 6.81 ms for 15 prediction horizons, significantly lower than FlightBERT's 201.41 ms.
  • Outlier Reduction: Histograms of distance errors clearly show that FlightBERT++ produces fewer extreme outliers compared to the original FlightBERT, validating the effectiveness of the differential prediction paradigm.

Limitations and Future Work

  • Model Size: The paper notes that FlightBERT and FlightBERT++ have larger model sizes compared to some simpler models, though this is justified by their enhanced capabilities.
  • Future Work: While not explicitly detailed as future work, the paper's success in capturing flight intents suggests potential for integrating this into more advanced ATC decision support systems, such as conflict detection and airspace planning, to improve overall safety and efficiency. The authors also highlight the importance of the differential prompted mechanism for long-horizon predictions, implying that further research into robust differential learning could be beneficial.

Summary updated

BibTeX
@inproceedings{guo2024flightbert,
  author = {Dongyue Guo and Zheng Zhang and Zhen Yan and Jianwei Zhang and Yi Lin},
  title = {{FlightBERT++: A Non-autoregressive Multi-Horizon Flight Trajectory Prediction Framework}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i1.27763},
}
Figure