TempFuser: Learning Tactical and Agile Flight Maneuvers in Aerial Dogfights using a Long Short-Term Temporal Fusion Transformer

Hyunki Seong, David Hyunchul Shim

Abstract

Aerial dogfighting requires a complex balance between long-term tactical positioning and immediate aerodynamic control. This paper introduces TempFuser, a novel reinforcement learning architecture that utilizes a Long Short-Term Temporal Fusion Transformer to bridge this gap. By processing dual-scale state histories—short-term dense trajectories for agility and long-term sparse trajectories for tactics—the agent learns to execute advanced maneuvers in high-fidelity simulations, outperforming expert-level rule-based systems and standard deep learning baselines.

Introduction

Dogfighting is often considered the "grand challenge" of autonomous flight. It demands that a pilot (or AI) manage high-dimensional, nonlinear flight dynamics while simultaneously planning several steps ahead to outmaneuver an opponent. Traditional Deep Reinforcement Learning (DRL) models often struggle with this "temporal paradox," either focusing too much on immediate pursuit (leading to energy depletion) or failing to react quickly enough to fleeting opportunities.

The Temporal Paradox in Aerial Combat

To succeed in a 1v1 engagement, an agent must address three primary hurdles:

  1. Nonlinearity: Aerodynamics change drastically across different flight envelopes (e.g., subsonic vs. supersonic).
  2. Multi-Scale Planning: Tactical goals (getting "on the six") operate on a different timescale than dynamic adjustments (preventing a stall).
  3. Generalization: AI must adapt to various aircraft types with differing performance characteristics.

Standard Multi-Layer Perceptrons (MLPs) lack the memory structure to handle these scales, while basic Recurrent Neural Networks (RNNs) can suffer from vanishing gradients over long sequences.

The TempFuser Architecture

The core of the proposed solution is the Long Short-Term Temporal Fusion Transformer (TempFuser).

The TempFuser architecture: Fusing short-term dense and long-term sparse state trajectories via a Transformer Encoder.

The architecture employs a dual-scale input strategy:

  • Short-Term Dense Trajectory ($s_t^s$): Captures immediate physics from the most recent frames to inform agile maneuvers.
  • Long-Term Sparse Trajectory ($s_t^l$): Samples state history over a longer duration to capture the overall "flow" of the engagement.

These trajectories are processed by separate LSTM embeddings and then fused using a Transformer Encoder. The self-attention mechanism allows the model to dynamically weigh the importance of immediate aerodynamic constraints against long-term tactical objectives.

Training and Evaluation in DCS

The agent was trained using Soft Actor-Critic (SAC) within the Digital Combat Simulator (DCS World), a high-fidelity environment. To ensure robust performance, the agent (an F-16) was pitted against a variety of opponents, including the F-15E, F/A-18A, and the aerodynamically superior Su-27. An energy-embedded reward function was utilized to penalize inefficient maneuvers, forcing the agent to master "energy maneuverability."

Results and Emergent Tactics

TempFuser demonstrated significant superiority over the built-in "DCS-Ace" AI and standard DRL baselines. It achieved a 93.5% win rate against the F-15E and a 92.5% win rate against the Su-27. Notably, against the Su-27, a standard MLP-based agent achieved only a 1.7% win rate, highlighting the necessity of the temporal fusion approach.

Visualization of pursuit geometries and 3D flight trajectories used to evaluate tactical maneuvers.

Without explicit instruction, the agent "discovered" classic Basic Fighter Maneuvers (BFM):

  • Low Yo-Yo: Trading altitude for speed to close the distance during a turn.
  • Flat Scissors: Forcing an opponent to overshoot through a series of horizontal turns.

The model also showed high generalization capabilities, maintaining an 86.1% win rate against the Su-30, an aircraft it had never encountered during training.

Conclusion and Future Work

TempFuser successfully bridges the gap between tactical planning and agile execution in aerial combat. While currently limited to 1v1 scenarios, the multi-scale temporal fusion approach offers a promising path forward for complex robotics tasks. Future research will focus on multi-agent (N-vs-N) engagements and addressing the sim-to-real gap for deployment on physical unmanned aerial vehicles (UAVs).

Summary updated

BibTeX
@article{seong2024tempfuser,
  author = {Hyunki Seong and David Hyunchul Shim},
  title = {{TempFuser: Learning Tactical and Agile Flight Maneuvers in Aerial Dogfights using a Long Short-Term Temporal Fusion Transformer}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2024},
  doi = {10.1109/lra.2024.3477092},
}
Figure