Abstract
Aerial dogfighting requires a complex balance between long-term tactical positioning and immediate aerodynamic control. This paper introduces TempFuser, a novel reinforcement learning architecture that utilizes a Long Short-Term Temporal Fusion Transformer to bridge this gap. By processing dual-scale state histories—short-term dense trajectories for agility and long-term sparse trajectories for tactics—the agent learns to execute advanced maneuvers in high-fidelity simulations, outperforming expert-level rule-based systems and standard deep learning baselines.
Introduction
Dogfighting is often considered the "grand challenge" of autonomous flight. It demands that a pilot (or AI) manage high-dimensional, nonlinear flight dynamics while simultaneously planning several steps ahead to outmaneuver an opponent. Traditional Deep Reinforcement Learning (DRL) models often struggle with this "temporal paradox," either focusing too much on immediate pursuit (leading to energy depletion) or failing to react quickly enough to fleeting opportunities.
The Temporal Paradox in Aerial Combat
To succeed in a 1v1 engagement, an agent must address three primary hurdles:
- Nonlinearity: Aerodynamics change drastically across different flight envelopes (e.g., subsonic vs. supersonic).
- Multi-Scale Planning: Tactical goals (getting "on the six") operate on a different timescale than dynamic adjustments (preventing a stall).
- Generalization: AI must adapt to various aircraft types with differing performance characteristics.
Standard Multi-Layer Perceptrons (MLPs) lack the memory structure to handle these scales, while basic Recurrent Neural Networks (RNNs) can suffer from vanishing gradients over long sequences.
The TempFuser Architecture
The core of the proposed solution is the Long Short-Term Temporal Fusion Transformer (TempFuser).
The architecture employs a dual-scale input strategy:
- Short-Term Dense Trajectory ($s_t^s$): Captures immediate physics from the most recent frames to inform agile maneuvers.
- Long-Term Sparse Trajectory ($s_t^l$): Samples state history over a longer duration to capture the overall "flow" of the engagement.
These trajectories are processed by separate LSTM embeddings and then fused using a Transformer Encoder. The self-attention mechanism allows the model to dynamically weigh the importance of immediate aerodynamic constraints against long-term tactical objectives.
Training and Evaluation in DCS
The agent was trained using Soft Actor-Critic (SAC) within the Digital Combat Simulator (DCS World), a high-fidelity environment. To ensure robust performance, the agent (an F-16) was pitted against a variety of opponents, including the F-15E, F/A-18A, and the aerodynamically superior Su-27. An energy-embedded reward function was utilized to penalize inefficient maneuvers, forcing the agent to master "energy maneuverability."
Results and Emergent Tactics
TempFuser demonstrated significant superiority over the built-in "DCS-Ace" AI and standard DRL baselines. It achieved a 93.5% win rate against the F-15E and a 92.5% win rate against the Su-27. Notably, against the Su-27, a standard MLP-based agent achieved only a 1.7% win rate, highlighting the necessity of the temporal fusion approach.
Without explicit instruction, the agent "discovered" classic Basic Fighter Maneuvers (BFM):
- Low Yo-Yo: Trading altitude for speed to close the distance during a turn.
- Flat Scissors: Forcing an opponent to overshoot through a series of horizontal turns.
The model also showed high generalization capabilities, maintaining an 86.1% win rate against the Su-30, an aircraft it had never encountered during training.
Conclusion and Future Work
TempFuser successfully bridges the gap between tactical planning and agile execution in aerial combat. While currently limited to 1v1 scenarios, the multi-scale temporal fusion approach offers a promising path forward for complex robotics tasks. Future research will focus on multi-agent (N-vs-N) engagements and addressing the sim-to-real gap for deployment on physical unmanned aerial vehicles (UAVs).
Summary updated