Beyond-Visual-Range Air Combat Tactics Auto-Generation by Reinforcement Learning

Haiyin Piao, Zhixiao Sun, Guanglei Meng, Hechang Chen, Bohao Qu, Kuijun Lang, Yang Sun, Shengqi Yang, Xuanqi PengIJCNN 2020 · OpenReview

Abstract

For quite a long time, effective Beyond-Visual-Range (BVR) air combat tactics can only be discovered by human pilots in the actual combat process. However, due to the lack of actual combat opportunities, making new air combat tactics innovation was generally considered quite difficult. To address this challenge, we first introduced a solely end-to-end Reinforcement Learning (RL) approach for training competitive air combat agents with adversarial self-play from scratch in a high fidelity air combat simulation environment during training. Furthermore, a Key Air Combat Event Reward Shaping (KAERS) mechanism was proposed to provide sparse but objective shaped rewards beyond episodic win/lose signal to accelerate the initial machine learning process. Experimental results showed that multiple valuable air combat tactical behaviors emerged progressively. We hope this study could be extended to the future of air combat machine intelligence research.
BibTeX
@inproceedings{piao2020beyond,
  author = {Haiyin Piao and Zhixiao Sun and Guanglei Meng and Hechang Chen and Bohao Qu and Kuijun Lang and Yang Sun and Shengqi Yang and Xuanqi Peng},
  title = {{Beyond-Visual-Range Air Combat Tactics Auto-Generation by Reinforcement Learning}},
  booktitle = {IJCNN},
  year = {2020},
  url = {https://openreview.net/forum?id=KH7r43VQYCC},
}

Loading related content...

Figure