Enemy Spotted: in-game gun sound dataset for gunshot classification and localization

Junwoo Park, Youngwoo Cho, Gyuhyeon Sim, Hojoon Lee, Jaegul Choo

TL;DR

Researchers have developed a massive new gunshot dataset, BGG, by recording audio from the popular FPS game PlayerUnknown’s Battlegrounds (PUBG). By leveraging the high-fidelity acoustic engines of modern games, they demonstrated that models trained on synthetic "in-game" gunshots significantly outperform those trained on scarce real-world data for both identifying firearms and localizing shooters.

What Problem Does This Paper Address?

Deep learning has revolutionized sound classification, but it has a "data hunger" problem that is particularly acute for gunshots. In the real world, gunshots are rare, unpredictable, and—most importantly—dangerous to record. Existing audio datasets like AudioSet or UrbanSound8k contain very few gunshot samples (often less than 5%), leading to models that struggle to distinguish a rifle shot from a plastic bag bursting.

Furthermore, localizing a shooter (determining their distance and angle) requires precise ground-truth labeling that is incredibly difficult to capture in live environments. While some controlled datasets like Forensic exist, they lack the variety of "engagement" scenarios—such as the sound of a supersonic bullet whizzing past a listener—that are critical for military and law enforcement applications.

Methods

The authors treat both firearm identification and localization as classification problems. They recorded stereo audio in PUBG, varying the gun type, distance (0m to 600m), and direction.

The Pipeline:

  1. Preprocessing: Raw stereo waveforms are converted into spectrograms using a Short-time Fourier Transform (STFT).
  2. Feature Extraction: The authors tested several architectures, including DCNNs (Deep Convolutional Neural Networks), LSTMs, and CRNNs.
  3. The "Winner": A CNN-Transformer architecture performed best. It uses convolutional layers to extract local temporal patterns from the spectrogram and a Transformer block to learn long-distance dependencies.
  4. Multitask Learning: For localization, the model uses a shared feature extractor that feeds into two separate classifiers: one for distance and one for direction.

Datasets

The paper introduces the BGG (Battleground Gunshot) dataset, which is split into two subsets:

  • BGG-CLS: Focused on classification, featuring 37 different firearm types plus a "No gun" noise class.
  • BGG-LOC: Focused on localization, featuring 3 specific guns across 6 distances (up to 600m) and 5 directions.
Fig. 1. The distribution of firearms in the BGG-CLS dataset is plotted. The blue and red bars indicate the number of train and test samples, respectively.

The researchers also used UrbanSound8k and Forensic as real-world benchmarks to prove that their in-game data translates to the physical world.

Key Insights and Ideas

  • Game Engines as Simulators: Modern game engines accurately reproduce complex acoustic phenomena. For example, the researchers noted that at 600 meters, the "blunt" sound of bullet friction (the crack as it passes the player) is more prominent than the "sharp" muzzle blast of the actual gunshot.
  • Visualizing the Sound: Figure 3 (below) illustrates how distance and direction change the waveform. Notice the volume difference between the left and right channels depending on the shooter's position.
Fig. 3. Sample audio samples are visualized in the waveform. All samples are recorded in stereo. For each sound sample, the left and the right channel audio samples are plotted at the top and the bottom, respectively.
  • Convolution is Essential: The study found that models relying solely on LSTMs or Transformers performed poorly. Convolutional layers are necessary to capture the "local temporal patterns"—the specific texture of a gunshot—within a spectrogram.
  • Transfer Learning Works: One of the most important findings is that pre-training a model on BGG data and then fine-tuning it on real-world data (like the Forensic set) yields better results than training on real-world data alone. This confirms the "realism" of the game audio.

Key Results

  • Top Performance: The CNN-Transformer achieved 93.6% accuracy on the BGG-CLS test set.
  • Real-World Boost: Adding BGG data to the UrbanSound8k training set improved the model's ability to identify real-world gunshots significantly. For a DCNN, the "Gun accuracy" jumped from 26.6% to 47.1%.
  • Localization: The CNN-Transformer reached 91.5% accuracy for distance and 93.1% for direction on the BGG-LOC dataset.

Limitations and Future Work

While the results are promising, the authors noted some confusion in distance estimation, particularly between 0m and 100m. This suggests that the model might benefit from a continuous regression approach rather than discretized distance classes.

Additionally, while PUBG includes environmental noises like wind and footsteps, real-world urban environments are much noisier. Future work could involve "stress-testing" these models against more complex, multi-source real-world audio environments.

Summary updated

Abstract

Recently, deep learning-based methods have drawn huge attention due to their simple yet high performance without domain knowledge in sound classification and localization tasks. However, a lack of gun sounds in existing datasets has been a major obstacle to implementing a support system to spot criminals from their gunshots by leveraging deep learning models. Since the occurrence of gunshot is rare and unpredictable, it is impractical to collect gun sounds in the real world. As an alternative, gun sounds can be obtained from an FPS game that is designed to mimic real-world warfare. The recent FPS game offers a realistic environment where we can safely collect gunshot data while simulating even dangerous situations. By exploiting the advantage of the game environment, we construct a gunshot dataset, namely BGG, for the firearm classification and gunshot localization tasks. The BGG dataset consists of 37 different types of firearms, distances, and directions between the sound source and a receiver. We carefully verify that the in-game gunshot data has sufficient information to identify the location and type of gunshots by training several sound classification and localization baselines on the BGG dataset. Afterward, we demonstrate that the accuracy of real-world firearm classification and localization tasks can be enhanced by utilizing the BGG dataset.
BibTeX
@inproceedings{park2022enemy,
  author = {Junwoo Park and Youngwoo Cho and Gyuhyeon Sim and Hojoon Lee and Jaegul Choo},
  title = {{Enemy Spotted: in-game gun sound dataset for gunshot classification and localization}},
  booktitle = {CoG},
  year = {2022},
  doi = {10.1109/cog51982.2022.9893670},
}
Figure