TL;DR
Researchers have developed a massive new gunshot dataset, BGG, by recording audio from the popular FPS game PlayerUnknown’s Battlegrounds (PUBG). By leveraging the high-fidelity acoustic engines of modern games, they demonstrated that models trained on synthetic "in-game" gunshots significantly outperform those trained on scarce real-world data for both identifying firearms and localizing shooters.
What Problem Does This Paper Address?
Deep learning has revolutionized sound classification, but it has a "data hunger" problem that is particularly acute for gunshots. In the real world, gunshots are rare, unpredictable, and—most importantly—dangerous to record. Existing audio datasets like AudioSet or UrbanSound8k contain very few gunshot samples (often less than 5%), leading to models that struggle to distinguish a rifle shot from a plastic bag bursting.
Furthermore, localizing a shooter (determining their distance and angle) requires precise ground-truth labeling that is incredibly difficult to capture in live environments. While some controlled datasets like Forensic exist, they lack the variety of "engagement" scenarios—such as the sound of a supersonic bullet whizzing past a listener—that are critical for military and law enforcement applications.
Methods
The authors treat both firearm identification and localization as classification problems. They recorded stereo audio in PUBG, varying the gun type, distance (0m to 600m), and direction.
The Pipeline:
- Preprocessing: Raw stereo waveforms are converted into spectrograms using a Short-time Fourier Transform (STFT).
- Feature Extraction: The authors tested several architectures, including DCNNs (Deep Convolutional Neural Networks), LSTMs, and CRNNs.
- The "Winner": A CNN-Transformer architecture performed best. It uses convolutional layers to extract local temporal patterns from the spectrogram and a Transformer block to learn long-distance dependencies.
- Multitask Learning: For localization, the model uses a shared feature extractor that feeds into two separate classifiers: one for distance and one for direction.
Datasets
The paper introduces the BGG (Battleground Gunshot) dataset, which is split into two subsets:
- BGG-CLS: Focused on classification, featuring 37 different firearm types plus a "No gun" noise class.
- BGG-LOC: Focused on localization, featuring 3 specific guns across 6 distances (up to 600m) and 5 directions.
The researchers also used UrbanSound8k and Forensic as real-world benchmarks to prove that their in-game data translates to the physical world.
Key Insights and Ideas
- Game Engines as Simulators: Modern game engines accurately reproduce complex acoustic phenomena. For example, the researchers noted that at 600 meters, the "blunt" sound of bullet friction (the crack as it passes the player) is more prominent than the "sharp" muzzle blast of the actual gunshot.
- Visualizing the Sound: Figure 3 (below) illustrates how distance and direction change the waveform. Notice the volume difference between the left and right channels depending on the shooter's position.
- Convolution is Essential: The study found that models relying solely on LSTMs or Transformers performed poorly. Convolutional layers are necessary to capture the "local temporal patterns"—the specific texture of a gunshot—within a spectrogram.
- Transfer Learning Works: One of the most important findings is that pre-training a model on BGG data and then fine-tuning it on real-world data (like the Forensic set) yields better results than training on real-world data alone. This confirms the "realism" of the game audio.
Key Results
- Top Performance: The CNN-Transformer achieved 93.6% accuracy on the BGG-CLS test set.
- Real-World Boost: Adding BGG data to the UrbanSound8k training set improved the model's ability to identify real-world gunshots significantly. For a DCNN, the "Gun accuracy" jumped from 26.6% to 47.1%.
- Localization: The CNN-Transformer reached 91.5% accuracy for distance and 93.1% for direction on the BGG-LOC dataset.
Limitations and Future Work
While the results are promising, the authors noted some confusion in distance estimation, particularly between 0m and 100m. This suggests that the model might benefit from a continuous regression approach rather than discretized distance classes.
Additionally, while PUBG includes environmental noises like wind and footsteps, real-world urban environments are much noisier. Future work could involve "stress-testing" these models against more complex, multi-source real-world audio environments.
Summary updated