TL;DR
This paper introduces "Adaptive Distance Detection" (ADD) and "Adaptive Instance Segmentation" (AIS), two novel architectures designed to balance the high-stakes trade-off between speed and precision in real-time FPS game environments. By using a lightweight classifier to switch between fast object detection and pixel-perfect segmentation based on target distance and count, the author achieves a "best of both worlds" approach for enemy detection in Counter-Strike: Condition Zero.
What Problem Does This Paper Address?
In competitive first-person shooters (FPS), reaction time—or "latency"—is the difference between winning and losing. For an AI agent, this translates to inference time.
Current computer vision techniques offer a difficult choice:
- Object Detection (e.g., YOLO): Extremely fast but provides only a bounding box, which can be imprecise for aiming at distant or oddly shaped targets.
- Semantic Segmentation (e.g., U-Net): Highly precise at a pixel level but computationally expensive.
The author identifies a critical "exponential blowup" in latency: as models become more complex, the inference time for segmentation grows significantly faster than for object detection. The paper seeks a way to maintain high precision for difficult targets without sacrificing the frame rate needed for close-quarters combat.
Methods
The author proposes two "Adaptive" frameworks that use a custom CNN classifier as a decision-making bridge:
- Adaptive Distance Detection (ADD): A classifier determines if an enemy is "near" or "far."
- Near: The system uses YOLOv5s. The logic is that for large, close targets, the proportional error of a bounding box is negligible.
- Far: The system switches to a custom U-Net semantic segmentation model to ensure pixel-perfect accuracy for small targets.
- Adaptive Instance Segmentation (AIS): This addresses the "overkill" of using panoptic segmentation (which identifies individual instances) on every frame.
- If only one enemy is present, it uses standard Semantic Segmentation.
- If multiple enemies are detected, it switches to Maskformer (Panoptic Segmentation) to differentiate between targets.
The "bridge" classifier is a custom 8-layer CNN utilizing ReLU activations and a Sigmoid output to handle the binary decision-making (Near vs. Far or Single vs. Multiple).
Datasets
The study relies on a mix of public and custom-labeled data:
- Object Detection: A Roboflow dataset for Counter-Strike containing 248 images.
- Semantic Segmentation: A custom-built dataset of 33 images from CS: Condition Zero, manually annotated using Label Studio.
- Classifiers: Custom datasets of 102 images (for ADD) and 350 images (for AIS), augmented with rotations, flips, and brightness changes to prevent overfitting.
- Panoptic Segmentation: A pre-trained Maskformer model trained on the ADE20K dataset.
Key Insights and Ideas
- The Proportional Error Argument: One of the most intuitive "aha moments" is the author's geometric argument. Using a circle-in-a-square analogy, they demonstrate that as a target gets larger (closer), the "wasted space" or error in a bounding box becomes proportionally smaller relative to the target's area. This justifies using "looser" object detection for close-up enemies.
- Latency Scaling: The paper highlights that while the speed difference between YOLO and U-Net might seem small on simple models (e.g., 0.27s vs 0.43s), this gap widens exponentially as the models scale, making adaptive switching mandatory for high-performance systems.
- Computational Efficiency: AIS recognizes that panoptic segmentation is a heavy hammer. By only "powering up" the panoptic model when multiple instances exist, the system saves significant clock cycles.
Key Results
The adaptive methods successfully found a middle ground in the accuracy-latency spectrum:
- ADD Performance: While pure semantic segmentation was the most accurate (0.56 IoU) but slowest (0.48s), and YOLO was the fastest (0.28s) but least precise (0.18 IoU), Adaptive Distance Detection balanced these at 0.36 IoU and 0.42s latency.
- AIS Performance: The adaptive approach achieved a 0.72 IoU with an inference time of 1.87s, nearly doubling the speed of pure panoptic segmentation (3.83s) while maintaining comparable accuracy.
- Classifier Success: The custom classifiers reached 100% test accuracy on the ADD task and 97% on the AIS task, proving that a lightweight "bridge" model is viable for real-time switching.
Limitations and Future Work
- Dataset Scale: The custom segmentation dataset was extremely small (33 images). This led to qualitative errors, such as the model occasionally segmenting the player's own hand or knife as an "enemy."
- Classifier Latency: While the classifier is lightweight, it still adds a small overhead. In real-world testing, the "perfect" theoretical speedup was slightly dampened by the time taken for the classifier to run.
- Generalization: The models were trained specifically on Condition Zero. The author suggests that future work should involve training on more diverse game environments to create a more "universal" FPS bot.
- The "Bot" Integration: The current research focuses on detection. The next logical step is integrating these adaptive outputs into an automated aiming and shooting system.
Summary updated