Abstract
This paper provides a comprehensive survey of the evolution of 2D object detection, tracing its journey from early geometric-based methods to modern deep learning architectures. Crucially, it explores how these detection techniques serve as a backbone for complex video event recognition, enabling computers to move beyond simple image classification toward understanding dynamic human activities.
Introduction
While computer vision has made massive strides in identifying "what" is in an image, understanding "what is happening" in a video remains a significant challenge. Traditional approaches often rely on global video features extracted via 3D CNNs, which frequently fail at fine-grained tasks—for instance, distinguishing between a "birthday party" and an "anniversary" because both involve similar crowds and celebrations.
The authors argue that the key to solving this lies in robust 2D object detection. By identifying and tracking specific entities (like a baseball bat or a specific player) frame-by-frame, systems can build a more nuanced understanding of temporal events. This paper bridges the gap between static image detection and dynamic video understanding.
Architectural Evolution in Object Detection
The paper categorizes object detection into three main eras and architectural styles:
- Early Geometrical & Shape-based Approaches: Before the deep learning boom, detection relied on pattern matching and "pictorial structures." These methods broke objects down into fundamental parts with fixed spatial relationships.
- Monolithic (One-Stage) Architectures: Models such as YOLO (You Only Look Once) and SSD (Single Shot Detector) treat detection as a single regression problem. They take an image and directly predict bounding boxes and class probabilities in one pass, prioritizing speed for real-time applications.
- Two-Stage Architectures: Models like the R-CNN family (Faster R-CNN, Mask R-CNN) first generate "region proposals" and then classify those regions. While generally slower than one-stage models, they historically offered higher accuracy.
Recent shifts have also moved toward Vision Transformers (ViT) and anchor-free methods like CornerNet, which simplify the detection pipeline by removing the need for predefined "anchor boxes."
Benchmarking and Datasets
The field has been driven forward by a wide array of benchmarks:
- General Purpose: PASCAL VOC and MS COCO, the latter being the current gold standard.
- Specialized Domains: DOTA and xView for aerial imagery; COD10k for camouflaged objects.
- Large-Scale/Long-Tail: LVIS, featuring a vocabulary of 1,000 categories to test handling of rare objects.
- Video-Specific: YouTube-BoundingBoxes (YT-BB) and OAK, which provide annotations across video frames to support temporal tracking.
Temporal Tracking and Event Recognition
Object detection is a primary step for downstream tasks like instance segmentation and activity recognition. In video event recognition, the authors emphasize using Intersection-over-Union (IoU) overlap between consecutive frames. Since objects move minimally between frames, high IoU helps maintain "same-instance tracking," which is vital for understanding actions.
For fine-grained discrimination (e.g., "Musical Chairs"), tracking the specific inter-frame motion of people and objects is more effective than global scene analysis.
Performance Analysis
A comparative performance summary on the MS COCO dataset reveals:
- Top Performers: The Swin Transformer currently leads with a 58.7 mAP, followed by DetectoRS at 55.7 mAP.
- Efficiency vs. Accuracy: While monolithic models like YOLOv4 (43.5 mAP) are faster, two-stage models like Cascade R-CNN (45.8 mAP) maintain a slight edge in precision.
- Video Success: In sports analytics, Mask R-CNN has proven effective in detecting small objects like balls, which is critical for accurate activity classification.
Limitations and Future Directions
Several hurdles remain, including computational complexity and the difficulty of detecting small or crowded objects in dense scenes. There is also a growing need for "explainable" object detection to avoid false negatives in critical applications like surveillance or rescue operations. Future research is pointing toward few-shot/zero-shot learning and 3D object detection in point clouds for autonomous vehicles.
Summary updated