Abstract
The employment of convolutional neural networks has led to significant performance improvement on the task of object detection. However, when applying existing detectors to continuous frames in a video, we often encounter momentary miss-detection of objects, that is, objects are undetected exceptionally at a few frames, although they are correctly detected at all other frames. In this paper, we analyze the mechanism of how such miss-detection occurs. For the most popular class of detectors that are based on anchor boxes, we show the followings: i) besides apparent causes such as motion blur, occlusions, background clutters, etc., the majority of remaining miss-detection can be explained by an improper behavior of the detectors at boundaries of the anchor boxes; and ii) this can be rectified by improving the way of choosing positive samples from candidate anchor boxes when training the detectors.
Introduction
Modern object detectors, while highly accurate on static images, often exhibit temporal instability when applied to video sequences. A primary manifestation of this instability is Momentarily Missed Detection (MMD): a phenomenon where an object is correctly identified in consecutive frames $t-1$ and $t+1$ but fails to be detected in frame $t$, despite no significant change in visibility or appearance.
This "flickering" behavior is problematic for downstream applications like tracking or autonomous navigation. This research investigates the root causes of MMD, categorizing them into external environmental factors and internal structural flaws within anchor-based detection architectures.
Analysis of Missed Detections
The authors categorize MMD into two main types:
- External Factors: These include motion blur, partial or full occlusions, and cluttered backgrounds that confuse the detector.
- Internal Factors: These are misses that occur even in clear frames, hypothesized to be caused by the detector's internal mechanics.
Through a novel Image Warping analysis—applying minute geometric shifts to frames—the researchers discovered that internal MMD often occurs at the boundaries of anchor boxes. As an object moves, the responsibility for its detection shifts between predefined anchors. At the exact boundary (in terms of scale, position, or aspect ratio), the confidence scores of neighboring anchors often drop simultaneously, creating a "valley" where the detection falls below the required threshold.
Proposed Solution: Soft Sampling
Standard training for anchor-based detectors (like SSD or Faster R-CNN) uses a "hard" Intersection over Union (IOU) threshold to assign labels. An anchor is a positive sample if its IOU with a ground truth box exceeds a fixed value (e.g., 0.5), and negative otherwise. This creates sharp, brittle decision boundaries.
To mitigate this, the authors propose a Soft Sampling Method. Instead of binary labels, they use a logistic sigmoid function to assign continuous weights to anchor boxes based on their IOU. This approach encourages the network to produce overlapping confidence peaks, effectively filling the "valleys" at anchor boundaries and providing smoother transitions as objects move through the scene.
Experimental Results
The method was evaluated using the DAVIS 2017 video dataset for MMD analysis and PASCAL VOC for standard accuracy metrics.
- MMD Reduction: The soft sampling method significantly improved temporal consistency. In an SSD-VGG16 model, the number of MMD frames attributed to internal anchor boundary issues was reduced from 73 to 11.
- Robustness to External Factors: Interestingly, the method also reduced some misses caused by external factors like blur, suggesting that the increased structural robustness helps the model maintain detections under slight image degradation.
- Detection Accuracy: Crucially, this gain in stability did not come at the cost of accuracy. The Mean Average Precision (mAP) on PASCAL VOC slightly increased (e.g., from 76.5% to 77.2% for SSD-VGG16).
Conclusion
This work identifies that a significant portion of temporal instability in video object detection is an artifact of anchor-based training regimes. By replacing hard IOU thresholds with soft sampling, detectors can achieve much higher temporal stability without sacrificing—and even slightly improving—standard detection performance. While external factors like heavy occlusion remain a challenge, addressing internal boundary issues is a vital step toward reliable video-based perception.
Summary updated