Abstract
Temporal Action Detection (TAD) in untrimmed videos requires simultaneously identifying action boundaries and categories. Existing one-stage detectors often struggle with a feature representation conflict: classification tasks favor feature invariance, while localization tasks require sensitivity to temporal offsets. This paper presents Decouple-SSAD, a one-stage architecture that resolves this conflict by employing specialized, parallel branches for classification and localization. By integrating high-level semantic context via deconvolutional layers, the model achieves state-of-the-art performance on the THUMOS14 benchmark, significantly narrowing the gap between one-stage and two-stage detection frameworks.
Introduction
Temporal Action Detection (TAD) is a critical task in video understanding, requiring the precise localization of action instances within long, untrimmed video streams. Current methodologies generally follow two paradigms: two-stage frameworks, which generate proposals before classification, and one-stage frameworks, which perform both tasks simultaneously. While one-stage methods offer superior efficiency, they historically lag in accuracy.
The authors identify a fundamental "tug-of-war" in shared-feature one-stage designs. Classification aims to group similar actions together, whereas localization must distinguish between clips based on precise timing offsets. This conflict often leads to suboptimal feature representations. Decouple-SSAD addresses this by explicitly separating the feature learning processes for each sub-task.
Methodology
The proposed Decouple-SSAD (Decoupled Single Shot temporal Action Detection) architecture introduces a multi-branch design to handle the divergent requirements of TAD:
- Main Stream: A standard one-stage backbone that provides initial joint predictions.
- Classification Branch: A parallel path focused on action identification. It utilizes deconvolutional layers to pass semantic information from deeper layers back to shallower ones, providing necessary context for detecting short-duration actions.
- Proposal Branch: A dedicated path for refining temporal boundaries (start and end timestamps).
The network is trained end-to-end, allowing branches to benefit from joint optimization while maintaining specialized "private" feature maps. Final detections are produced through an average fusion strategy that combines outputs from all three components.
Experimental Results
The model was evaluated on the THUMOS14 dataset, using feature extractors pre-trained on UCF101 and Kinetics.
Performance Analysis
Decouple-SSAD demonstrates significant improvements over baseline one-stage architectures. Using UCF101 features, the model achieved a 35.8% [email protected], a 4.6% increase over the non-decoupled baseline. When utilizing more advanced Kinetics features, performance reached 44.2% [email protected], highlighting the architecture's ability to leverage high-quality data representations.
Comparison with State-of-the-Art
As shown in the comparative analysis, Decouple-SSAD outperforms previous leading one-stage methods like SS-TAD by 6.6% and remains highly competitive with complex two-stage systems such as BSN, particularly at higher IoU thresholds.
Conclusion
Decouple-SSAD proves that architectural specialization is key to high-performance one-stage temporal action detection. By mitigating the feature conflict between classification and localization, the model achieves state-of-the-art results while maintaining the efficiency of a single-shot framework. Future research will focus on extending this decoupling principle to spatio-temporal detection and developing fully end-to-end systems that operate directly on raw video pixels.
Summary updated