Introduction
Temporal Action Detection (TAD) is a fundamental task in video understanding that involves identifying the start and end times of specific actions within untrimmed video sequences. Despite significant progress, two primary challenges persist: Movement Feature Submergence and Scale Variability.
Movement Feature Submergence occurs when static background context dominates the visual features, making it difficult to distinguish between an action and its surrounding environment. For instance, a person holding an accordion looks nearly identical to a person playing one if the model focuses solely on appearance. Scale Variability refers to the wide range of action durations, from a few seconds to several minutes, which standard architectures often fail to capture uniformly.
The Movement Enhance Network (MENet)
To address these issues, this paper introduces the Movement Enhance Network (MENet). The architecture is designed to isolate dynamic movement from static context and specialize feature layers for different action scales.
Movement Enhance Module (MEM)
The MEM is the core component designed to combat background noise. It utilizes a Movement Feature Extractor (MFE) based on a Siamese network architecture.
As shown in the figure above, the MFE processes a video snippet through two paths:
- Video Snippet Path: Extracts features ($f_{Total}$) from the dynamic sequence.
- Frame Duplication Path: Extracts features ($f_{Static}$) from a single duplicated static frame.
By calculating the difference ($f_{Movement} = f_{Total} - f_{Static}$), the module effectively "denoises" the video features, highlighting the changes that represent the actual action. These features are further refined by the Multi-Relation Enhance Module (MREM), which combines 1D convolutions for local temporal context and LSTMs for long-range dependencies.
Scale Feature Pyramid Network (SFPN)
To handle scale variability, MENet employs a Scale Feature Pyramid Network (SFPN) with a unique Two-Stage Training Strategy.
- Generalization Stage: The network is initially trained to recognize actions across all scales.
- Specialization Stage: Each level of the feature pyramid is assigned a specific temporal scale range. Layers are biased during training to become "experts" in their assigned range (e.g., top layers for short actions, bottom layers for long actions). This specialization prevents the "jack of all trades" performance drop typical in standard FPNs.
Experimental Evaluation
MENet was evaluated on two major benchmarks: ActivityNet-1.3 and THUMOS-14.
- ActivityNet-1.3: The model achieved a state-of-the-art average mAP of 37.7%, a 1.1% improvement over existing methods.
- THUMOS-14: MENet demonstrated superior performance in high-precision scenarios. At a tIoU threshold of 0.7, it reached 34.0% mAP, significantly outperforming previous leaders like ActionFormer.
Qualitative analysis indicates that MENet produces more precise boundary localizations, especially in cluttered scenes where movement is subtle relative to the background.
Conclusion
MENet effectively addresses the limitations of current TAD systems by explicitly isolating movement features and implementing a specialized multi-scale training strategy. By "subtracting" the background and training pyramid layers as scale experts, the model achieves significant performance gains on competitive benchmarks, particularly in boundary precision. Future work may explore adaptive scale boundaries and the integration of Video Transformer backbones to further enhance feature representation.
Summary updated