Abstract
Action recognition in low-light environments is a critical challenge for computer vision applications such as autonomous robotics and night-time surveillance. Traditional models often fail due to low contrast and significant sensor noise. This paper presents a two-stream architecture that processes raw dark video alongside an enhanced light stream generated by a lightweight Self-Calibrated Illumination (SCI) module. By utilizing a weight-shared R(2+1)D backbone and a Graph Convolutional Network (GCN) for temporal feature fusion, the proposed method achieves state-of-the-art performance on the ARID dataset, reaching 95.86% Top-1 accuracy with improved parameter efficiency.
Introduction
Identifying human actions in video is a fundamental task in computer vision. However, most existing research assumes well-lit conditions, leaving a gap in performance for low-light scenarios. Dark videos suffer from reduced visibility, heavy noise, and a lack of distinct spatio-temporal features. Previous attempts to solve this often relied on computationally expensive image enhancement techniques that do not necessarily translate well to video action recognition. This work addresses these issues by proposing an efficient dual-stream approach that leverages both raw and enhanced data.
Proposed Methodology
The architecture consists of three primary components designed for efficiency and accuracy in dark environments.
Dual-Stream Processing
The network employs two parallel streams:
- Dark Stream: Processes raw, low-light video frames directly to preserve deep textural features that might be obscured or distorted by enhancement algorithms.
- Light Stream: Uses a Self-Calibrated Illumination (SCI) module to brighten frames. SCI is a cascaded, lightweight framework that provides illumination enhancement without the high computational overhead of traditional deep learning-based enhancers.
Weight-Shared Feature Extraction
Both streams utilize an R(2+1)D backbone for feature extraction. This architecture decomposes 3D convolutions into separate spatial and temporal components, which enhances learning capacity while reducing complexity. To minimize the model's footprint, the two streams share weights, allowing the network to learn a unified feature representation for both dark and enhanced inputs.
Temporal Graph Encoder
The extracted features from both streams are concatenated and fed into a three-layer Graph Convolutional Network (GCN). The GCN serves as a temporal graph encoder, modeling the relationships between frames over time. This step effectively "denoises" the temporal signal and selects the most salient spatio-temporal information for final classification.
Experimental Evaluation
The model was benchmarked using the Action Recognition in Dark (ARID) dataset, which contains real-world low-light video sequences.
Performance Comparison
The proposed method was compared against 15 state-of-the-art (SOTA) models, including various 3D CNN architectures and previous dual-stream approaches.
As shown in the results table, the proposed method achieves a Top-1 accuracy of 95.86%, significantly outperforming the previous best model, DarkLight-R(2+1)D-34, which reached 94.04%. Furthermore, the model achieved a Top-5 accuracy of 99.87%.
Efficiency Gains
Beyond accuracy, the proposed architecture is notably more efficient. By employing weight sharing and the R(2+1)D backbone, the model reduces the total parameter count by over one million compared to the DarkLight baseline. This makes it a more viable candidate for deployment on edge devices with limited computational resources.
Conclusion
This research demonstrates that combining raw and enhanced video streams through a GCN-based fusion mechanism provides a robust solution for action recognition in the dark. The synergy between the dark stream's texture preservation and the light stream's clarity, coupled with efficient temporal encoding, sets a new standard for the ARID benchmark. Future work will focus on hyperparameter optimization and improving robustness in near-zero light conditions.
Summary updated