Abstract
Video Snapshot Compressive Imaging (SCI) captures high-speed motion by compressing multiple temporal frames into a single 2D measurement. However, reconstructing high-quality video from these measurements remains computationally prohibitive for large-scale data. This paper introduces EfficientSCI, a novel end-to-end architecture that achieves state-of-the-art reconstruction quality with significantly reduced computational overhead. By employing hierarchical dense connections and space-time factorization, EfficientSCI enables the first high-fidelity reconstruction of UHD color video from a single snapshot.
Introduction
Video SCI systems offer a cost-effective alternative to high-speed cameras by using a standard sensor and a temporal modulation mask. The primary challenge is the reconstruction algorithm, which must solve an ill-posed inverse problem. While deep learning models have improved accuracy, they often suffer from high memory consumption and slow inference speeds, particularly when scaling to 4K/UHD resolutions. EfficientSCI addresses these limitations through a design focused on memory efficiency and linear computational complexity.
Proposed Method: EfficientSCI
The EfficientSCI architecture is built upon two core innovations: the ResDNet block and the CFormer module.
Hierarchical Dense Connections (ResDNet)
Traditional residual networks often lack sufficient feature reuse. EfficientSCI introduces the ResDNet Block, which incorporates dense connections within a single residual unit. This design aggregates features at a more granular level, boosting learning capacity while reducing overall model complexity compared to standard dense architectures.
Space-Time Factorization (CFormer)
To handle the high-dimensional nature of video data efficiently, the network utilizes CFormer blocks. These blocks factorize the processing into spatial and temporal domains:
- Spatial Domain: 2D convolutions are used to exploit local spatial correlations, maintaining low memory usage and high speed.
- Temporal Domain: A Temporal Self-Attention Branch (TSAB) captures long-range dependencies across frames. By restricting attention to the temporal axis, the model avoids the quadratic computational explosion typical of full 3D Transformers relative to spatial resolution.
Experiments and Results
The model was evaluated on various benchmarks, ranging from 256×256 grayscale simulations to UHD (1644×3840) color videos.
Quantitative Performance
EfficientSCI-L (the "Large" variant) achieves a PSNR of 36.92 dB on standard grayscale benchmarks, setting a new state-of-the-art. More importantly, the "Tiny" variant (EfficientSCI-T) provides a significant speedup—nearly 8x faster than previous leading models like DUN-3DUnet—while maintaining competitive quality.
Efficiency and Scalability
A key highlight is the model's ability to reconstruct UHD video. On the "Football" UHD dataset, EfficientSCI-S achieved a 2.5 dB improvement over PnP-FastDVDnet while being approximately 200 times faster, reducing processing time from hours to minutes. The linear complexity of the temporal attention mechanism ensures that memory usage remains manageable even for extremely high resolutions.
Conclusion
EfficientSCI represents a significant advancement in video snapshot compressive imaging. By combining the strengths of CNNs for spatial processing and Transformers for temporal modeling, it breaks the computational barriers that previously limited SCI to low-resolution applications. The architecture's efficiency and scalability open new possibilities for real-time high-speed imaging and large-scale video reconstruction tasks, including potential applications in spectral SCI.
Summary updated