Abstract
The compressed sensing (CS) reconstruction of stitched images is critical for communication in autonomous driving, intelligent vehicles, and unmanned aerial vehicles (UAVs). This paper proposes a novel CS reconstruction method specifically for stitched images, integrating an image stitching method, a Deformable Convolution Module (DCM), SCNet, and a Deformable Deconvolution Module (DDM). The DCM is designed to extract features from stitched images effectively despite geometric distortions, while the DDM ensures consistency between the original stitched image and its reconstruction. Experimental results demonstrate that the proposed method outperforms standard image stitching and SCNet approaches in terms of Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR), providing superior visual quality.
Introduction
In fields such as autonomous driving and drone surveillance, a wide Field of View (FoV) is essential for situational awareness. This is typically achieved by stitching multiple images into a single panoramic view. However, transmitting these high-resolution stitched images over wireless networks poses a significant data bottleneck. Compressed Sensing (CS) offers a solution by compressing images during acquisition for later reconstruction.
Traditional CS reconstruction models often rely on fixed-grid convolutions, which struggle with the geometric distortions and irregular feature alignments common at the seams of stitched images. This leads to artifacts and loss of detail. The proposed DCM-DDM-SCNet architecture addresses these challenges by treating geometric distortion as a learnable feature rather than noise.
Methodology
The proposed pipeline consists of four primary stages:
- Image Stitching: Multiple source images are combined into a wide-FoV image using standard stitching algorithms.
- Deformable Convolution Module (DCM): Unlike standard convolutions that sample pixels in a rigid square grid, DCM learns spatial offsets. This allows the network to adapt its receptive field to the actual shapes and distortions present in the stitched image, making feature extraction much more robust.
- SCNet Backbone: A standard deep learning architecture used for the core compressed sensing reconstruction task.
- Deformable Deconvolution Module (DDM): Mirroring the DCM, the DDM ensures geometric consistency during the upsampling phase, preserving the spatial context of the reconstructed features and maintaining consistency with the original stitched input.
Experimental Results
The method was evaluated using two specific test cases, "Scene 1" and "Scene 2." Quantitative analysis shows significant improvements over the baseline (Image Stitching + SCNet).
At a compression ratio of 0.01 (99% compression), the proposed method achieved a PSNR of 15.39 dB and an SSIM of 60.45, compared to 14.17 dB and 58.35 for the baseline. At a higher CS ratio of 0.04, the PSNR reached 18.58 dB, a significant jump over the baseline's 16.43 dB.
Qualitative results demonstrate that the DCM-DDM-SCNet method produces sharper edges and fewer blocky artifacts in the reconstructed scenes compared to standard SCNet.
The system operates at approximately 10 Frames Per Second (FPS), with the stitching process taking roughly 0.33 to 0.34 seconds per scene. This minor speed penalty compared to the baseline (11 FPS) is a viable trade-off for the significant gains in image clarity.
Discussion and Conclusion
The integration of deformable modules allows the network to handle the unique geometric properties of stitched images effectively. By maintaining end-to-end consistency through DCM and DDM, the model preserves the integrity of stitched boundaries. While the results are promising, future work will focus on expanding dataset diversity to include environments like night-time driving and high-speed aerial footage. Additionally, further optimization of the deformable modules is necessary to reach the 30+ FPS required for real-time autonomous vehicle deployment.