TL;DR
This paper introduces OSLO-IC, an improved end-to-end learned compression framework for 360-degree images. By incorporating attention modules, residual blocks, and a novel spherical transposed convolution for upsampling, OSLO-IC achieves a significant 23.1% bitrate reduction while also reducing model complexity.
What Problem Does This Paper Address?
The increasing prevalence of virtual reality and autonomous driving technologies has led to a surge in demand for high-resolution omnidirectional (360-degree) images. Storing and transmitting these images efficiently presents a significant challenge due to their large size. Traditional 2D image compression techniques are not directly applicable to omnidirectional images because of the inherent geometric distortions introduced by common representations like the Equirectangular Projection (ERP). While previous "On-the-Sphere Learning" (OSLO) frameworks have shown promise, they haven't incorporated recent advancements from 2D compression, such as attention mechanisms and more efficient upsampling methods, leading to suboptimal performance and larger model sizes.
Methods
OSLO-IC builds upon the OSLO framework by introducing several key enhancements:
- Spherical Attention Modules: Inspired by 2D compression, these modules help the model focus on important image regions, allocating more bits to complex areas and fewer to simpler ones.
- Residual Blocks: These replace the Generalized Divisive Normalization (GDN) layers, offering improved performance for modeling Gaussian densities and increasing receptive fields.
- Spatial Autoregressive Context Model: This model leverages a spherical masked convolution to predict the distribution of mean and standard deviations of latent representations, improving compression efficiency.
- Spherical Transposed Convolution: This is a novel upsampling operator designed to replace the computationally expensive pixel shuffling method. It significantly reduces the number of trainable parameters (by a factor of four) while maintaining comparable expressiveness and compression performance.
Datasets
The authors used the SUN360 dataset, which contains 2170 ERP images of resolution 9104 × 4552. This dataset was split into 1737 training, 10 validation, and 423 testing images, following the split used in prior work [9]. During training, random patches equivalent to 256 × 256 were used.
Key Insights and Ideas
- Bridging the Gap: The paper successfully demonstrates that recent advancements in 2D learned image compression can be effectively adapted to the unique challenges of omnidirectional image compression.
- Efficiency Through Novelty: The spherical transposed convolution is a significant contribution, offering a substantial reduction in model complexity without sacrificing compression performance. This is crucial for deploying compression models on resource-constrained devices.
- Context is Key: The spatial autoregressive context model highlights the importance of considering neighboring pixel information for more accurate latent representation prediction, leading to better compression.
- Attention Improves Focus: The integration of attention modules shows that directing computational resources to salient image features is as beneficial for spherical images as it is for planar ones.
Key Results
- Bitrate Reduction: OSLO-IC achieves a 23.1% bitrate reduction compared to the spherical scale hyperprior model, measured in terms of WS-PSNR BD rate.
- Model Complexity: The proposed spherical transposed convolution reduces the number of trainable parameters by a factor of four compared to pixel shuffling, leading to a more efficient model.
- Attention and Residual Blocks: These modules alone contribute an approximate 10% bitrate reduction with only a moderate increase in trainable parameters.
- Transposed Convolution Benefits: Models using the spherical transposed convolution consistently achieve better reconstruction quality at higher bitrates compared to their pixel shuffling counterparts.
Limitations and Future Work
The authors acknowledge that the number of trainable parameters in their full proposed model is still considerably higher than some baseline models, primarily due to the pixel shuffling operation (though their transposed convolution addresses this).
Future work includes:
- Extending the OSLO framework with a computationally efficient channel-wise context model.
- Exploring different model architectures, including transformer-based networks, for further improvements.
Summary updated