Introduction
Autonomous vehicles rely on a 360° surround view for safe navigation. However, high-quality panoramic data is expensive to collect and annotate. Most researchers currently use "stitched" images from multiple pinhole cameras, which are often riddled with artifacts: misalignments at seams, exposure differences, and missing information. Standard generative models trained on this "messy" data inherit these errors, producing incoherent panoramas that can confuse vehicle AI.
Percep360 is a novel generative framework designed to solve this by "hallucinating" high-quality, coherent 360° panoramic street-view images. It treats the panorama as a continuous spherical manifold and uses a dynamic prompting mechanism to ensure both spatial continuity and geographic accuracy.
Methodology
The authors introduce two primary innovations within a Latent Diffusion Model (LDM) architecture, utilizing a side-control branch to process Bird's Eye View (BEV) maps and depth information.
Local Scenes Diffusion Method (LSDM)
Instead of treating a panoramic image as a flat rectangle, LSDM treats it as a spherical manifold. During training, the model applies random horizontal "circular shifts" (rotations) to the input images and control signals. This forces the network to learn that the far-left edge of the image is adjacent to the far-right edge, effectively "erasing" the seams and ensuring spatial continuity across the entire 360° field.
Probabilistic Prompting Method (PPM)
To maintain "controllability"—ensuring the generated image matches the intended road layout—PPM dynamically modulates the attention mechanism. It integrates multiple control cues, such as depth maps and text descriptions, and uses an element-wise maximum operation on attention maps to select the most relevant spatial priors for any given pixel. This prevents the model from "drifting" away from the ground truth while it attempts to fix stitching errors.
Experimental Results
The framework was evaluated on the nuScenes-360 dataset, which contains over 34,000 panoramic images stitched from standard six-camera setups.
Image Quality and Coherence
Percep360 achieved superior scores in No-Reference Image Quality Assessment (NRIQA). Specifically, it reached a PIQE metric of 11.44, surpassing even the original stitched "ground truth" images (12.06). This suggests the model performs a form of "super-resolution" and "inpainting" that makes the synthetic data more useful than the original imperfect captures.
Downstream Perception Impact
The ultimate validation was training a Bird’s Eye View (BEV) segmentation model. Using Percep360-generated data led to a 2.5% improvement in mIoU (mean Intersection over Union) compared to models trained without synthetic data. It also outperformed existing state-of-the-art models like MagicDrive and BEVControl in terms of both visual coherence and layout accuracy.
Conclusion
Percep360 demonstrates that generative models can learn from imperfect data to produce spatially continuous, high-fidelity scenes. By mathematically enforcing horizontal continuity and using probabilistic prompting for control, the system provides a powerful tool for generating high-quality synthetic data for autonomous driving. Future work will focus on direct 3D feature exploitation and more complex textual prompts to further diversify generated environments.
Summary updated