Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting

Fei Teng, Kai Luo, Sheng Wu, Siyu Li, Pujun Guo, Jiale Wei, Kunyu Peng, Jiaming Zhang, Kailun YangarXiv 2025 · DOI · arXiv

Introduction

Autonomous vehicles rely on a 360° surround view for safe navigation. However, high-quality panoramic data is expensive to collect and annotate. Most researchers currently use "stitched" images from multiple pinhole cameras, which are often riddled with artifacts: misalignments at seams, exposure differences, and missing information. Standard generative models trained on this "messy" data inherit these errors, producing incoherent panoramas that can confuse vehicle AI.

Percep360 is a novel generative framework designed to solve this by "hallucinating" high-quality, coherent 360° panoramic street-view images. It treats the panorama as a continuous spherical manifold and uses a dynamic prompting mechanism to ensure both spatial continuity and geographic accuracy.

Methodology

The authors introduce two primary innovations within a Latent Diffusion Model (LDM) architecture, utilizing a side-control branch to process Bird's Eye View (BEV) maps and depth information.

The overall Percep360 architecture, featuring the LSDM and PPM modules within a denoising diffusion framework.

Local Scenes Diffusion Method (LSDM)

Instead of treating a panoramic image as a flat rectangle, LSDM treats it as a spherical manifold. During training, the model applies random horizontal "circular shifts" (rotations) to the input images and control signals. This forces the network to learn that the far-left edge of the image is adjacent to the far-right edge, effectively "erasing" the seams and ensuring spatial continuity across the entire 360° field.

Probabilistic Prompting Method (PPM)

To maintain "controllability"—ensuring the generated image matches the intended road layout—PPM dynamically modulates the attention mechanism. It integrates multiple control cues, such as depth maps and text descriptions, and uses an element-wise maximum operation on attention maps to select the most relevant spatial priors for any given pixel. This prevents the model from "drifting" away from the ground truth while it attempts to fix stitching errors.

Detailed structures of LSDM (spherical manifold rotation) and PPM (probabilistic attention modulation).

Experimental Results

The framework was evaluated on the nuScenes-360 dataset, which contains over 34,000 panoramic images stitched from standard six-camera setups.

Image Quality and Coherence

Percep360 achieved superior scores in No-Reference Image Quality Assessment (NRIQA). Specifically, it reached a PIQE metric of 11.44, surpassing even the original stitched "ground truth" images (12.06). This suggests the model performs a form of "super-resolution" and "inpainting" that makes the synthetic data more useful than the original imperfect captures.

Downstream Perception Impact

The ultimate validation was training a Bird’s Eye View (BEV) segmentation model. Using Percep360-generated data led to a 2.5% improvement in mIoU (mean Intersection over Union) compared to models trained without synthetic data. It also outperformed existing state-of-the-art models like MagicDrive and BEVControl in terms of both visual coherence and layout accuracy.

Conclusion

Percep360 demonstrates that generative models can learn from imperfect data to produce spatially continuous, high-fidelity scenes. By mathematically enforcing horizontal continuity and using probabilistic prompting for control, the system provides a powerful tool for generating high-quality synthetic data for autonomous driving. Future work will focus on direct 3D feature exploitation and more complex textual prompts to further diversify generated environments.

Summary updated

Abstract

Panoramic perception holds significant potential for autonomous driving, enabling vehicles to acquire a comprehensive 360° surround view in a single shot. However, autonomous driving is a data-driven task. Complete panoramic data acquisition requires complex sampling systems and annotation pipelines, which are time-consuming and labor-intensive. Although existing street view generation models have demonstrated strong data regeneration capabilities, they can only learn from the fixed data distribution of existing datasets and cannot leverage stitched pinhole images as a supervisory signal. In this paper, we propose the first panoramic generation method Percep360 for autonomous driving. Percep360 enables coherent generation of panoramic data with control signals based on the stitched panoramic data. Percep360 focuses on two key aspects: coherence and controllability. Specifically, to overcome the inherent information loss caused by the pinhole sampling process, we propose the Local Scenes Diffusion Method (LSDM). LSDM reformulates the panorama generation as a spatially continuous diffusion process, bridging the gaps between different data distributions. Additionally, to achieve the controllable generation of panoramic images, we propose a Probabilistic Prompting Method (PPM). PPM dynamically selects the most relevant control cues, enabling controllable panoramic image generation. We evaluate the effectiveness of the generated images from three perspectives: image quality assessment (i.e., no-reference and with reference), controllability, and their utility in real-world Bird's Eye View (BEV) segmentation. Notably, the generated data consistently outperforms the original stitched images in no-reference quality metrics and enhances downstream perception models. The source code will be publicly available at https://github.com/FeiT-FeiTeng/Percep360.
BibTeX
@misc{teng2025hallucinating,
  author = {Fei Teng and Kai Luo and Sheng Wu and Siyu Li and Pujun Guo and Jiale Wei and Kunyu Peng and Jiaming Zhang and Kailun Yang},
  title = {{Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting}},
  year = {2025},
  eprint = {2507.06971},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.06971},
}
Figure