Target Scanpath-Guided 360-Degree Image Enhancement

Yujia Wang, Fang-Lue Zhang, Neil A. DodgsonAAAI 2025 · DOI

Abstract

In immersive 360-degree environments, viewers often experience information overload, leading them to miss critical content or lose focus. This paper presents PSEM (Progressive Scanpath-Guided Enhancement Method), a novel framework that edits 360° images to guide a viewer’s gaze along a specific target trajectory. By identifying and removing distracting objects that deviate from a desired "target scanpath," the system reshapes how users navigate immersive scenes. Our approach integrates spatiotemporal clustering, a dual-stream difference encoder, and fine-tuned diffusion models to achieve effective gaze guidance while maintaining visual realism.

Introduction

Traditional image enhancement for 360° content often relies on static saliency maps—heatmaps indicating where people are likely to look. However, these maps ignore the temporal dimension: the specific sequence and timing of gaze points, known as a scanpath. Guiding attention in a spherical environment is challenging due to individual variability in viewing behavior and the geometric distortions inherent in equirectangular projections.

PSEM addresses these challenges by treating attention guidance as a spatiotemporal alignment problem. Instead of merely highlighting important areas, the framework focuses on the strategic removal of visual "magnets" that pull the eye away from the intended path, thereby simplifying the scene and naturally directing the viewer's focus.

Methodology

The PSEM framework operates through a three-stage pipeline designed to bridge the gap between current viewing patterns and target trajectories.

The three-stage PSEM architecture including source scanpath generation, object selection, and image enhancement.

Stage 1: Source Scanpath Generation (TASSC)

To establish a baseline of how viewers currently interact with a scene, the system generates multiple potential scanpaths using an external prediction model. These paths are synthesized into a single representative "average" path using a new algorithm: TASSC (Time-Alignment and Spatial Similarity Clustering). TASSC is specifically optimized for 360° data, utilizing Great Circle Distance to account for spherical distortion and FastDTW (Dynamic Time Warping) to align sequences chronologically.

Stage 2: Object Selection (DSDE)

The representative source scanpath is compared against the user-provided target scanpath. A Dual-Stream Scanpath Difference Encoder (DSDE) identifies spatiotemporal discrepancies between the two. These differences guide the Segment Anything Model (SAM) to automatically generate masks for objects identified as distractions relative to the target path.

Stage 3: Image Enhancement (LoRA-SD)

The final stage utilizes Stable Diffusion fine-tuned with LoRA (Low-Rank Adaptation) to realistically remove the identified objects. The system "inpaints" the background to ensure the final 360° image remains visually coherent. This "subtraction as enhancement" strategy effectively removes the visual competition that would otherwise disrupt the desired gaze flow.

Experimental Results

The authors evaluated PSEM using a custom dataset of 1,200 360° images. Performance was measured using standard scanpath similarity metrics: Levenshtein Distance (LEV), Dynamic Time Warping (DTW), and Recurrence Quantification Analysis (REC).

Quantitative evaluation showing significant improvements in scanpath similarity metrics across source and enhanced images.

Quantitative analysis demonstrates that enhanced images successfully "pull" predicted gaze patterns closer to the target paths. As shown in the results table, the Levenshtein Distance (LEV) decreased by 10.98% on average, while the Recurrence (REC) metric—indicating the overlap between paths—improved by 17.84%. When comparing representative clustered paths, the LEV improvement reached 18.68%, confirming that the model effectively alters the predicted gaze sequence toward the user's goal.

Discussion and Conclusion

PSEM represents a significant shift in immersive content editing by prioritizing the temporal flow of attention. By integrating spherical awareness and diffusion-based inpainting, the framework provides a robust tool for creators to manage viewer focus in complex 360° scenes.

Future work will look to expand the editing vocabulary beyond object removal—incorporating changes in color, contrast, or the addition of new elements—to provide even more granular control over the viewer's journey. Additionally, as more real-world human eye-tracking data in VR becomes available, the underlying prediction models can be further refined for higher accuracy.

Summary updated

BibTeX
@inproceedings{wang2025target,
  author = {Yujia Wang and Fang-Lue Zhang and Neil A. Dodgson},
  title = {{Target Scanpath-Guided 360-Degree Image Enhancement}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i8.32881},
}
Figure