Visual Comfort Assessment for Stereoscopic Image Retargeting
Introduction
As 3D content consumption expands from specialized cinemas to diverse personal devices like smartphones and tablets, the need for effective "retargeting"—the process of resizing content to fit different aspect ratios and resolutions—has become critical. While 2D image retargeting is well-understood, stereoscopic (3D) images present a unique challenge. Beyond maintaining visual sharpness, retargeting must preserve visual comfort. Disruption of the delicate balance between eye focus (accommodation) and convergence (vergence) can lead to significant eye strain and a poor viewing experience.
This paper addresses the lack of standardized benchmarks by introducing the Stereoscopic Image Retargeting Database (SIRD) and a novel objective metric, VCA-SIR, designed to predict the visual comfort of resized 3D images.
The Stereoscopic Image Retargeting Database (SIRD)
To understand how humans perceive retargeted 3D content, the authors conducted a comprehensive subjective study. They compiled 100 source images from established datasets (IEEE-SA and IVY) covering various environments and textures. Each source image was processed using four distinct retargeting strategies:
- Stereo Cropping: Removing peripheral pixels to change the aspect ratio.
- Stereo Scaling: Uniformly stretching or squashing the image.
- Stereo Seam Carving: An intelligent content-aware method that removes low-importance pixel paths.
- Stereo Multi-operator: A hybrid approach combining the strengths of the above methods.
A group of 28 viewers evaluated the resulting 400 images, providing scores for visual comfort, image quality, and depth perception. This database serves as a vital benchmark for testing objective quality metrics.
The VCA-SIR Metric: Objective Comfort Prediction
The proposed VCA-SIR (Visual Comfort Assessment for Stereoscopic Image Retargeting) model aims to automate the evaluation process. It extracts four key feature sets that correlate with human visual fatigue:
- Disparity Range (DR): Ensures the 3D depth remains within the human "comfort zone" to prevent excessive vergence effort.
- Boundary Disparity (BD): Detects "window violations" where foreground objects are cut off by the screen edge, a common cause of binocular rivalry and brain-processing conflict.
- Disparity Intensity Distribution (DID): Utilizes the Just Noticeable Depth Difference (JNDD) to identify "depth hopping"—jarring, non-smooth transitions in depth that often occur during seam carving.
- 2D Image Quality: Integrates standard quality metrics to account for blur or geometric distortions introduced by the resizing process.
These features are fused using Support Vector Regression (SVR) to output a final predicted comfort score.
Key Insights into 3D Perception
The research highlights several critical factors in 3D comfort:
- The Primacy of the A/V Conflict: The mismatch between where the eyes focus and where they converge remains the leading cause of discomfort.
- Edge Sensitivity: In 3D, edges are more than just boundaries; they are depth cues. If an object appears in front of the screen but is clipped by the frame, it creates a sensory contradiction that is physically tiring for the viewer.
- Depth Smoothness: The model emphasizes that the amplitude of depth changes is more important than the frequency. Smooth depth maps are significantly more comfortable than those with "jagged" depth transitions.
Results and Performance
The VCA-SIR metric demonstrated high accuracy in predicting human subjective ratings. In full-reference tests (comparing the retargeted image to the original), it achieved a Pearson Linear Correlation Coefficient (PLCC) of 0.9024. Even in no-reference (blind) scenarios, the model maintained a strong correlation of 0.8670. These results confirm that a holistic approach—combining 3D disparity analysis with 2D image quality—is essential for accurate comfort assessment.
Conclusion and Future Directions
While VCA-SIR represents a significant step forward, the authors note that the current study is limited to static images. Future work will involve extending these metrics to 3D video, where temporal consistency and motion-induced sickness become major factors. Additionally, as AI-driven generative retargeting methods emerge, the SIRD database will need to expand to include these new types of distortions.
Summary updated