Visual Comfort Assessment for Stereoscopic Image Retargeting

Ya Zhou, Wei Zhou, Ping An, Zhibo Chen

Visual Comfort Assessment for Stereoscopic Image Retargeting

Introduction

As 3D content consumption expands from specialized cinemas to diverse personal devices like smartphones and tablets, the need for effective "retargeting"—the process of resizing content to fit different aspect ratios and resolutions—has become critical. While 2D image retargeting is well-understood, stereoscopic (3D) images present a unique challenge. Beyond maintaining visual sharpness, retargeting must preserve visual comfort. Disruption of the delicate balance between eye focus (accommodation) and convergence (vergence) can lead to significant eye strain and a poor viewing experience.

This paper addresses the lack of standardized benchmarks by introducing the Stereoscopic Image Retargeting Database (SIRD) and a novel objective metric, VCA-SIR, designed to predict the visual comfort of resized 3D images.

The Stereoscopic Image Retargeting Database (SIRD)

To understand how humans perceive retargeted 3D content, the authors conducted a comprehensive subjective study. They compiled 100 source images from established datasets (IEEE-SA and IVY) covering various environments and textures. Each source image was processed using four distinct retargeting strategies:

  • Stereo Cropping: Removing peripheral pixels to change the aspect ratio.
  • Stereo Scaling: Uniformly stretching or squashing the image.
  • Stereo Seam Carving: An intelligent content-aware method that removes low-importance pixel paths.
  • Stereo Multi-operator: A hybrid approach combining the strengths of the above methods.
Original source image (a) and its retargeted versions (b-e) using different operators.

A group of 28 viewers evaluated the resulting 400 images, providing scores for visual comfort, image quality, and depth perception. This database serves as a vital benchmark for testing objective quality metrics.

The VCA-SIR Metric: Objective Comfort Prediction

The proposed VCA-SIR (Visual Comfort Assessment for Stereoscopic Image Retargeting) model aims to automate the evaluation process. It extracts four key feature sets that correlate with human visual fatigue:

  1. Disparity Range (DR): Ensures the 3D depth remains within the human "comfort zone" to prevent excessive vergence effort.
  2. Boundary Disparity (BD): Detects "window violations" where foreground objects are cut off by the screen edge, a common cause of binocular rivalry and brain-processing conflict.
  3. Disparity Intensity Distribution (DID): Utilizes the Just Noticeable Depth Difference (JNDD) to identify "depth hopping"—jarring, non-smooth transitions in depth that often occur during seam carving.
  4. 2D Image Quality: Integrates standard quality metrics to account for blur or geometric distortions introduced by the resizing process.

These features are fused using Support Vector Regression (SVR) to output a final predicted comfort score.

Key Insights into 3D Perception

The research highlights several critical factors in 3D comfort:

  • The Primacy of the A/V Conflict: The mismatch between where the eyes focus and where they converge remains the leading cause of discomfort.
  • Edge Sensitivity: In 3D, edges are more than just boundaries; they are depth cues. If an object appears in front of the screen but is clipped by the frame, it creates a sensory contradiction that is physically tiring for the viewer.
  • Depth Smoothness: The model emphasizes that the amplitude of depth changes is more important than the frequency. Smooth depth maps are significantly more comfortable than those with "jagged" depth transitions.

Results and Performance

The VCA-SIR metric demonstrated high accuracy in predicting human subjective ratings. In full-reference tests (comparing the retargeted image to the original), it achieved a Pearson Linear Correlation Coefficient (PLCC) of 0.9024. Even in no-reference (blind) scenarios, the model maintained a strong correlation of 0.8670. These results confirm that a holistic approach—combining 3D disparity analysis with 2D image quality—is essential for accurate comfort assessment.

Conclusion and Future Directions

While VCA-SIR represents a significant step forward, the authors note that the current study is limited to static images. Future work will involve extending these metrics to 3D video, where temporal consistency and motion-induced sickness become major factors. Additionally, as AI-driven generative retargeting methods emerge, the SIRD database will need to expand to include these new types of distortions.

Summary updated

Abstract

In recent years, visual comfort assessment (VCA) for 3D/stereoscopic content has aroused extensive attention. However, much less work has been done on the perceptual evaluation of stereoscopic image retargeting. In this paper, we first build a Stereoscopic Image Retargeting Database (SIRD), which contains source images and retargeted images produced by four typical stereoscopic retargeting methods. Then, the subjective experiment is conducted to assess four aspects of visual distortion, i.e. visual comfort, image quality, depth quality and the overall quality. Furthermore, we propose a Visual Comfort Assessment metric for Stereoscopic Image Retargeting (VCA-SIR). Based on the characteristics of stereoscopic retargeted images, the proposed model introduces novel features like disparity range, boundary disparity as well as disparity intensity distribution into the assessment model. Experimental results demonstrate that VCA-SIR can achieve high consistency with subjective perception.
BibTeX
@inproceedings{zhou2018visual,
  author = {Ya Zhou and Wei Zhou and Ping An and Zhibo Chen},
  title = {{Visual Comfort Assessment for Stereoscopic Image Retargeting}},
  booktitle = {ISCAS},
  year = {2018},
  url = {https://openreview.net/forum?id=m3s5hJKfKXa},
}
Figure