Motion parallax in stereo 3D: model and applications

Petr Kellnhofer, Piotr Didyk, Tobias Ritschel, Belén Masiá, Karol Myszkowski, Hans-Peter Seidel

Abstract

Stereoscopic 3D (S3D) displays are fundamentally limited by a "depth budget" to prevent viewer discomfort and image artifacts. This paper presents a computational model that quantifies the interaction between binocular disparity and motion parallax. By leveraging motion as a supplementary depth cue, the authors developed a remapping algorithm that compresses disparity in moving scene elements and reallocates it to static regions. Experimental results demonstrate that this approach enhances perceived depth and reduces visual artifacts on autostereoscopic screens, with a significant preference from users.

Introduction: The Depth Budget Challenge

A core challenge in Stereoscopic 3D (S3D) is balancing depth perception with viewer comfort. To avoid "vergence-accommodation conflict"—where eyes focus on the screen but converge elsewhere—the range of binocular disparity must be kept narrow. On glasses-free (autostereoscopic) displays, high disparity further degrades quality by introducing ghosting and aliasing.

However, binocular disparity is not the only cue for depth. Motion parallax—the relative movement of objects at different distances—provides a powerful monocular depth signal. This research investigates whether motion parallax can compensate for reduced disparity, allowing content creators to "borrow" depth budget from moving objects and reinvest it into static ones.

Methods: Modeling and Remapping

The researchers developed their approach in three key stages:

  1. Psychophysical Modeling: Through experiments using random-dot stereograms, the team established a mathematical relationship between motion and perceived depth. They found that motion parallax can induce a depth sensation equivalent to approximately 20% of the "veridical" geometric depth.
  2. Rigidity Detection: To ensure the model applies correctly to complex scenes, the system distinguishes between camera-induced parallax and non-rigid motion (e.g., a person moving).
  3. Disparity Remapping: Using the model, a histogram-based algorithm calculates the "parallax contribution" for every pixel. In regions where motion provides a strong depth cue, the algorithm compresses the binocular disparity. This "saved" budget is then redistributed to static regions where depth perception would otherwise be weak.
Geometric relationship between parallax and disparity.

The study utilized custom synthetic stimuli for model fitting and four distinct validation video sequences (City flight, Amusement park, Mountain, and Street drive) to test real-world effectiveness.

Key Insights and Results

The model identified a "sweet spot" for motion: retinal velocity must exceed approximately 2.36 arcmin/s for the brain to utilize the parallax cue. Once this threshold is met, the contribution to perceived depth saturates quickly.

In a validation study, participants preferred the remapped 3D videos over the originals 60% of the time. The manipulated videos were perceived as "more 3D" despite using the same total disparity budget. Furthermore, the method successfully reduced aliasing on autostereoscopic displays by removing unnecessary disparity from moving regions.

The disparity remapping pipeline: disparity is compressed in moving regions (trees) and expanded in static regions (flower).

The implementation achieved real-time performance, processing 720p video at approximately 35ms per frame, making it suitable for gaming and live broadcasts.

Discussion and Future Work

While the current model provides a robust framework for rigid motion, it has limitations. In sequences like City flight, the model's performance was hampered by user attention; viewers focused on the static background rather than the moving foreground. Future iterations could incorporate saliency predictors to prioritize disparity for objects of interest. Additionally, extending the model to account for non-rigid deformations and other monocular cues like defocus blur remains an area for future research.

Summary updated

Abstract

Binocular disparity is the main depth cue that makes stereoscopic images appear 3D. However, in many scenarios, the range of depth that can be reproduced by this cue is greatly limited and typically fixed due to constraints imposed by displays. For example, due to the low angular resolution of current automultiscopic screens, they can only reproduce a shallow depth range. In this work, we study the motion parallax cue, which is a relatively strong depth cue, and can be freely reproduced even on a 2D screen without any limits. We exploit the fact that in many practical scenarios, motion parallax provides sufficiently strong depth information that the presence of binocular depth cues can be reduced through aggressive disparity compression. To assess the strength of the effect we conduct psycho-visual experiments that measure the influence of motion parallax on depth perception and relate it to the depth resulting from binocular disparity. Based on the measurements, we propose a joint disparity-parallax computational model that predicts apparent depth resulting from both cues. We demonstrate how this model can be applied in the context of stereo and multiscopic image processing, and propose new disparity manipulation techniques, which first quantify depth obtained from motion parallax, and then adjust binocular disparity information accordingly. This allows us to manipulate the disparity signal according to the strength of motion parallax to improve the overall depth reproduction. This technique is validated in additional experiments.
BibTeX
@article{kellnhofer2016motion,
  author = {Petr Kellnhofer and Piotr Didyk and Tobias Ritschel and Belén Masiá and Karol Myszkowski and Hans-Peter Seidel},
  title = {{Motion parallax in stereo 3D: model and applications}},
  journal = {ACM Trans. Graphics},
  year = {2016},
  doi = {10.1145/2980179.2980230},
}
Figure