Quadric Transfer for Immersive Curved Screen Displays

Ramesh Raskar, Jeroen van Baar, Thomas Willwacher, Srinivas RaoComputer Graphics Forum 2004 · DOI

TL;DR

This paper introduces a novel "quadric transfer" technique for seamlessly projecting images onto curved displays using multiple projectors. It overcomes the limitations of existing methods by providing an automated, robust calibration and rendering pipeline that works even with casually aligned projectors, enabling high-quality immersive experiences.

What Problem Does This Paper Address?

Creating seamless, high-resolution images on large, curved displays using multiple projectors is a significant challenge. Current methods for planar displays often rely on complex camera-in-the-loop calibration or manual alignment, which is time-consuming and impractical for curved surfaces. Existing techniques for curved screens are cumbersome, requiring meticulous manual adjustments to achieve the desired alignment. This paper addresses the need for an automated, robust, and efficient method to achieve sub-pixel registration and seamless image blending on curved quadric surfaces (like spheres and cylinders) without requiring precise projector placement.

Methods

The core of the paper is the quadric transfer function, a parametric approach that extends the concept of homography (used for planar displays) to curved surfaces.

  • Quadric Transfer: Instead of a simple 3x3 homography matrix, this method uses a more complex transfer function that models the geometry of quadric surfaces. This allows for accurate mapping of pixels between different views (e.g., from a camera's view to a projector's view).
  • Calibration: The process involves:
  • Stereo Camera Pair: A calibrated stereo camera pair is used to capture 3D points on the display surface by projecting structured light patterns.
  • Quadric Fitting: A 3D quadric surface (represented by a 4x4 symmetric matrix) is fitted to these 3D points.
  • Projector Pose Estimation: The pose (position and orientation) of each projector relative to the camera is estimated by finding correspondences between projected pixels and the reconstructed 3D points. This step is crucial and is handled using a pseudo-Euclidean approach to improve robustness, especially with near-planar point distributions.
  • Quadric Transfer Parameter Estimation: The parameters of the quadric transfer function between the camera and each projector are then computed. This is refined using non-linear optimization to minimize pixel re-projection errors.
  • Rendering: A single-pass rendering approach is employed. For 3D scenes, the virtual view is rendered, and then the quadric transfer function is applied to warp the image directly into the projector's framebuffer. This avoids costly post-rendering warping and potential aliasing artifacts.
  • Intensity Blending: A parametric approach is used to calculate alpha maps for smooth intensity blending in overlapping regions, ensuring perceptually seamless transitions between projector images.

Datasets

Not discussed in the paper. The authors use their own experimental setup with Mitsubishi X80 LCD projectors and Logitech USB webcams for calibration.

Key Insights and Ideas

  • Parametric Modeling for Curves: The key insight is that a parametric model (quadric transfer) can elegantly handle the complex geometry of curved surfaces, offering significant advantages over non-parametric, look-up table-based methods.
  • Robustness to Casual Alignment: The method is designed to work with "casually aligned" projectors, meaning precise manual alignment is not required, significantly reducing setup time and cost.
  • Pseudo-Euclidean Approach: The paper highlights the importance of a pseudo-Euclidean approach for estimating projector parameters, especially when dealing with near-planar 3D points, which are common on curved surfaces. This improves the robustness of the calibration.
  • Single-Pass Rendering for Curved Displays: Implementing a single-pass rendering pipeline for curved displays is a significant contribution, enabling efficient rendering without sacrificing image quality.
  • Head-Tracking Integration: The system seamlessly integrates head-tracking, allowing for an immersive experience where the virtual viewpoint updates dynamically as the user moves.

Key Results

  • Sub-pixel Registration: The method achieves sub-pixel registration accuracy after non-linear refinement, which is crucial for perceptually seamless displays.
  • Reduced Calibration Time: While the non-linear optimization step takes time (40-55 seconds per projector in their unoptimized code), it's significantly faster and more automated than manual alignment.
  • Improved Accuracy: The RMS re-projection errors are reduced to about one pixel after non-linear refinement, which is considered acceptable. This contrasts with linear estimation, which can result in errors of 15-30 pixels on screen.
  • Generality: The technique is demonstrated to work for both concave and convex quadric surfaces, as well as planar surfaces (a degenerate quadric).

Limitations and Future Work

  • Calibration Time: The current non-optimized calibration code is relatively slow (45-60 seconds per projector).
  • Non-linear Optimization: The non-linear optimization step, while effective, can be sensitive to initial guesses and requires careful parameter tuning.
  • Feature Detection on Complex Surfaces: Detecting features for calibration can be more difficult on surfaces with less distinct patterns or significant curvature.
  • Future Work: The authors plan to extend their technique to general curved surfaces (beyond quadrics) and explore scenarios where quadric transfers cannot be easily cascaded, such as wrap-around rear or front projection displays. They also aim to improve the ease of use and further reduce calibration time.

Summary updated

BibTeX
@article{raskar2004quadric,
  author = {Ramesh Raskar and Jeroen van Baar and Thomas Willwacher and Srinivas Rao},
  title = {{Quadric Transfer for Immersive Curved Screen Displays}},
  journal = {Computer Graphics Forum},
  year = {2004},
  doi = {10.1111/j.1467-8659.2004.00776.x},
}
Figure