DottyToto: a measurement engine for aligning multiprojector display systems

Mark Hereld, Ivan R. Judson, Rick StevensProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2003 · DOI

TL;DR

This paper introduces DottyToto, an efficient and robust system for precisely measuring and aligning multi-projector display systems. It overcomes challenges like optical distortion and projector misalignment, enabling the creation of truly seamless, high-resolution displays for applications like scientific visualization.

What Problem Does This Paper Address?

Creating large, high-resolution displays often involves tiling the output of multiple projectors. The primary challenge is achieving a seamless visual experience, where the individual projected images blend together without visible seams or geometric distortions. This is complicated by several factors:

  • Optical Distortion: Projector lenses are not perfect and can introduce distortions (like pincushion or barrel distortion) that warp the projected image.
  • Misalignment: Projectors need to be precisely positioned and oriented relative to each other and the display surface. Even small misalignments in six degrees of freedom (position and orientation) can lead to noticeable gaps or overlaps.
  • Scalability: As the number of projectors increases, the complexity of aligning them grows significantly.

The goal is to map pixels from each projector's frame buffer to a unified coordinate system for the entire display, making it appear as a single, continuous surface.

Methods

DottyToto employs a novel measurement process that is designed to be efficient, inexpensive, and tolerant of various imperfections. Key aspects include:

  • Dense Dot Sampling: Instead of relying on complex patterns, DottyToto uses a large number of high-contrast "dots" projected onto the display. This dense sampling provides robust data for mapping.
  • Single Uncalibrated Camera: The system uses a single, inexpensive, uncalibrated digital camera. This simplifies setup and eliminates the need for precise camera calibration or complex pan-tilt mechanisms. The camera's pose is determined implicitly.
  • Few Exposures: The method aims to capture all necessary measurement data in a minimal number of images, ideally one or two for non-overlapping displays, and up to five for overlapping ones. This reduces susceptibility to environmental changes like lighting or camera drift.
  • Direct Solution: The analysis is direct, avoiding iterative processes. It uses a least-squares minimization to fit a parameterized model to the measured dot positions.
  • Parameterized Models:
    • Tile-to-Camera Model: A third-order polynomial model (10 parameters for X and Y) is used to map projector tile coordinates to camera pixel coordinates. This model accounts for both projector and camera optical distortions.
  • Camera-to-Mural Model: Another third-order polynomial model is derived to map camera coordinates to a unified mural coordinate system. This step estimates relative projector offsets and creates an approximate camera-to-mural mapping.
  • Aligned Array Assumption: The method assumes the projectors are already roughly aligned, which helps preserve the native resolution of individual projectors and simplifies the initial layout discovery.

Datasets

The paper does not describe the use of pre-existing datasets for training or evaluation. Instead, it introduces its own methodology for generating measurement data using custom test patterns projected onto a physical multi-projector display system called "ActiveMural."

  • ActiveMural: A 15-projector display system used for experiments.
  • Test Patterns: Two types of test patterns are used:
  • A "tile discovery" pattern with a large, bright rectangular region for easy identification of individual tiles.
  • A "dot pattern" with a dense grid of high-contrast dots, used for detailed mapping measurements.

Key Insights and Ideas

  • Subpixel Accuracy with Simple Tools: The core insight is that by using a dense sampling of simple features (dots) and a single uncalibrated camera, one can achieve subpixel accuracy in measuring the complex mappings required for seamless multi-projector displays.
  • Robustness to Imperfections: The method is designed to be tolerant of common issues like lighting variations, camera optics, and slight camera pose changes, making it practical for real-world deployments.
  • Efficiency through Direct Solution: Avoiding iterative fitting processes significantly speeds up the measurement and calibration pipeline.
  • Unified Mapping: The system generates a comprehensive set of transformations that can be used for various correction strategies, from simple offsets to complex image warping.

Key Results

  • Accuracy: The method demonstrated the ability to detect subpixel displacements. In an accuracy test, the measured shifts were within +/- 0.21 pixels, well below a single pixel, indicating high precision.
  • Scalability: The system was successfully applied to a 15-projector ActiveMural system (4800 x 2100 pixels) using a 1600 x 1200 pixel camera.
  • Speed: Processing five images for the ActiveMural on a 1.2 GHz Pentium III laptop took under 2.5 minutes.
  • Correction: The paper shows examples of corrected images, demonstrating the elimination of visible distortions and seams in areas of projector overlap.

Limitations and Future Work

  • Speed Optimization: While fast, the authors suggest further optimization of image acquisition and analysis for real-time applications.
  • Integration: They aim to integrate real-time image warp corrections into a wider range of applications and development tools for seamless transparency.
  • Initial Alignment Tool: The development of a tool to assist with the initial alignment of tiled displays (automated or manual) is identified as a future direction.
  • Model Complexity: The current third-order polynomial model is sufficient but could potentially be extended to handle more complex distortions if needed.
  • Assumed Alignment: The method assumes a roughly aligned array. While it can correct for imperfections, a very poorly aligned initial state might still pose challenges.

Summary updated

Abstract

Tiled displays systems built by combining the images from arrays of projectors can provide huge numbers of pixel elements to applications needing to visually represent lots of information. Such applications are already coming into wide usage and include large scientific visualizations, collaborative virtual environments, and rich multimedia spaces. It is, however, difficult to create the illusion of a unified seamless display for a variety of reasons including optical distortion of the individual projector images due to imperfections in the lenses and basic alignment of the projectors. In this paper we describe an efficient and optimized measurement process using inexpensive components that is tolerant of a wide range of imperfections in components and measurement setup (lighting conditions, camera optics, etc.). Our method nonetheless is capable of accurate and detailed measurement of the layout of all projector images, including the generation of a detailed model of the distortions in each projector optical system. It performs these measurements on the entire array of projectors at once. Once the detailed mapping between projector pixels and mural pixels is measured, the resulting relations can be used in any of a number of ways to improve the appearance of images projected on the display.
BibTeX
@inproceedings{hereld2003dottytoto,
  author = {Mark Hereld and Ivan R. Judson and Rick Stevens},
  title = {{DottyToto: a measurement engine for aligning multiprojector display systems}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2003},
  doi = {10.1117/12.473845},
}
Figure