Vision-based Augmented Reality Applications

Yuko Uematsu, Hideo SaitôInTech 2008 · DOI · OpenAlex

Abstract

This paper introduces a vision-based registration method using multiple planar markers placed at arbitrary positions and poses. Unlike previous multi-marker methods, this approach requires no time-consuming manual measurement of marker arrangements. By estimating marker positions within a reconstructed Projective Space, markers can be distributed freely on tabletops or throughout a room. We demonstrate the system's performance through two interactive AR applications—baseball and bowling—which run on standard consumer hardware. The method enables intuitive viewpoint changes through physical movement, offering a future-oriented 3D gaming experience without specialized equipment.

Introduction

Standard marker-based Augmented Reality (AR) systems often rely on a single planar marker, which limits the user's range of motion and leads to tracking instability at sharp viewing angles or during partial occlusion. While multi-marker setups can mitigate these issues, they traditionally require tedious manual measurement of the 3D coordinates and orientations of every marker relative to one another.

This research seeks to make multi-marker setups "plug-and-play" by automating the registration of markers using only camera data. By utilizing a "Projective Space" reconstructed from two reference images, the system automatically estimates the arrangement of markers placed at arbitrary angles and locations, enabling robust, real-time tracking for interactive tabletop games.

Methodology

The core innovation is the use of a 3D Projective Space as an intermediary coordinate system. The system operates in two distinct phases: an initialization phase and a real-time tracking phase.

System overview showing the two-phase processing: initial projective space construction and real-time marker integration.

Projective Reconstruction and Marker Mapping

In the first phase, the system captures a short sequence of the scene and automatically selects two "reference images" from different viewpoints. Using the fundamental matrix between these images, it constructs a non-Euclidean 3D Projective Space. For each marker $i$, the system computes a transformation matrix ($T_i^{WP}$) that relates the marker’s local coordinate system to the Projective Space. This is done by detecting the marker's corners and projecting them into the 3D space.

The relationship between marker coordinate systems, the intermediate Projective Space, and the final image plane.

Redundant Tracking and Integration

During live operation (the second phase), the system calculates a projection matrix for every visible marker. These are merged into a single, robust projection matrix ($P^{PI}$) using a least-squares method. One marker is designated as the "base," and virtual objects are defined relative to it. However, they can be rendered accurately as long as any of the registered markers are visible to the camera. This redundancy allows the system to overcome the "low-angle" problem; if a marker on the table becomes unreadable because the camera is too low, a marker tilted at a different angle can take over tracking duties seamlessly.

Experimental Results

The authors evaluated their system using custom-built experimental setups, including AR Baseball and AR Bowling.

  • Performance: Both applications achieved a frame rate of approximately 30 FPS on a consumer-grade PC (Pentium IV 3.2-3.6 GHz), which is sufficient for smooth, interactive gameplay.
  • User Efficiency: In a small study, users were asked to change their viewpoints (e.g., moving from a "catcher's view" to a "pitcher's view"). Using the AR system (physically moving the camera), users took an average of 7 seconds, compared to 43 seconds using a traditional keyboard/mouse interface.
  • Stability: The system successfully maintained tracking even when markers were occluded by the rolling bowling ball or viewed at extreme angles that would typically break single-marker AR systems.
  • Tangible Interaction: In the bowling application, the authors map the 2D position of a real-world red ball onto a "top-down" virtual view using homography. This allows a physical object to interact with virtual pins, creating a natural "tangible interface."

Conclusion

The proposed vision-only registration method significantly simplifies the deployment of multi-marker AR environments. By eliminating the need for manual measurements and specialized sensors, it makes robust AR more accessible for home users. While initialization currently takes about one minute, the system's ability to handle arbitrary marker placement and provide intuitive interaction marks a significant advancement for future-oriented 3D gaming.

Summary updated

Abstract

In this chapter, we introduced a vision-based registration method using multiple planar markers placed at arbitrary positions and poses is introduced. We also demonstrate the performance of the proposed method by applying it to two AR applications. In contrast with the previous methods using multiple markers, this method requires no time-consuming measurement of the marker arrangement. Since the marker arrangement can be estimated by the Projective Space, the markers can be distributed at arbitrary positions and poses without a prior knowledge. This method can be applied not only for on the tabletop but also inside the room. Both of the baseball system and bowling system can be enjoyed on the tabletop in the real world only with a web-camera and a handheld monitor connected a general PC. It is a big advantage for home users that these applications do not require any special device. Users can interactively change their view points by moving around the tabletop because of this registration method with multiple markers. In contrast with usual CG viewers in which a mouse or a keyboard is used for changing view points, changing view points by moving of the users is very intuitive and easy. By visualising 3D objects in front of the users, these applications will be future-oriented 3D game.
BibTeX
@inproceedings{uematsu2008vision,
  author = {Yuko Uematsu and Hideo Saitô},
  title = {{Vision-based Augmented Reality Applications}},
  booktitle = {InTech},
  year = {2008},
  doi = {10.5772/6157},
}
Figure