Abstract
An augmented book is an application that augments virtual 3D objects to a real book via AR technology. For augmented books, some markerless methods have been proposed so far. However, they can only recognize one page at a time. This leads to restrictions on the utilization of augmented books. In this paper, we present a novel markerless tracking method capable of recognizing and tracking multiple pages in real-time. The proposed method builds on our previous work using the generic randomized forest (GRF). The previous work finds out one page in the entire image using the GRF, whereas the proposed method detects multiple pages by dividing an image into subregions, applying the GRF to each subregion and discovering spatial locality from the GRF results.
Introduction
Augmented Reality (AR) has transformed physical books into interactive experiences by overlaying 3D virtual objects onto real-world pages. Traditionally, these systems relied on fiducial markers—visually intrusive codes that disrupt the aesthetic and layout of a book. While markerless tracking has emerged as a cleaner alternative, existing solutions are largely restricted to identifying and tracking a single page at a time. This limitation is problematic in real-world scenarios where a reader might have a book spread open to two pages or interact with multiple supplemental materials simultaneously.
This paper introduces a real-time, markerless system designed to recognize and track multiple pages concurrently. By leveraging a Generic Randomized Forest (GRF) and a novel spatial locality filtering mechanism, the system achieves robust performance across large page databases.
Methodology
The system architecture is divided into three primary phases: pre-processing, multi-page recognition, and adaptive tracking.
Pre-processing and Feature Extraction
The initial stage involves extracting keypoints from the camera feed. To ensure scale invariance, the system utilizes the FAST detector across three levels of an image pyramid. This allows the system to maintain tracking even as the user moves the camera closer to or further from the book.
Multi-Page Recognition via Generic Randomized Forest
The core of the recognition engine is the Generic Randomized Forest (GRF). The GRF is a machine learning structure trained to classify keypoints into specific page IDs and local coordinates. Unlike standard approaches that analyze the entire frame as a single entity, our method divides the image into an $m \times n$ grid of subregions. Each subregion is processed independently by the GRF to identify the most likely page candidates.
Spatial Locality Filtering
To confirm the presence of a page and filter out noise, the system employs a "spatial locality" check. If multiple adjacent subregions return high probability scores for the same page ID, the system concludes that the page is indeed present in that portion of the frame. This clustering approach naturally segments different pages without requiring complex edge detection or prior scene segmentation.
Adaptive Keyframe-Based Tracking
Once a page is identified, the system must estimate its 3D pose. Standard markerless systems often track against a static "training" image. However, environmental factors like sensor noise and varying illumination often make the training image a poor reference for live tracking, leading to "jitter."
To combat this, we implemented an adaptive keyframe approach. The system replaces the static reference with the "best" live frame captured during the session. Frames are evaluated based on a scoring function that considers clarity (lack of motion blur), visibility (how much of the page is in view), and orthogonality (how "top-down" the view is).
Experimental Results
The system was evaluated using a 166-page elementary science textbook. Training involved extracting 250 keypoints from a single high-quality image of each page.
Performance and Stability
The proposed method successfully tracked up to four pages simultaneously at a consistent 30 frames per second on a standard desktop environment. A critical achievement was the reduction of tracking jitter. By moving from a fixed training reference to adaptive keyframes, average jitter was reduced from 1.002 pixels to 0.026 pixels—a nearly 40-fold improvement in stability.
Robustness
Testing demonstrated that the system remains robust under challenging conditions, including significant viewpoint changes, partial occlusions (such as a reader's hand), and varying lighting conditions. The spatial locality filter proved particularly effective at maintaining tracking even when pages were partially obscured.
Conclusion
We have presented a novel method for multi-page recognition and tracking in augmented books. By combining the classification power of Generic Randomized Forests with a subregion-based spatial locality filter and adaptive keyframe tracking, we have overcome the single-page limitation of previous markerless systems. Future work will focus on optimizing the integration between recognition and tracking to support even larger numbers of simultaneous pages without compromising real-time performance.
Summary updated