Abstract
Projector-camera systems (ProCams) offer a versatile way to transform ordinary surfaces into interactive displays. However, traditional methods struggle with visual interference from the projected image and the difficulty of distinguishing touch from hover using a single camera. This paper presents a minimalist solution using Slope Disparity Gating (SDG). By synchronizing a laser projector's scanning pattern with a camera's rolling shutter, the system creates a geometric "gate" that isolates a thin volume just above the surface. This approach effectively filters out the projected content and environmental noise, enabling robust, low-latency touch sensing with a single visible-light camera.
Introduction
The promise of ubiquitous computing often involves turning everyday objects—desks, walls, or tables—into interactive interfaces. Projector-camera systems are a natural fit for this goal, but they face significant technical hurdles. First, the camera must distinguish between the user's physical hand and the image of a hand it might be projecting. Second, accurately detecting the moment of contact (touch) versus a finger hovering just above the surface is challenging without expensive depth sensors like the Kinect or specialized infrared arrays.
This research introduces a method to achieve high-accuracy touch sensing using only a standard laser projector and a rolling-shutter camera. The key is a technique called Slope Disparity Gating (SDG), which uses hardware-level synchronization to optically filter the scene.
Methodology: Slope Disparity Gating
The core of the system is the precise synchronization between the projector and the camera. Most laser projectors scan the image line-by-line, and rolling-shutter cameras capture the scene in a similar sequential fashion. By introducing a specific time delay between these two processes, the researchers create a 3D "gate" or "slice" in space.
Only light reflected from objects within this specific volume—a thin layer a few centimeters thick hovering just above the table—is captured by the camera. This "hardware-as-a-filter" approach means the camera never even "sees" the projected image on the table surface or the background of the room. It only sees the user's fingertip as it enters the gated zone.
Touch Detection and Mapping
Because the hardware performs the heavy lifting of background subtraction, the software processing is remarkably simple. The system identifies non-zero pixels (the finger), calculates their center of gravity, and uses a homography matrix to map these camera coordinates directly to the projected UI coordinates.
Evaluation and Results
The system was evaluated for accuracy and robustness through a series of user trials and stress tests.
Accuracy
In tests with five participants performing 110 touch trials, the system achieved an average horizontal error of 16.8 pixels and a vertical error of 4.2 pixels on a 720p display. This precision is well within the footprint of a human finger, ensuring a natural and responsive user experience.
Robustness to Content and Environment
A significant advantage of SDG is its immunity to visual interference. In a "fake hand" test, the system was challenged with projected images of hands. While traditional computer vision libraries like HandTrack were fooled by the projected content, the SDG system ignored it entirely, focusing only on the physical finger.
Furthermore, the system proved robust across extreme lighting conditions, maintaining consistent accuracy from bright office environments (710 lux) to near-total darkness (5 lux).
Discussion and Future Work
By leveraging the inherent properties of rolling-shutter sensors and scanning projectors, this method provides a low-cost, privacy-preserving solution for interactive surfaces. Because the camera only captures a thin slice of space near the surface, it naturally avoids capturing sensitive information like the user's face.
Current limitations include support for only single-touch interactions and sensitivity to the physical alignment of the projector and camera. Future work will focus on extending the logic to multi-touch support and implementing temporal filters to reduce sensing jitter.
Summary updated