SpeckleSense: fast, precise, low-cost and compact motion sensing using laser speckle

Jan Zizka, Alex Olwal, Ramesh RaskarUIST 2011 · DOI

TL;DR

This paper introduces SpeckleSense, a novel motion sensing technique using laser speckle that offers fast, precise, and compact interaction for human-computer interfaces. By leveraging the interference patterns of laser light, SpeckleSense overcomes limitations of traditional camera-based methods, enabling low-cost, high-performance motion tracking for applications ranging from input devices to mobile interactions.

What Problem Does This Paper Address?

Traditional optical motion sensing for user interfaces, especially those relying on cameras, often face limitations in resolution, frame rates, and computational demands. While inertial sensors are common in consumer electronics, they can be bulky or require calibration. The paper aims to bridge this gap by proposing a motion sensing approach that is fast, precise, extremely compact, low-cost, and computationally efficient, making it suitable for a wide range of HCI applications, including embedded systems and mobile devices.

Methods

The core of SpeckleSense is the utilization of laser speckle. When coherent light (like from a laser) illuminates a rough surface, it creates a complex interference pattern of bright and dark spots known as speckle. This pattern is highly sensitive to the relative motion between the light source, the surface, and the sensor.

The paper details several key methodological aspects:

  • Speckle Generation: Speckle patterns can be generated either by illuminating a surface with a laser (creating a "virtual speckle projector") or by using a dedicated "Speckle Projector" which combines a laser diode with a diffuser.
  • Speckle Tracking: Image processing algorithms are used to track the movement of these speckle patterns. The key insight is that a small translation of the sensor or projector results in a predictable shift of the speckle image.
  • High-Speed Sensing: The system leverages high frame-rate sensors (up to 10,000 Hz for specialized sensors, or repurposed optical mouse sensors capable of 1,000 Hz output) to capture rapid changes in the speckle pattern, enabling fast motion tracking.
  • Distance Approximation: The size of speckles increases as the distance between the projector and sensor decreases. By analyzing the average speckle size (related to average intensity), the system can approximate distance.
  • Configurations: The paper outlines four general SpeckleSense configurations:
  • Handheld Speckle Projector (A): A device with a laser projects a pattern, tracked by a fixed optical sensor for relative 2D input.
  • Handheld Speckle Sensor (B): A device with a sensor tracks a fixed projected pattern in the environment for mid-air interaction.
  • Self-tracked Device (C): Both laser and sensor are embedded in the device (like a mouse) to track motion on or above a surface.
  • Speckle Tracking of Hands (D): The body acts as a speckle-generating surface, illuminated by a laser and tracked by a sensor for gestural interaction.
  • Hybrid Tracking: The paper also explores combining SpeckleSense with other sensors (like accelerometers) or using multiple speckle components for more complex 3D tracking and multi-modal interactions.

Datasets

Not discussed in the paper. The paper focuses on demonstrating the technique with custom prototypes and does not rely on pre-existing datasets for training or evaluation.

Key Insights and Ideas

  • Leveraging "Noise" as Signal: Laser speckle, often considered a nuisance in optical systems, is ingeniously turned into a high-fidelity motion signal. The inherent randomness and sensitivity of speckle patterns are key to its tracking capabilities.
  • Compactness and Low Cost: The use of small laser diodes and readily available optical mouse sensors (with lenses removed) makes the technology exceptionally compact and cost-effective, paving the way for integration into everyday devices.
  • High Frame Rate Potential: By relying on the speckle pattern itself rather than forming a detailed image, SpeckleSense can achieve much higher effective frame rates than traditional camera-based systems, crucial for precise and responsive interaction.
  • Versatile Interaction Paradigms: The flexibility in configuring projectors and sensors allows for a wide range of interaction styles, from traditional pointing devices to mid-air gestures and even tracking hands as surfaces.
  • Optical Communication: The ability to transmit motion data optically, without radio or wired connections, simplifies hardware design and enables unique interaction scenarios, like controlling multiple devices with a single projector.

Key Results

  • High Sensitivity: Speckle tracking can achieve sensitivities down to 50 µm.
  • High Sampling Rate: The system can achieve very high sampling rates (up to 10,000 Hz for specialized sensors, and 1,000 Hz output from repurposed mouse sensors).
  • Effective Range: Demonstrated effective ranges of up to 3 meters with diffused lasers, and precise tracking up to 20 cm above a reference sheet with a 5 mW laser.
  • Accuracy: In experiments to recover cumulative error, the average deviation was 0.0217% (0.076 mm) over a 35 cm trajectory.
  • Robustness to Ambient Light: The combination of a lensless sensor and coherent light provides robustness against ambient lighting conditions.
  • Prototypes: The paper presents several functional prototypes, including a "Touch Controller" for 3D model manipulation, a "Mobile Viewport" for spatially tracked mobile displays, and a low-cost system for public displays.

Limitations and Future Work

  • Cumulative Error: The paper acknowledges that the high sampling rate can lead to cumulative errors over time, especially in purely optical tracking without a ground truth reference.
  • Actuation in Public Displays: The public display prototype lacked direct actuation, making it difficult to exit tracking mode or reposition the hand.
  • User Feedback: Preliminary user feedback indicated that the public display prototype was the hardest to use, and suggested improvements like multiple sensors for expanded tracking area and depth sensing.
  • Future Work:
    • Refining prototypes and conducting formal user studies to compare SpeckleSense with established technologies.
  • Developing more sophisticated speckle analysis methods.
  • Switching to developer hardware to access raw image data from high-speed sensors (10,000 fps).
  • Exploring depth control and improved visual feedback for interactions.
  • Creating more compact sensor versions.

Summary updated

BibTeX
@inproceedings{zizka2011specklesense,
  author = {Jan Zizka and Alex Olwal and Ramesh Raskar},
  title = {{SpeckleSense: fast, precise, low-cost and compact motion sensing using laser speckle}},
  booktitle = {UIST},
  year = {2011},
  doi = {10.1145/2047196.2047261},
}
Figure