Mathematical Design of a Novel Gesture-Based Instruction/Input Device Using Wave Detection

Can Yang, Hongyu Liu, Yuliang Wang, Can Yang 0002SIAM J. Imaging sci 2016 · DOI

TL;DR

This paper introduces a novel gesture recognition system that uses wave detection instead of traditional cameras. By analyzing how waves scatter off a person's hand movements, the system can identify gestures and translate them into computer commands, offering a new paradigm for human-computer interaction.

What Problem Does This Paper Address?

Traditional gesture recognition systems, often relying on cameras and computer vision, face challenges in accuracy and robustness, especially in varying lighting conditions or complex environments. The authors aim to overcome these limitations by proposing a fundamentally different approach: using wave detection. This method treats gestures as "scatterers" and analyzes the patterns of scattered waves to identify them. This opens up possibilities for more discreet and potentially more reliable gesture interfaces, moving beyond the visual domain.

Methods

The core of the proposed system is an inverse scattering problem formulation. A transmitter emits time-harmonic wave signals, and an array of receivers captures the scattered waves when these signals interact with a person's hand (the "gesture"). The gestures are modeled as shapes of impenetrable or penetrable scatterers belonging to a predefined "dictionary" of possible gestures.

The recognition algorithm operates in two main steps:

  1. Location Determination: Using a low wavenumber (longer wavelength) incident wave, the system first estimates the approximate location of the gesture. This step leverages the fact that longer wavelengths are less sensitive to fine shape details and are better for coarse localization.
  2. Shape Identification: With the location known, a regular wavenumber (shorter wavelength) incident wave is used. The system then identifies the specific shape of the gesture by analyzing the scattered wave data.

A key mathematical technique employed is the use of integral equations and asymptotic expansions to model the wave propagation and scattering phenomena. The authors also explore the use of both full and phaseless measurement data, with the latter being particularly interesting for practical applications where phase information might be difficult to obtain.

Datasets

Not discussed in the paper. The paper focuses on the mathematical design and theoretical validation of the method. Numerical experiments are conducted using simulated data based on a dictionary of six predefined scatterer shapes (gestures), each composed of four unit cubes.

Key Insights and Ideas

  • Wave-based gesture recognition: Shifting from visual (camera) to wave-based detection offers a novel and potentially more robust approach to gesture recognition.
  • Gestures as scatterers: Modeling human gestures as physical objects (scatterers) that interact with waves provides a strong mathematical framework for analysis.
  • Two-step localization and identification: Decoupling the problem into first finding the location and then the shape of the gesture simplifies the overall recognition process.
  • Importance of wavenumber selection: Using different wavenumbers (wavelengths) for localization and identification is crucial for the algorithm's effectiveness.
  • Phaseless data potential: The ability to perform recognition using only amplitude (phaseless) data is a significant practical advantage, as phase information is often harder to acquire accurately.

Key Results

The paper presents a rigorous mathematical framework and theoretical justification for the proposed wave-based gesture recognition system. Numerical experiments demonstrate the effectiveness of the method:

  • The system successfully identifies all six predefined gestures in the dictionary using both full and phaseless measurement data.
  • The recognition process is shown to be robust to a 5% relative random noise in the phaseless measurement data.
  • The entire recognition process (location and shape identification) takes less than one second on a personal computer, excluding the precomputation of far-field patterns.
  • The method shows good performance even when some gestures are modeled as penetrable (medium) scatterers, which is a more realistic scenario.

Limitations and Future Work

  • Dictionary Dependence: The current approach relies on a predefined "dictionary" of gestures. The system can only recognize gestures that are present in this dictionary.
  • Computational Precomputation: While the recognition itself is fast, the method requires precomputing and storing the scattering information for all possible gestures in the dictionary.
  • Practical Implementation Challenges: The paper focuses on the mathematical design. Real-world implementation would involve challenges related to transmitter/receiver design, signal processing, and environmental factors.

The authors suggest that the approximate location can be found efficiently using local optimization algorithms. Future work could explore expanding the dictionary of gestures, optimizing the precomputation process, and investigating real-world implementation challenges.

Summary updated

BibTeX
@article{yang2016mathematical,
  author = {Can Yang and Hongyu Liu and Yuliang Wang and Can Yang},
  title = {{Mathematical Design of a Novel Gesture-Based Instruction/Input Device Using Wave Detection}},
  journal = {SIAM J. Imaging sci},
  year = {2016},
  doi = {10.1137/16m1063551},
}
Figure