Continuous micro finger writing recognition with a commodity smartwatch: demo abstract

Wenqiang Chen, Lin Chen, Meiyi Ma, Farshid Salemi Parizi, Shwetak N. Patel, John A. Stankovic

Abstract

Input is a significant problem for wearable devices, particularly for head-mounted virtual and augmented reality systems. Contemporary AR/VR systems use in-air gestures or handheld controllers for interactivity. However, mid-air handwriting provides a natural, subtle, and easy-to-use way to input commands and text. In this demo, we propose and investigate ViFin, a new technique for input commands and text entry which tracks continuous micro finger-level writing with a commodity smartwatch through vibrations. Inspired by the recurrent neural aligner and transfer learning, ViFin recognizes continuous finger writing and works across different users and achieves an accuracy of 90% and 91% for recognizing numbers and letters, respectively. Finally, a real-time writing system with two specific applications using AR smartglasses are implemented.
BibTeX
@inproceedings{chen2020continuous,
  author = {Wenqiang Chen and Lin Chen and Meiyi Ma and Farshid Salemi Parizi and Shwetak N. Patel and John A. Stankovic},
  title = {{Continuous micro finger writing recognition with a commodity smartwatch: demo abstract}},
  booktitle = {SenSys},
  year = {2020},
  doi = {10.1145/3384419.3430417},
}

Loading related content...

Figure