Ring-a-Pose: A Ring for Continuous Hand Pose Tracking

Tianhong Catherine Yu, Guilin Hu, Ruidong Zhang, Hyunchul Lim, Saif Mahmud, Chi-Jung Lee, Ke Li, Devansh Agarwal, Shuyang Nie, Jin-Seok Oh, François Guimbretière, Cheng Zhang

TL;DR

Ring-a-Pose is a single, untethered smart ring that enables continuous 3D hand pose tracking and micro-gesture recognition using active acoustic sensing. By emitting inaudible sound waves and analyzing their reflection patterns off the hand, the system achieves high-precision tracking (10.3mm joint error) without the bulk of data gloves or the occlusion issues of wrist-worn cameras.

What Problem Does This Paper Address?

Tracking the complex, 21-degree-of-freedom movement of the human hand is a holy grail for AR/VR and human-computer interaction. Current solutions fall into three camps, each with significant drawbacks:

  1. Vision-based systems: Cameras (on-body or environmental) suffer from privacy concerns and "line-of-sight" issues—if a sleeve covers your wrist or a finger is hidden behind the palm, the system fails.
  2. Glove-based systems: These provide excellent data but are socially awkward, hot, and cumbersome for daily wear.
  3. Wrist-worn sensors: Devices like smartwatches are often too far from the fingers to capture fine-grained "micro-gestures" and struggle with the orientation disparity between the wrist and the moving hand.

The authors of Ring-a-Pose ask: Can we achieve full 3D hand tracking using just a single, low-profile ring?

Methods

The core innovation is the use of Active Acoustic Sensing via a technique called C-FMCW (Correlation-based Frequency Modulated Continuous Wave).

  • The Hardware: The ring contains a tiny speaker and a microphone. The speaker emits an inaudible acoustic signal (20-24 kHz).
  • The Physics: As the hand moves, the palm and fingers reflect these sound waves. Because the ring is located "inside" the hand's geometry, every pose creates a unique "echo profile."
  • The Pipeline: The system generates two types of data:
  • Original Echo Profile: Captures the static geometry of the hand.
  • Differential Echo Profile: Highlights changes between frames to amplify subtle movements.
  • The AI: These profiles are fed into a ResNet18-based encoder-decoder architecture. The model is trained to perform two tasks: regressing the 3D coordinates of 20 hand joints and classifying specific micro-gestures.

Datasets

The researchers conducted three distinct user studies with a total of 36 participants:

  • Study 1 (Worn Finger Comparison): Evaluated how tracking performance changes depending on which finger wears the ring.
  • Study 2 (Continuous Tracking): The main benchmark, involving 20 terminal poses (including all 10 ASL digits). This study produced 18.02 hours of training data from 16 different hands.
  • Study 3 (Micro-gestures): Focused on 7 discrete thumb-to-index finger gestures (tap, pinch, etc.).
  • Ground Truth: Data was labeled using MediaPipe Hands, a standard vision-based hand tracking framework, synchronized with the acoustic signals.

Key Insights and Ideas

  • Occlusion as a Feature, Not a Bug: In computer vision, if a finger blocks the camera, data is lost. In acoustic sensing, that "occlusion" creates a strong, repeatable reflection pattern. The system actually uses the "blocking" of the sensor to better identify the pose.
  • The "Inside-Out" Advantage: By placing the sensor on the finger rather than the wrist, the researchers eliminated the "orientation disparity." The sensor moves with the hand, providing a consistent frame of reference.
  • Anatomical Consistency: The system works surprisingly well across different users without per-user training (User-Independent mode). This is because human hand anatomy is structurally similar enough that reflection patterns generalize across the population.
  • Low Power, High Resolution: The system consumes only 148.0 mW, making it feasible for future integration into commodity smart rings like the Oura or Samsung Galaxy Ring.

Key Results

  • Tracking Accuracy:
    • User-Independent: 14.1 mm mean per-joint position error (MPJPE).
  • User-Dependent (Fine-tuned): 10.3 mm MPJPE.
  • Gesture Recognition:
    • Achieved 90.60% accuracy for 7-class micro-gestures out-of-the-box.
  • Accuracy jumped to 99.27% with just 9.3 minutes of user-specific training data.
  • Robustness: The system remained accurate even in the presence of environmental noise (music, talking) and when the hand was near different surfaces (plastic, fabric, or metal).

Limitations and Future Work

  • Latency: The current prototype has a total delay of 413–492 ms. While much of this is due to Bluetooth (BLE) overhead and visualization rendering, it is currently too slow for high-speed gaming or real-time AR.
  • Battery Life: The 70mAh battery lasts only about 1.75 hours in its current unoptimized state.
  • Kinematic Constraints: The deep learning model occasionally predicts "broken" fingers (anatomically impossible poses). The authors suggest integrating kinematic models (like MANO) in the future to enforce realistic hand movements.
  • Dynamic Environments: While tested against static noise, the impact of "global" hand movements (like walking or running) on the acoustic profile remains an open research question.

Summary updated

Abstract

We present Ring-a-Pose, a single untethered ring that tracks continuous 3D hand poses. Located in the center of the hand, the ring emits an inaudible acoustic signal that each hand pose reflects differently. Ring-a-Pose imposes minimal obtrusions on the hand, unlike multi-ring or glove systems. It is not affected by the choice of clothing that may cover wrist-worn systems. In a series of three user studies with a total of 30 participants, we evaluate Ring-a-Pose's performance on pose tracking and micro-finger gesture recognition. Without collecting any training data from a user, Ring-a-Pose tracks continuous hand poses with a joint error of 14.1mm. The joint error decreases to 10.3mm for fine-tuned user-dependent models. Ring-a-Pose recognizes 7-class micro-gestures with a 90.60% and 99.27% accuracy for user-independent and user-dependent models, respectively. Furthermore, the ring exhibits promising performance when worn on any finger. Ring-a-Pose enables the future of smart rings to track and recognize hand poses using relatively low-power acoustic sensing.
BibTeX
@misc{yu2024ring,
  author = {Tianhong Catherine Yu and Guilin Hu and Ruidong Zhang and Hyunchul Lim and Saif Mahmud and Chi-Jung Lee and Ke Li and Devansh Agarwal and Shuyang Nie and Jin-Seok Oh and François Guimbretière and Cheng Zhang},
  title = {{Ring-a-Pose: A Ring for Continuous Hand Pose Tracking}},
  year = {2024},
  eprint = {2404.12980},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.12980},
}
Figure