Introduction
As Virtual Reality (VR) technology matures, the industry is shifting from traditional handheld controllers toward "non-contact" gesture recognition. This transition aims to provide a more natural and immersive user experience. However, a significant challenge remains: how to effectively measure the quality of a gesture-based interface. Traditional metrics often focus exclusively on technical accuracy or subjective user preference. This paper proposes a holistic dual-evaluation framework that bridges the gap between technical performance and psychological user satisfaction.
Methodology: The Dual-Evaluation Framework
The proposed methodology evaluates the "interactive experience" through two distinct tracks:
- Performance Measurement: This track utilizes objective metrics to quantify efficiency, including:
- Task Completion: Success or failure of the intended action.
- Time Limitation: The duration required to execute a gesture.
- Mistake Count: Frequency of recognition errors.
- Effectiveness ($\eta$): The ratio of completed work to time elapsed.
- Emotional Evaluation: This track captures the user's subjective state using the ITU-T P.910 ACR (Absolute Category Rating) scale. Participants rate their experience from 1 (Terrible) to 5 (Excellent), allowing researchers to correlate technical friction with user frustration.
System Implementation
The researchers implemented a prototype using a smartphone-based VR headset integrated with a Leap Motion controller. The software environment was built using the Leap Motion SDK and Unity, capable of tracking 44 joint coordinates per frame at high frequencies (up to 120fps).
The system visualizes hand movements as a skeletal model within the virtual environment, ensuring that the user receives immediate visual feedback on their physical gestures.
Experimental Findings
The study involved 20 participants testing 10 specific gestures (A1–A10) covering navigation (forward, backward, left, right) and rotation.
Performance vs. Complexity
The results revealed a clear "Frustration Threshold." When gestures became too complex or recognition failed repeatedly, user sentiment dropped sharply. Simple directional movements (forward, left, right) exhibited the highest efficiency and emotional scores (approx. 4.5/5). In contrast, backward movements and complex two-handed rotations (A9, A10) suffered from high error rates and low subjective ratings (approx. 3/5).
Spatial Asymmetry
A key insight was the discovery of spatial asymmetry in recognition. While the sensor handled lateral and forward movements with ease, "backward" operations were significantly more difficult for the system to track accurately, likely due to the physical geometry of the depth-sensing hardware.
Conclusion
This research demonstrates that gesture recognition quality is not merely a technical hurdle but a primary driver of user retention and emotional engagement. By integrating objective performance data with subjective emotional feedback, developers can better identify "low-emotion" interactions early in the design phase. Future work will focus on expanding the task library and adapting these metrics to a broader range of demographics and hardware configurations.
Summary updated