TL;DR
This paper introduces a novel asynchronous human-machine interface (HMI) for controlling humanoid robots, combining electrooculography (EOG) for double-blink detection and a gyroscope for head rotation. This system allows users to issue a rich set of commands with high accuracy and low latency, demonstrating improved performance in simulated home service tasks compared to traditional manual control.
What Problem Does This Paper Address?
The core challenge addressed is the development of effective human-machine interfaces (HMIs) for robot teleoperation, particularly for individuals with motor disabilities. Existing manual interfaces like joysticks and keyboards become unusable for those with severe neuromuscular disorders. While non-manual HMIs like Brain-Computer Interfaces (BCIs) using EEG or EOG show promise, they often struggle with the trade-off between providing a sufficient number of commands and maintaining high accuracy with low response times. Specifically, traditional BCI paradigms like P300 and SSVEP are limited by the number of targets they can present without increasing response time. EOG-based systems, while offering good signal-to-noise ratios, have faced challenges in creating asynchronous systems that provide ample commands with low latency, often relying on synchronous flashing cues or limited eye movements.
Methods
The proposed HMI utilizes two key inputs:
- Electrooculogram (EOG): Specifically, a "double blink" gesture is used as a trigger. This is detected by filtering the EOG signal, identifying two distinct peaks (peak1 and peak2), and analyzing their duration and energy.
- Gyroscope: This device measures head rotation, providing yaw and roll angles. These angles are used to calculate cursor movement on a graphical user interface (GUI).
These inputs are integrated into a multilevel Graphical User Interface (GUI). The GUI features a navigation panel and a joint control panel.
- Navigation Panel: Offers 8 navigation buttons and 3 speed control buttons.
- Joint Control Panel: Operates in three levels:
- Joint Selection: Users choose a specific joint (e.g., head, shoulder, elbow).
- Joint Motion Selection: Based on the selected joint, users choose from 2-4 motion options (e.g., two-axis rotations).
- Stopping Level: A large button to halt any ongoing motion.
A novel continuous control strategy is implemented. This strategy breaks down robot actions into two phases: a "motion start" phase initiated by selecting a navigation or joint command, and a "motion stop" phase triggered by the double blink when the robot reaches the desired state. This avoids the need to combine multiple commands for a single action or convert complex motions into discrete commands.
The system's algorithm involves:
- Cursor Position Calculation: Using gyroscope yaw and roll angles to update cursor position on the GUI.
- Double Blink Detection: Employing Pearson correlation coefficient (PCC) to assess similarity to a double-blink segment and extracting features like duration and energy to confirm a blink.
- Threshold Calculation: A calibration process is performed for each user to determine personalized thresholds for blink detection.
Datasets
Not discussed in the paper. The experiments were conducted with human subjects performing tasks in a simulated environment.
Key Insights and Ideas
- Asynchronous Control with Rich Commands: The HMI successfully achieves an asynchronous control paradigm, allowing users to issue commands at their own pace, while simultaneously providing a comprehensive set of 40 commands (navigation, joint motion, speed, and stop).
- Double Blink as a Robust Trigger: The double blink, combined with cursor position, proves to be a reliable and accurate trigger mechanism, achieving very high accuracy in button-clicking tasks.
- Continuous Control Strategy: Disassembling robot motion into "start" and "stop" phases significantly simplifies control and leads to more natural, fluid robot movements, reducing the cognitive load on the user.
- Synergy of EOG and Gyroscope: The combination of eye movements (for triggering) and head movements (for cursor control) offers a powerful and intuitive way to interact with the robot.
- Improved Home Service Task Performance: The system demonstrates a tangible benefit in practical applications, showing improved efficiency and command usage in simulated home service scenarios compared to manual control.
Key Results
- Target Button Triggering (Experiment I):
- Accuracy (ACC): 99.2%
- Response Time (RT): 2.34 seconds
- False Positive Rate (FPR): 0.34 events/min (idle state)
- The system outperformed previous BCI and EOG-based HMIs in terms of ACC and RT.
- Humanoid Robot Control for Home Services (Experiment II):
- Average Accuracy (ACC): 99.3%
- Average Information Transfer Rate (ITR): 113.9 bits/min
- Time Ratio (HMI vs. Manual Control): 1.22 (meaning HMI control took 1.22 times longer on average)
- Command Ratio (HMI vs. Manual Control): 1.18 (meaning HMI control used 1.18 times more commands on average)
- All subjects completed the home service tasks without failure.
- The results indicate that the proposed HMI and continuous control strategy can improve performance in humanoid robot control for home services.
Limitations and Future Work
- Visual Feedback: Subjects in the home service experiment required more visual feedback beyond the robot's monocular vision to accurately perceive positional relationships between the robot's arm and objects.
- Machine Vision Comparison: The HMI control was not as precise as machine vision for object manipulation, as automatic control can leverage location data for optimal trajectory planning.
- Future Work:
- Reduce FPR: Improve the EOG detection algorithm and refine the continuous control strategy to further minimize false commands.
- Shared Control System: Integrate machine vision guidance with HMI control to create a shared control system. This would allow HMI to handle navigation and object identification, while machine vision handles automatic manipulation, reducing complexity and increasing precision.
- Automatic Grasp System: Develop an automatic grasp system guided by machine vision.
Summary updated