A Grasping Movement Intention Estimator for Intuitive Control of Assistive Devices

Etienne Moullet, François Bailly, Justin Carpentier, Christine CosteBiosystems 2024 · DOI · arXiv · OpenAlex

TL;DR

Researchers have developed i-GRIP, an algorithm that predicts which object a person wants to grab and how they intend to hold it, simply by analyzing their natural hand movements. By eliminating the need for clunky manual switches or complex neural interfaces, this "collaborative control" approach could make assistive devices like exoskeletons and prostheses feel like a natural extension of the body.

What Problem Does This Paper Address?

For individuals with upper-limb impairments—such as those resulting from spinal cord injuries or strokes—regaining the ability to grasp objects is a top priority for independence. While assistive technologies like robotic exoskeletons and Functional Electrical Stimulation (FES) exist, controlling them is often frustratingly unintuitive.

Currently, users often have to navigate "state machines," manually switching between modes using joysticks, buttons, or specific stereotypical movements (like a shoulder shrug) to tell the device what to do. This creates a high cognitive load, leads to jerky "saccadic" movements, and ultimately makes daily tasks feel like a chore rather than a natural action. The authors aim to bridge this gap by creating a system that "reads" the user's intent from the kinematics of their reach.

Methods

The i-GRIP (Grasping Movement Intention Estimator) algorithm acts as a middleman between a computer vision system and an assistive device. It assumes the environment is already being tracked (using 3D hand positions and 6D object poses) and then performs a real-time kinematic analysis.

The core of the algorithm is a global confidence score calculated from four distinct metrics:

  1. Ray Impacts: It projects a "cone of rays" from the hand's predicted future trajectory. If these rays hit an object's mesh, the score goes up. This is particularly effective for fast, curved movements.
  2. Distance Derivative: It tracks how fast the hand is closing the gap to an object's center. This works best for straight, high-speed approaches.
  3. Distance: A simple measure of how close the hand is to the object's surface.
  4. Future Distance: A measure of how close the predicted future position of the hand is to the object.

The system uses a velocity-dependent weighted sum of these metrics to identify the target. Once the target is identified, i-GRIP determines the appropriate grip (e.g., a palmar grasp for the middle of a bottle or a pinch grasp for the top) based on the hand's position relative to the object's geometry.

Datasets

The researchers evaluated i-GRIP using:

  • The YCB Object and Model Set: A standard benchmark in robotics. Specifically, they used a mustard bottle, a bleach bottle, a tomato can, and a box of Cheez-Its.
  • Experimental Data: They recorded 11 healthy participants performing 128 grasping movements.
  • Vision Tools: Hand tracking was handled by MediaPipe, and object poses were estimated using CosyPose, a state-of-the-art deep learning method for 6D pose estimation.

Key Insights and Ideas

  • Multi-Metric Fusion is Key: No single metric can capture intent across all types of movement. By combining "future-looking" metrics (rays and future distance) with "current" metrics (distance and velocity), the system remains robust whether the user is moving quickly and erratically or slowly and precisely.
  • Collaborative Control: The "aha" moment is the shift from command-based control to collaborative control. The user provides the high-level "where" by moving their arm, and the algorithm fills in the "what" and "how."
  • Exploiting Object Symmetry: For grip selection, the algorithm simplifies objects into cylinders. By looking at where the hand is relative to the cylinder's axis, it can reliably distinguish between a power grip on the body of a bottle and a precision pinch on the cap.

Key Results

The experimental results were highly promising, demonstrating both accuracy and speed:

  • Target Identification: 89.9% success rate.
  • Grip Selection: 94.8% success rate.
  • Responsiveness: The system identified the target within 0.53 seconds and the grip within 0.39 seconds on average.
  • Temporal Margin: Crucially, these identifications happened well before the movement ended (0.67s to 0.80s before completion), giving the assistive device plenty of time to prepare the physical grasp.

Limitations and Future Work

While i-GRIP performs well in a controlled lab setting, the authors acknowledge several hurdles:

  • Oblong Bias: The current grip selection logic is optimized for oblong objects. Irregularly shaped items might require more complex geometric analysis.
  • Healthy vs. Impaired Kinematics: The study used healthy participants. People with motor impairments often have different movement profiles (slower, more tremors, or different trajectories), which might require fine-tuning the algorithm's weights.
  • Real-World Noise: The study used offline processing of camera data. Moving to a fully real-time, wearable system will introduce challenges in lighting, occlusion (objects blocking each other), and computational constraints.

The next steps involve testing i-GRIP with the target population—individuals with upper-limb disabilities—to see if the "intuitive" promise holds up in real-world clinical scenarios.

Summary updated

BibTeX
@inproceedings{moullet2024grasping,
  author = {Etienne Moullet and François Bailly and Justin Carpentier and Christine Coste},
  title = {{A Grasping Movement Intention Estimator for Intuitive Control of Assistive Devices}},
  booktitle = {Biosystems},
  year = {2024},
  doi = {10.1007/978-3-031-77584-0\_32},
}

Loading related content...

Figure