Design and Evaluation of a Hair Combing System Using a General-Purpose Robotic Arm

Nathaniel Dennler, Eura Shin, Maja J. Mataric, Stefanos Nikolaidis

TL;DR

Researchers have developed a modular robotic system that allows a general-purpose arm to assist with hair combing, a vital but often overlooked activity of daily living (ADL). By modeling hair as a "flow field" rather than a static object, the system generates intuitive combing paths that users find significantly more effective than traditional geometric approaches.

What Problem Does This Paper Address?

For the millions of people living with limited mobility or dexterity, simple self-care tasks like combing hair can be exhausting or impossible. While assistive robots exist for tasks like eating or dressing, hair care has remained under-explored.

The challenge is twofold: perception and planning. Hair is notoriously difficult for computer vision to track due to its high variability in color and texture. Furthermore, "correct" combing isn't just about moving from point A to point B; it requires following the natural grain and curvature of the hairstyle to avoid tangles and discomfort.

Methods

The authors propose a three-stage modular pipeline:

  1. Segmentation Module: Using a SqueezeNet-based encoder-decoder architecture, the system identifies hair in RGB images. To handle the "noise" of individual strands, they apply temporal smoothing (averaging masks over time) and spatial filtering to ensure the robot focuses on the most relevant area.
  2. Path Planning Module: This is the "brain" of the system. Instead of just moving downward, the system treats hair as a differential orientation field.
  3. It uses Coherence-Enhancing Shock Filters to reduce the visual noise of stray strands and highlight the dominant "flow" of the hair.
  4. A Gradient Structure Tensor then calculates the local orientation of the hair at every pixel.
  5. When a user clicks a starting point, the system "flows" along these orientations to generate a natural-looking stroke.
  6. Trajectory Generation: The 2D image path is mapped to 3D space using depth data from a Kinect sensor. The robot (a Kinova Gen2) then aligns the comb teeth orthogonally to the hair surface and moves along the planned path.

Datasets

  • Figaro 1k: Used to train the image segmentation module; it contains a diverse set of labeled hair segments from various camera angles.
  • USC Hair Salon Database: Used for the path planning evaluation. The researchers selected 30 hairstyles (short, medium, and long) in five different colors to test the robustness of their algorithm.

Key Insights and Ideas

  • Hair as a Vector Field: The most significant insight is that hair should be modeled like a fluid or a field rather than a rigid mesh. By following the "flow," the robot mimics how a human would naturally brush hair.
  • User-in-the-Loop Autonomy: The system isn't fully autonomous, which is a deliberate design choice. By letting the user select the starting point, the system maintains user agency while reducing the physical effort required to execute the stroke.
  • Perception of Force: In the qualitative study, users often perceived the robot as using too much force if the mannequin's head moved, even when the actual measured force was well within safe limits (under 15N). This highlights that in assistive robotics, visual cues of "gentleness" are just as important as actual safety.

Key Results

  • Path Quality: In a study with 48 participants, the image-based "flow" method was rated significantly higher for both "completeness" and "effectiveness" compared to a baseline "mesh-based" method (which simply calculated the shortest path to the bottom of the hair).
  • Functional Success: A qualitative study with 30 users watching the robot in action confirmed that the system successfully made messy wigs look "straighter" and "neater."
  • Safety: Physical testing showed that peak forces stayed below 15N, comparable to the force used in other delicate tasks like robotic shaving or face wiping.

Limitations and Future Work

  • The "Silence" Problem: Interestingly, users found the silence of the robot during the task "unsettling." The authors suggest adding auditory cues to help users anticipate the robot's movements.
  • Tactile Feedback: The current system relies on vision. The authors note that integrating hybrid position-force control would allow the robot to "feel" tangles and adjust its speed or pull accordingly, much like a human does.
  • Hairstyle Diversity: While the system handled various lengths, it hasn't yet been validated on very curly or sparse hair, which presents different mechanical challenges.
  • Speed: Users noted the robot moved quite slowly. Future iterations will need to balance the need for safety with the user's desire for an efficient routine.

Summary updated

Abstract

This work introduces an approach for automatic hair combing by a lightweight robot. For people living with limited mobility, dexterity, or chronic fatigue, combing hair is often a difficult task that negatively impacts personal routines. We propose a modular system for enabling general robot manipulators to assist with a hair-combing task. The system consists of three main components. The first component is the segmentation module, which segments the location of hair in space. The second component is the path planning module that proposes automatically-generated paths through hair based on user input. The final component creates a trajectory for the robot to execute. We quantitatively evaluate the effectiveness of the paths planned by the system with 48 users and qualitatively evaluate the system with 30 users watching videos of the robot performing a hair-combing task in the physical world. The system is shown to effectively comb different hairstyles.
BibTeX
@inproceedings{dennler2021design,
  author = {Nathaniel Dennler and Eura Shin and Maja J. Mataric and Stefanos Nikolaidis},
  title = {{Design and Evaluation of a Hair Combing System Using a General-Purpose Robotic Arm}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=4OHxyklRvy},
}

Loading related content...

Figure