NICOL: A Neuro-inspired Collaborative Semi-humanoid Robot that Bridges Social Interaction and Reliable Manipulation

Matthias Kerzel, Philipp Allgeuer, Erik Strahl, Nicolas Frick, Jan-Gerrit Habekost, Manfred Eppe, Stefan Wermter

Introduction

In the current robotics landscape, a distinct "personality split" exists between social robots and industrial manipulators. Social robots, such as Pepper or Nao, excel at human interaction through facial expressions and gaze but lack the physical strength and reach required for meaningful labor. Conversely, industrial arms like the UR-5 offer high precision and power but are often perceived as intimidating and remain "blind" to social cues, making them difficult for non-experts to collaborate with safely.

NICOL (Neuro-Inspired COLlaborator) is designed to bridge this gap. As an adult-sized, semi-humanoid platform, NICOL combines the expressive capabilities of a social robot with the robust manipulation power of industrial-grade hardware, creating a "missing link" for true human-robot collaboration.

Hardware Design and the Kinematic Challenge

NICOL’s architecture is modular, designed to facilitate both social signaling and complex physical tasks. Its head, based on the predecessor NICO, features LED arrays for facial expressions and dual 4K cameras for high-resolution visual perception. This social interface is mounted alongside two Robotis OpenManipulator-P arms, each providing 6 Degrees of Freedom (DoF), and Seed Robotics RH8D five-fingered hands.

One of the primary challenges in developing NICOL was the workspace scale. While many humanoid robots operate in small, tabletop environments, NICOL is designed for an adult-sized workspace (100x100 cm). The researchers found that standard 6-DoF configurations were insufficient for this scale, often resulting in "dead zones" or self-collisions. By extending the system to 8 DoF per arm, the robot achieved the range of motion necessary to mimic human reach while maintaining specific hand orientations required for complex grasping.

Methodology: Hybrid Neuro-Genetic Learning

Early experiments revealed a "Scaling Wall": end-to-end deep learning models that worked for smaller robots (achieving 85% accuracy on NICO) failed significantly when applied to NICOL’s larger, high-dimensional workspace, dropping to near 0% accuracy. To overcome this, the authors proposed a modular hybrid pipeline:

  1. Visual Perception: A Convolutional Neural Network (CNN) identifies and locates target objects within the 4K camera feed.
  2. Coordinate Transformation: A Multi-Layer Perceptron (MLP) translates these 2D image coordinates into 3D world coordinates.
  3. GAIKPy (Genetic Algorithm for Inverse Kinematics): Rather than relying on traditional Jacobian-based solvers—which often struggle with local optima in high-DoF systems—the researchers developed GAIKPy. This novel solver uses evolutionary strategies combined with gradient-based optimization (SLSQP) to find optimal joint configurations for reaching and grasping.

Datasets and Evaluation

To support this research, the authors generated and released three new datasets, each containing 10,000 samples collected in simulated environments (CoppeliaSim/Gazebo). These datasets vary by the number of active joints (6, 7, or 8 DoF) and include egocentric 4K images, target coordinates, and successful joint configurations.

In comparative testing against state-of-the-art IK solvers such as KDL, TRAC-IK, and BIO-IK, the hybrid GAIKPy approach demonstrated superior reliability. While standard solvers peaked at approximately 95-96% accuracy, the NICOL pipeline achieved a 99.17% grasp accuracy.

Discussion and Future Work

The study highlights two critical insights for future humanoid development. First, the necessity of 8 DoF for adult-sized workspaces to avoid kinematic constraints. Second, the robustness of genetic algorithms over traditional calculus-based IK for complex, human-like manipulators.

However, this accuracy comes with a trade-off in computational speed; GAIKPy requires approximately one second to calculate a solution, which is slower than traditional solvers. While acceptable for many collaborative tasks, future work will focus on optimizing this latency. Additionally, the researchers plan to bridge the "sim-to-real" gap by testing the system against real-world sensor noise and integrating social feedback loops—such as the robot displaying a "confused" expression when it encounters a manipulation failure—to further enhance the collaborative experience.

Summary updated

Abstract

Robotic platforms that can efficiently collaborate with humans in physical tasks constitute a major goal in robotics. However, many existing robotic platforms are either designed for social interaction or industrial object manipulation tasks. The design of collaborative robots seldom emphasizes both their social interaction and physical collaboration abilities. To bridge this gap, we present the novel semi-humanoid NICOL, the Neuro-Inspired COLlaborator. NICOL is a large, newly designed, scaled-up version of its well-evaluated predecessor, the Neuro-Inspired COmpanion (NICO). NICOL adopts NICO's head and facial expression display and extends its manipulation abilities in terms of precision, object size, and workspace size. Our contribution in this paper is twofold -- firstly, we introduce the design concept for NICOL, and secondly, we provide an evaluation of NICOL's manipulation abilities by presenting a novel extension for an end-to-end hybrid neuro-genetic visuomotor learning approach adapted to NICOL's more complex kinematics. We show that the approach outperforms the state-of-the-art Inverse Kinematics (IK) solvers KDL, TRACK-IK and BIO-IK. Overall, this article presents for the first time the humanoid robot NICOL, and contributes to the integration of social robotics and neural visuomotor learning for humanoid robots.
BibTeX
@inproceedings{kerzel2023nicol,
  author = {Matthias Kerzel and Philipp Allgeuer and Erik Strahl and Nicolas Frick and Jan-Gerrit Habekost and Manfred Eppe and Stefan Wermter},
  title = {{NICOL: A Neuro-inspired Collaborative Semi-humanoid Robot that Bridges Social Interaction and Reliable Manipulation}},
  booktitle = {IEEE Access},
  year = {2023},
  url = {https://openreview.net/forum?id=iqKm0Z4XGB},
}
Figure