TL;DR
This paper introduces DE NIRO, a human-centered, mobile humanoid robot designed for domestic assistance, particularly for caregivers. It showcases DE NIRO's ability to autonomously navigate, recognize objects, and perform complex manipulation tasks, including dual-arm grasping, using a combination of off-the-shelf hardware and open-source software. The project emphasizes sharing its design and code to accelerate research in human-robot interaction and cognitive robotics.
What Problem Does This Paper Address?
The increasing presence of robots in domestic environments, especially for assisting the elderly, presents unique challenges. Unlike industrial robots confined to controlled settings, domestic robots must navigate dynamic, unpredictable spaces, interact safely and naturally with humans, and handle objects of varying shapes and sizes. Existing research often focuses on specialized robots or lacks a comprehensive, open-source platform for developing and testing human-centered manipulation capabilities. This paper addresses the need for a versatile, accessible research platform that can tackle these complex, real-world manipulation and interaction tasks, with a specific focus on supporting caregivers.
Methods
The authors developed DE NIRO, a humanoid robot built on a QUICKIE wheelchair base, equipped with two Baxter manufacturing-grade arms. Key methodological contributions include:
- Hardware Integration: Combining off-the-shelf components (Baxter arms, QUICKIE base, various sensors like Kinect, 2D/3D LIDARs, 360-degree cameras) with custom-designed elements.
- Custom Grippers: Two novel gripper designs (spherical and bio-inspired soft fingers) to handle a wider range of objects and provide haptic feedback.
- DE VITO Exoskeleton: A lightweight, open-source exoskeleton for intuitive teleoperation, enabling semi-autonomous control.
- ROS Middleware: Extensive use of the Robot Operating System (ROS) for seamless communication and integration of diverse hardware and software modules.
- State Machine Architecture: A modular, finite-state machine approach (using SMACH) to manage complex task sequences like fetching objects.
- Perception Pipeline: A multi-stage vision system for object recognition, segmentation, and pose estimation, utilizing fiducial markers (AprilTags) for robustness.
- Motion Planning: Employing MoveIt! with the RRT* algorithm for efficient and robust motion planning, and an Ackermann steering-inspired approach for dual-arm manipulation.
- Gazebo Simulation: A detailed simulation environment for safe testing and development before deployment on the physical robot.
Datasets
Not discussed in the paper. The paper focuses on the robot's capabilities and its own sensor data rather than relying on pre-trained datasets for its core functionalities.
Key Insights and Ideas
- Human-Centered Design is Crucial: The robot's design and capabilities are explicitly tailored for natural, safe, and productive interaction with humans, particularly in domestic settings.
- Open-Source and Off-the-Shelf Accelerates Research: By leveraging existing hardware and open-source software (ROS, Gazebo, SMACH), the authors demonstrate how to build a complex research platform rapidly, encouraging community adoption.
- Fiducial Markers for Robust Object Recognition: Despite the desire for more general object recognition, fiducial markers proved highly reliable for accurate object pose estimation in their challenges, highlighting a practical trade-off between generality and robustness.
- Dual-Arm Manipulation for Enhanced Capabilities: The paper emphasizes the significant advantages of using two arms for handling larger, unevenly shaped objects, offering greater maneuverability and control.
- The Importance of a Unified Framework: ROS acts as the central nervous system, enabling disparate sensors and actuators to work in concert, transforming individual components into a cohesive robotic system.
Key Results
The paper demonstrates DE NIRO's capabilities through three main challenges:
- Fetch Routine: DE NIRO successfully receives voice commands, navigates to an object warehouse, identifies and grasps a target object (using fiducial markers), and returns it to the user.
- Cognitively-Enhanced Manipulation: The robot demonstrates improved object recognition and grasping for small objects of unknown shape, showcasing its ability to adapt to less structured environments.
- Dual-Arm Manipulation: DE NIRO successfully grasps and manipulates larger box-shaped objects using both arms, highlighting its enhanced dexterity and control. The motion planning for this task achieved an approximate 90% success rate for physically reachable poses.
Limitations and Future Work
The authors acknowledge several limitations and suggest future research directions:
- Non-Holonomic Movement: DE NIRO's wheelchair base limits its movement to forward/backward and rotational, making it prone to getting stuck in corners.
- Payload Capacity: The 2.2 kg per arm payload limits the types of tasks it can perform (e.g., lifting a human is impossible).
- Limited Sensor Concurrency: The robot currently processes sensors sequentially rather than concurrently, limiting its ability to react to dynamic environments in real-time.
- General Object Recognition: While fiducial markers are robust, future work could focus on more general object recognition techniques (e.g., point cloud-based methods) for unknown environments.
- Enhanced Autonomy and Intelligence: The authors suggest improving DE NIRO's ability to understand human pose and form, learn skills through demonstration, and handle more complex collision scenarios.
- Integration of Capabilities: Combining all developed modules (object detection, dual-arm manipulation, fetch routine) into a single, cohesive system for live interaction is a key future step.
- Dynamic Environments: Further research is needed to improve navigation in highly dynamic environments with human presence.
Summary updated