Abstract
Against the backdrop of global economic development, industrial assembly in Germany is in a state of change, prompted by the country’s high-wage business environment. This article describes the need for new technologies to increase flexibility and security regarding human-robot cooperation. Furthermore, a possible model for managing this transition is described in detail, involving the use of an RGBD camera from Microsoft. With this camera, it should be possible to detect state and position changes of people in a human-robot workstation and consequently adapt the movements of the robot. Overall, the essential aim of this paper is to suggest ways to increase economic efficiency within assembly processes along with increasing security.
Introduction
In high-wage industrial environments, manufacturers face a dichotomy between highly efficient but rigid fully automated lines and flexible but error-prone manual assembly. The ideal solution lies in Human-Robot Cooperation (HRC), where robots manage repetitive, heavy tasks while humans contribute intuition and fine motor skills. However, traditional industrial robots require safety fences that limit flexibility. To remove these barriers, robots must become "aware" of their human counterparts. This research addresses the challenge of creating a reliable interface that monitors a worker's physical and emotional state—such as fatigue or distraction—to ensure safety in a shared, fenceless workspace.
Conceptual Framework of Interaction
To understand the transition toward fenceless environments, it is necessary to distinguish between different levels of human-robot interaction. The authors categorize these into three distinct stages:
- Coexistence: Humans and robots work in separate spaces with no shared workspace.
- Cooperation: Humans and robots share a workspace but perform tasks at different times.
- Collaboration: Humans and robots work in the same space simultaneously on the same task.
The ultimate goal of this research is to facilitate true collaboration by accounting for human unpredictability and internal factors like stress and fatigue.
Methodology and Experimental Setup
The researchers developed a demonstrator system centered around two primary hardware components: a Universal Robots UR3 lightweight robot and a Microsoft Kinect 2.0 RGBD camera. The UR3 was selected for its integrated force-limiting functions, which provide an inherent layer of safety for collaborative tasks.
The system architecture processes data through a multi-stage pipeline. The Kinect camera captures Red, Green, Blue, and Depth data, which is then analyzed using two primary methods:
- Facial Analysis: Monitoring eye and mouth states to detect fatigue and tracking head orientation (pitch, yaw, and roll) to determine the worker's focus.
- Skeletal Tracking: Mapping XYZ coordinates of human joints to verify if the worker is in the designated "reference" working position or has moved away.
This data is processed on a PC and communicated to the robot control unit, which can trigger safety stops or speed adjustments in real-time.
Key Results and Insights
The experimental evaluation revealed several critical findings regarding the feasibility of camera-based monitoring in industrial settings:
- Tracking Constraints: Facial tracking was found to be reliable only within specific rotation ranges: ±25° for pitch and yaw, and ±45° for roll. Beyond these limits, the sensor loses the ability to accurately track features.
- The Occlusion Challenge: A significant practical hurdle is that the robot arm itself often blocks the camera's view of the human. This suggests that single-camera setups are insufficient for robust safety.
- Safety Logic Implementation: The system successfully demonstrated automated responses, such as stopping the robot if the human's gaze left the work cell or if the human stood up. Additionally, the system can switch the robot to a "fully automated" mode to work on a buffer of parts if the human leaves the area.
Conclusion and Future Work
While the demonstrator proves the concept of using RGBD sensors for HRC safety, limitations remain. The Microsoft Kinect 2.0 is a laboratory tool and not safety-rated for industrial production. Future work should focus on integrating industrial-grade 3D sensors and utilizing multiple camera angles to eliminate "blind spots" caused by robot occlusion. Further development is also needed to move beyond binary safety stops toward fluid, real-time speed modulation based on human movement patterns.
Summary updated