Introduction
Safe and efficient human-robot interaction (HRI) in collaborative environments requires precise and reliable human tracking. While marker-based motion capture systems provide high accuracy, their intrusive nature and high cost limit their applicability in real-world settings. Marker-less tracking using Human Pose Estimation (HPE) models offers a more flexible alternative; however, these systems are prone to noise, depth inaccuracies, and tracking loss during occlusions. Such instabilities result in "robot jitter," which significantly undermines human trust and perceived safety. This paper presents a novel filtering pipeline designed to mitigate these issues by integrating biomechanical constraints and a dynamic permanence filter.
Methodology
The proposed system is a "plug-and-play" pipeline that processes raw HPE data before it reaches the robot controller. It consists of three primary modules:
- Spatial Evaluation: This module applies biomechanical heuristics to filter out physically impossible detections. By estimating subject height and verifying limb proportions (e.g., forearm length relative to height), the system identifies and penalizes "hallucinated" joints.
- Temporal Tracking: To maintain identity across frames, the system employs the Hungarian method. The cost matrix is uniquely constructed by combining Euclidean distance with prediction confidence, ensuring robust label assignment even during person-to-person intersections.
- Confidence-Based Permanence Filter: At the core of the pipeline is an Object Permanence Filter (OPF) implemented via a particle filter. Unlike static filters, this module dynamically adjusts its noise parameters based on the HPE confidence score. During occlusions, the system transitions to a dynamics module that predicts human movement based on prior trajectories, maintaining smooth interaction until the subject is re-acquired.
Experimental Evaluation
The pipeline was validated using an Intel RealSense D455 RGB-D camera and a Franka Emika Panda robot arm. Seven participants performed four collaborative tasks (T0-T3) designed to test various challenges, including non-linear motion and heavy occlusions (e.g., box handovers). Ground truth data was collected using an OptiTrack marker-based system to quantify tracking error.
Results and Discussion
The proposed filter demonstrated significant improvements over baseline HPE data and standard Kalman filter implementations. Quantitative analysis revealed a 49% reduction in Mean Absolute Error (MAE) and a 71% reduction in the standard deviation (STD) of the error. This reduction in STD is particularly critical, as it directly translates to the elimination of erratic robot movements.
Beyond technical metrics, the system significantly enhanced the user experience. Subjective surveys indicated that perceived safety and comfort scores rose from 2.2/5 with standard filtering to 4.6/5 with the proposed pipeline. Furthermore, the filter maintains real-time performance with a negligible latency of approximately 8ms, making it suitable for 30Hz robotic applications.
Conclusion
This research introduces a robust, real-time filtering pipeline that bridges the gap between noisy marker-less HPE and the high-precision requirements of HRI. By leveraging biomechanical common sense and dynamic confidence-based filtering, the system ensures smooth robot behavior even under challenging conditions. Future work will explore the integration of multi-camera setups and advanced intent prediction to further enhance tracking persistence during prolonged occlusions.
Summary updated