TL;DR
Researchers have developed a navigation framework that allows robots to identify smartphone-distracted pedestrians in real-time using 3D skeletal poses. By dynamically widening safety margins around "distracted" individuals in the robot's cost-map, the system significantly improves perceived navigation smoothness and predictability without sacrificing travel efficiency.
What Problem Does This Paper Address?
As autonomous service robots move into hospitals, offices, and malls, they face a growing challenge: the "smartphone zombie." Traditional social navigation algorithms often assume that pedestrians are attentive, cooperative agents who will notice a robot and adjust their path accordingly.
However, texting pedestrians suffer from "inattentional blindness." They deviate from social norms, walk slower, and—crucially—are unlikely to yield. When a pedestrian isn't looking up, the entire burden of conflict resolution falls on the robot. Existing methods that treat all pedestrians uniformly often result in reactive, jerky movements that feel unsafe or unpredictable to the humans involved. This paper seeks to bridge the gap between objective obstacle avoidance and socially legible navigation.
Methods
The authors propose a three-part framework integrated into a mobile robot platform (a WHILL Model CR wheelchair):
- 3D Pose & Attention Classification: Using an RGB-D camera and MediaPipe BlazePose, the system extracts 33 skeletal keypoints. A lightweight Random Forest classifier then determines if the person is "Attentive" or "Distracted." The model specifically looks for characteristic postures like raised forearms and downward head tilts.
- Trajectory Prediction: The robot uses a Human Scene Transformer (HST) to predict the pedestrian's future path based on their historical movement and skeletal data.
- Adaptive Cost-map Tuning: This is the core innovation. The robot maintains a "cost-map" (a grid representing the difficulty/risk of moving through space). The system adds a dynamic layer that assigns a safety margin around the predicted human path.
- Attentive pedestrians get a standard 0.3m margin.
- Distracted pedestrians get a wider 0.5m margin.
By widening the margin for distracted users, the robot plans a proactive, wider arc well before a conflict occurs, rather than making a last-minute adjustment.
Datasets
- Classifier Training: The authors created a custom dataset by manually labeling 304 images from the JRDB (Joint Robot-person Dataset), splitting them into 243 training and 61 test images.
- Trajectory Prediction: The Human Scene Transformer (HST) models were pre-trained on the full JRDB dataset.
- Evaluation: A within-subjects user study was conducted with 12 participants across 252 trials in a T-shaped indoor corridor.
Key Insights and Ideas
- Prediction $\neq$ Perception: One of the most striking findings is that trajectory prediction alone (Method B in the study) makes the robot move objectively smoother (lower jerk), but humans don't perceive it as better. It is only when the robot adjusts its behavior based on the human's attention state (Method C) that users report a significant boost in comfort and predictability.
- The "Why" of the Pose: The Random Forest model's feature importance (Figure 4) revealed that the right wrist, elbow, and neck are the most critical indicators of distraction. This confirms the model is successfully identifying the physical act of holding and looking at a phone rather than just general gait changes.
- Legibility is King: By consistently widening its path for distracted users, the robot’s intent becomes "legible." The human can easily anticipate where the robot is going, which reduces the stress of the encounter.
Key Results
The researchers compared three methods: (A) LiDAR baseline, (B) Trajectory prediction with fixed margins, and (C) The proposed attention-aware method.
- Objective Smoothness: Both prediction-based methods (B and C) reduced "jerk" (the rate of change of acceleration) by nearly three-fold compared to the LiDAR baseline.
- Subjective Preference: Method C was the clear winner across all seven qualitative metrics, including Safety ($p=0.003$) and Predictability ($p=0.034$).
- Efficiency: Remarkably, the proactive, wider avoidance of distracted pedestrians did not increase travel time. The robot simply started its maneuver earlier rather than slowing down or stopping.
Limitations and Future Work
While promising, the authors note several areas for improvement:
- Demographics: The user study was limited to 12 graduate students (ages 22–28). Future work needs to validate these findings with broader populations, such as the elderly or non-technical users.
- Temporal Dynamics: The current classifier works frame-by-frame. It doesn't "remember" if someone was distracted a second ago, which can lead to flickering classifications during transitions (e.g., someone putting their phone away).
- Multi-Pedestrian Scaling: The current study focused on single-pedestrian interactions. Real-world deployment will require the robot to prioritize safety margins across multiple people with different attention states simultaneously.
- Sensor Range: The forward-facing camera limits tracking to the robot's frontal cone; a 360-degree system would allow the robot to maintain "awareness" even after passing a pedestrian.
Summary updated