Safe and Efficient Navigation Considering Texting Pedestrians via Attention-Aware Cost-map Tuning

Shunya Tadano, Yusuke Tamura, Ankit A. Ravankar, Yasuhisa HirataarXiv 2026 · arXiv

TL;DR

Researchers have developed a navigation framework that allows robots to identify smartphone-distracted pedestrians in real-time using 3D skeletal poses. By dynamically widening safety margins around "distracted" individuals in the robot's cost-map, the system significantly improves perceived navigation smoothness and predictability without sacrificing travel efficiency.

What Problem Does This Paper Address?

As autonomous service robots move into hospitals, offices, and malls, they face a growing challenge: the "smartphone zombie." Traditional social navigation algorithms often assume that pedestrians are attentive, cooperative agents who will notice a robot and adjust their path accordingly.

However, texting pedestrians suffer from "inattentional blindness." They deviate from social norms, walk slower, and—crucially—are unlikely to yield. When a pedestrian isn't looking up, the entire burden of conflict resolution falls on the robot. Existing methods that treat all pedestrians uniformly often result in reactive, jerky movements that feel unsafe or unpredictable to the humans involved. This paper seeks to bridge the gap between objective obstacle avoidance and socially legible navigation.

Methods

The authors propose a three-part framework integrated into a mobile robot platform (a WHILL Model CR wheelchair):

  1. 3D Pose & Attention Classification: Using an RGB-D camera and MediaPipe BlazePose, the system extracts 33 skeletal keypoints. A lightweight Random Forest classifier then determines if the person is "Attentive" or "Distracted." The model specifically looks for characteristic postures like raised forearms and downward head tilts.
  2. Trajectory Prediction: The robot uses a Human Scene Transformer (HST) to predict the pedestrian's future path based on their historical movement and skeletal data.
  3. Adaptive Cost-map Tuning: This is the core innovation. The robot maintains a "cost-map" (a grid representing the difficulty/risk of moving through space). The system adds a dynamic layer that assigns a safety margin around the predicted human path.
  4. Attentive pedestrians get a standard 0.3m margin.
  5. Distracted pedestrians get a wider 0.5m margin.
Fig. 2: Framework of the proposed system. The system consists of three main modules: (1) pedestrian trajectory prediction based on historical position data, (2) Attention Classification using 3D skeletal keypoint extraction (33 points) from RGB-D camera images, (3) adaptive CostMap Generation using predicted trajectories based on attention state classification results.

By widening the margin for distracted users, the robot plans a proactive, wider arc well before a conflict occurs, rather than making a last-minute adjustment.

Datasets

  • Classifier Training: The authors created a custom dataset by manually labeling 304 images from the JRDB (Joint Robot-person Dataset), splitting them into 243 training and 61 test images.
  • Trajectory Prediction: The Human Scene Transformer (HST) models were pre-trained on the full JRDB dataset.
  • Evaluation: A within-subjects user study was conducted with 12 participants across 252 trials in a T-shaped indoor corridor.

Key Insights and Ideas

  • Prediction $\neq$ Perception: One of the most striking findings is that trajectory prediction alone (Method B in the study) makes the robot move objectively smoother (lower jerk), but humans don't perceive it as better. It is only when the robot adjusts its behavior based on the human's attention state (Method C) that users report a significant boost in comfort and predictability.
  • The "Why" of the Pose: The Random Forest model's feature importance (Figure 4) revealed that the right wrist, elbow, and neck are the most critical indicators of distraction. This confirms the model is successfully identifying the physical act of holding and looking at a phone rather than just general gait changes.
  • Legibility is King: By consistently widening its path for distracted users, the robot’s intent becomes "legible." The human can easily anticipate where the robot is going, which reduces the stress of the encounter.
Fig. 4: Feature importance of the Random Forest model. Feature importance scores indicating that wrist and shoulder positions are dominant factors.

Key Results

The researchers compared three methods: (A) LiDAR baseline, (B) Trajectory prediction with fixed margins, and (C) The proposed attention-aware method.

  • Objective Smoothness: Both prediction-based methods (B and C) reduced "jerk" (the rate of change of acceleration) by nearly three-fold compared to the LiDAR baseline.
  • Subjective Preference: Method C was the clear winner across all seven qualitative metrics, including Safety ($p=0.003$) and Predictability ($p=0.034$).
  • Efficiency: Remarkably, the proactive, wider avoidance of distracted pedestrians did not increase travel time. The robot simply started its maneuver earlier rather than slowing down or stopping.
TABLE III: Subjective evaluation results (Q1–Q7). Mean scores are shown for each method.

Limitations and Future Work

While promising, the authors note several areas for improvement:

  • Demographics: The user study was limited to 12 graduate students (ages 22–28). Future work needs to validate these findings with broader populations, such as the elderly or non-technical users.
  • Temporal Dynamics: The current classifier works frame-by-frame. It doesn't "remember" if someone was distracted a second ago, which can lead to flickering classifications during transitions (e.g., someone putting their phone away).
  • Multi-Pedestrian Scaling: The current study focused on single-pedestrian interactions. Real-world deployment will require the robot to prioritize safety margins across multiple people with different attention states simultaneously.
  • Sensor Range: The forward-facing camera limits tracking to the robot's frontal cone; a 360-degree system would allow the robot to maintain "awareness" even after passing a pedestrian.

Summary updated

Abstract

Smartphone-distracted pedestrians deviate from social norms unpredictably and cannot be relied upon to yield to approaching robots -- a safety challenge that existing navigation methods are poorly equipped to handle, since they treat all pedestrians uniformly regardless of attention state. We propose an attention-aware navigation framework that classifies each pedestrian as attentive or distracted in real-time using 3D skeletal features and a Random Forest classifier, then dynamically widens cost-map safety margins around distracted individuals to drive proactive avoidance. A within-subjects user study with 12 participants across 252 trials demonstrates significant improvements in perceived smoothness (p = 0.003) and predictability (p = 0.034) over both a LiDAR-only baseline and a trajectory-prediction-only method, without any increase in travel time. Notably, trajectory prediction alone does not improve subjective navigation quality unless coupled with attention-state-aware cost adjustment -- confirming that appropriately responding to who is in front of the robot can matter more than trajectory smoothness alone.
BibTeX
@misc{tadano2026safe,
  author = {Shunya Tadano and Yusuke Tamura and Ankit A. Ravankar and Yasuhisa Hirata},
  title = {{Safe and Efficient Navigation Considering Texting Pedestrians via Attention-Aware Cost-map Tuning}},
  year = {2026},
  eprint = {2611.16725},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16725},
}
Figure