TL;DR
This paper moves beyond standard geometric accuracy metrics to evaluate how monocular RGB tracking actually affects robot safety decisions. By introducing a two-factor Shapley attribution method, the authors can pinpoint whether a "bad" decision (like an unnecessary stop or a missed danger) was caused by the safety rule's geometry or by failures in the perception pipeline, such as missed detections or poor depth estimation.
What Problem Does This Paper Address?
In autonomous navigation, robots must constantly decide whether to stop or proceed based on the distance to nearby humans. While monocular RGB cameras are cheap and popular, they struggle with depth estimation. To compensate, many systems use a "lateral-only" distance rule—essentially ignoring depth and stopping if anyone is "to the side" of the robot.
Previous studies found that this approach leads to zero missed dangers but a massive number of unnecessary stops. However, it wasn't clear if these "false alarms" were due to the camera's inaccuracy or the conservative nature of the rule itself. This paper aims to untangle these factors, providing a framework to attribute decision-level errors to their specific sources.
Methods
The authors evaluate a full perception-to-decision pipeline (shown in Figure 1) consisting of:
- Detection: YOLOv8m-Pose to identify humans and keypoints.
- Localisation: Comparing three methods: a geometric baseline (M0), a calibrated ground-plane projection (M1), and a learned depth model, Depth Anything V2 (M2).
- Tracking: A constant-velocity Kalman filter.
- Decision Rules: Comparing the "lateral-only" rule against a standard planar distance rule and a proposed "uncertainty margin" rule ($d - k\sigma < \tau$) that uses the Kalman filter's variance to buy safety.

The core innovation is the use of two-factor Shapley attribution. This mathematical framework splits the total decision error into "rule shares" and "perception shares." This allows researchers to see, for example, exactly how much of a missed danger was due to a person being missed by the detector versus being localized incorrectly.
Datasets
The study uses real-world pedestrian video from the ETH and UCY datasets. These datasets provide metric ground-plane annotations for hundreds of pedestrians. Because these are static videos, the authors used simulated stationary robots placed on a 1m grid to evaluate potential interactions. They also employed a strict calibration/evaluation split to ensure that no individual pedestrian seen during the "tuning" phase was used for the final results.
Key Insights and Ideas
- The Lateral Rule is a Geometric Bound: The authors prove (Proposition 1) that the lateral-only distance is a mathematical lower bound on the true planar distance. This means its 0% miss rate isn't a "win" for the sensor; it's an inherent property of the geometry that comes at the cost of extreme inefficiency.
- Detection is the Weakest Link: In these elevated views, missed detections (low recall) were the primary source of missed dangers. If the detector doesn't see a person, the safety rule can't react, regardless of how good the depth estimation is.
- Uncertainty Margins Beat "Hacks": Instead of discarding depth information (the lateral rule), using a 2-sigma uncertainty margin on a planar distance rule provided a better trade-off. It matched the safety of the lateral rule while significantly reducing unnecessary stops.
- Localisation Hierarchy: A simple, calibrated ground-plane mapping (M1) outperformed both the anthropometric geometric baseline and the sophisticated "Depth Anything V2" model in these specific scenes.
Key Results
- Localisation Accuracy: The ground-plane method (M1) achieved a planar RMSE of 0.51m–0.59m, significantly better than the geometric baseline (1.39m–1.72m) and Depth Anything V2 (0.68m–0.81m).
- Decision Efficiency: On annotated positions, the lateral rule had a stop precision as low as 5.6%, meaning nearly 95% of its stops were unnecessary.
- Safety Trade-offs: As shown in Figure 3, the lateral rule's precision drops as the scene gets deeper, a trend that closely matches a theoretical "uniform-occupancy" model.

Limitations and Future Work
The authors acknowledge several limitations:
- Simulated Interaction: The robots were stationary and simulated; real-world dynamic avoidance was not tested.
- Camera Perspective: The ETH/UCY datasets use elevated, near-nadir views. Results might differ significantly for a robot-mounted camera at eye level.
- Ground Truth Quality: The "ground truth" consists of manual annotations projected through homographies, which contains its own inherent errors.
The next steps involve repeating this protocol using motion-capture ground truth and testing from actual robot-level viewpoints to see how perspective affects the perception-to-decision error split.
Summary updated