Natural Criteria for Comparison of Pedestrian Flow Forecasting Models

Tomás Vintr, Zhi Yan, Kerem Eyisoy, Filip Kubis, Jan Blaha, Jirí Ulrich, Chittaranjan Srinivas Swaminathan, Sergi Molina Mellado, Tomasz Kucner, Martin Magnusson, Grzegorz Cielniak, Jan Faigl, Tom Duckett, Achim J. Lilienthal, Tomás Krajník

Abstract

Models of human behaviour, such as pedestrian flows, are beneficial for safe and efficient operation of mobile robots. We present a new methodology for benchmarking of pedestrian flow models based on the afforded safety of robot navigation in human-populated environments. While previous evaluations of pedestrian flow models focused on their predictive capabilities, we assess their ability to support safe path planning and scheduling. Using real-world datasets gathered continuously over several weeks, we benchmark state-of-the-art pedestrian flow models, including both time-averaged and time-sensitive models. In the evaluation, we use the learned models to plan robot trajectories and then observe the number of times when the robot gets too close to humans, using a predefined social distance threshold. The experiments show that while traditional evaluation criteria based on model fidelity differ only marginally, the introduced criteria vary significantly depending on the model used, providing a natural interpretation of the expected safety of the system. For the time-averaged flow models, the number of encounters increases linearly with the percentage operating time of the robot, as might be reasonably expected. By contrast, for the time-sensitive models, the number of encounters grows sublinearly with the percentage operating time, by planning to avoid congested areas and times.
BibTeX
@inproceedings{vintr2020natural,
  author = {Tomás Vintr and Zhi Yan and Kerem Eyisoy and Filip Kubis and Jan Blaha and Jirí Ulrich and Chittaranjan Srinivas Swaminathan and Sergi Molina Mellado and Tomasz Kucner and Martin Magnusson and Grzegorz Cielniak and Jan Faigl and Tom Duckett and Achim J. Lilienthal and Tomás Krajník},
  title = {{Natural Criteria for Comparison of Pedestrian Flow Forecasting Models}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  doi = {10.1109/iros45743.2020.9341672},
}

Loading related content...

Figure