Abstract
This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i.e. 2D position plus 1D rotation within the world coordinate system). Our approach, T-Pose-LSTM (Temporal 3DOF-Pose Long-Short-Term Memory), is trained using long-term data from real-world robot deployments and aims to learn context-dependent (environment- and time-specific) human activities. Our approach incorporates long-term temporal information (i.e. date and time) with short-term pose observations as input. A sequence-to-sequence LSTM encoder-decoder is trained, which encodes observations into LSTM and then decodes as predictions. For deployment, it can perform on-the-fly prediction in real-time. Instead of using manually annotated data, we rely on a robust human detection, tracking and SLAM system, providing us with examples in a global coordinate system. We validate the approach using more than 15K pedestrian trajectories recorded in a care home environment over a period of three months. The experiment shows that the proposed T-Pose-LSTM model advances the state-of-the-art 2D-based method for human trajectory prediction in long-term mobile robot deployments.
Introduction
Predicting human movement is essential for mobile robots to navigate safely and socially. However, most existing research suffers from three main issues: a dimensionality gap where models ignore orientation; a coordinate mismatch between image-space and real-world global coordinates; and a lack of environmental context. Human behavior is often periodic—people cluster in specific areas at specific times, such as reception desks in the morning or coffee machines during breaks.
The authors propose T-Pose-LSTM (Temporal 3DOF-Pose Long-Short-Term Memory) to bridge these gaps. By utilizing long-term deployment data, the model learns how spatial and temporal context influences human movement, predicting not just position but also orientation (3DOF).
Methodology
The core of the proposed system is a sequence-to-sequence LSTM encoder-decoder. Unlike traditional models that take only (x, y) coordinates, T-Pose-LSTM incorporates:
- 3DOF Pose Input: 2D position plus 1D rotation (yaw), represented as quaternions.
- Temporal Context: Long-term features such as the calendar day and the time of day (hour/minute/second) are injected alongside short-term pose observations.
- Probabilistic Output: The model outputs parameters of a bi-variant Gaussian distribution for position, allowing the robot to quantify prediction uncertainty.
Datasets and Evaluation
The study utilizes two primary datasets:
- STRANDS Dataset: Collected over 19 weeks in a care home, containing over 17,000 trajectories. This serves as the primary source for learning periodic behaviors.
- L-CAS Dataset: Collected using a 3D LiDAR in a busy university canteen, capturing complex group behaviors across 935 trajectories.
The researchers used the robot’s own SLAM and tracking systems to generate a massive labeled dataset automatically, avoiding the need for manual annotation.
Key Results
T-Pose-LSTM was compared against the Social-LSTM baseline. On the STRANDS dataset, T-Pose-LSTM achieved a lower Average Displacement Error (ADE) across all time horizons. For a 9-second prediction, the model achieved an ADE of 1.38m, a nearly 30% improvement over the baseline's 1.95m.
Ablation studies confirmed that including temporal context and orientation significantly reduces error. Orientation acts as a leading indicator of intent, while time-of-day features allow the model to learn environmental "rhythms." Despite its complexity, the system runs at 10Hz, making it feasible for real-time mobile robot deployment.
Conclusion
By incorporating 3DOF poses and long-term temporal context, T-Pose-LSTM significantly improves the accuracy of pedestrian trajectory prediction in real-world environments. This approach allows autonomous robots to better anticipate human movement by understanding the "when" and "where" of their surroundings.
Summary updated