TrajGATFormer: A graph-based transformer approach for worker and obstacle trajectory prediction in off-site construction environments

Mohammed Alduais, Xinming Li, Qipei Mei

In the high-stakes environment of off-site construction, efficiency is king, but safety is the foundation. As workers, heavy machinery, and massive structural panels move in tight, synchronized dances, the risk of "struck-by" accidents—one of the industry's "Focus Four" hazards—remains a critical concern.

A new paper from researchers at the University of Alberta introduces TrajGATFormer, a sophisticated AI framework designed to predict the future paths of both workers and moving obstacles, providing the "eyes" for future collision-avoidance systems.

TL;DR

TrajGATFormer is a hybrid deep learning framework that combines Graph Attention Networks (GAT) with Transformers to predict trajectories in dynamic construction environments. By modeling both social interactions and long-term movement patterns, it reduces trajectory prediction errors by up to 38% compared to traditional models like LSTMs.

What Problem Does This Paper Address?

Construction sites are chaotic. Unlike a robot in a controlled factory, a construction worker’s movement is influenced by their task, their colleagues, and moving equipment. Traditional prediction methods often fall short:

  • Knowledge-based models (like Kalman Filters) assume constant velocity, failing when a worker suddenly stops or turns.
  • Standard Data-driven models (like LSTMs) struggle to capture the "social" context—how a worker moves differently because a 2-ton floor panel is swinging toward them.

The authors identify a gap in existing research: most models treat obstacles as stationary points. In reality, off-site construction involves moving components (panels, cranes) that fundamentally alter worker behavior.

Methods

The TrajGATFormer framework is a multi-stage pipeline:

  1. Detection & Tracking: It uses YOLOv10n for real-time object detection and DeepSORT to maintain unique IDs for every worker and obstacle across video frames.
  2. Spatial Modeling (The GAT): The system treats the site as a graph where workers and obstacles are nodes. A Graph Attention Network calculates the "influence" entities have on each other, effectively modeling social and environmental interactions.
  3. Temporal Modeling (The Transformer): While the GAT handles the where, the Transformer handles the when. It uses an encoder-decoder architecture to process sequences of past movements and predict future coordinates over a 4.8-second horizon.
  4. Transfer Learning: The models were first trained on general pedestrian data (ETH dataset) to learn basic human motion before being fine-tuned on specific construction site data.

The authors developed two versions: TrajGATFormer (focused on workers) and TrajGATFormer-Obstacle (which adds a dedicated encoder to process moving panels).

Datasets

  • ETH Zurich Dataset: A standard benchmark containing pedestrian trajectories in urban settings, used here for pre-training.
  • New Construction Dataset: A real-world dataset recorded at an off-site construction facility in Edmonton, Alberta. It features workers interacting with moving floor and roof panels. This dataset is unique because it captures "stop fractions"—the reality that construction workers spend significant time stationary while performing tasks, a behavior quite different from typical city pedestrians.

Key Insights and Ideas

  • Obstacles are Informative: Surprisingly, including moving obstacles in the model (TrajGATFormer-Obstacle) didn't just help predict the obstacle; it actually improved the accuracy of worker trajectory predictions. Knowing where the panel is going helps the AI understand why the worker is moving out of the way.
  • The Hybrid Advantage: By combining GATs and Transformers, the model solves two problems at once: the GAT captures the "social force" (spatial), while the Transformer captures the "intent" or long-term trend (temporal).
  • Construction vs. Commuting: The paper highlights that workers have a high "stop fraction" (often stopped >70% of the time). Models trained only on pedestrians (who are usually walking from A to B) fail on construction sites because they expect constant motion.

Key Results

The proposed models significantly outperformed baselines like Social-LSTM and Social-GAN:

  • Accuracy Boost: TrajGATFormer-Obstacle achieved an Average Displacement Error (ADE) of 1.03 meters and a Final Displacement Error (FDE) of 2.06 meters.
  • Comparative Gains: This represents a 35% reduction in ADE and a 38% reduction in FDE compared to traditional LSTM-based approaches.
  • Real-time Potential: By using the lightweight YOLOv10n backbone, the framework is designed with the efficiency needed for eventual deployment in real-time alarm systems.

Limitations and Future Work

While a major step forward, the authors acknowledge several hurdles:

  • Data Scarcity: There is still limited annotated data for diverse moving obstacles, which led to lower accuracy for panel trajectories compared to worker trajectories.
  • Sudden Maneuvers: The model still struggles with "sharp turns" or sudden, unpredictable changes in movement that don't follow a clear historical trend.
  • Long-term Drift: Accuracy naturally declines as the prediction horizon extends.

Future Direction: The team aims to integrate these predictions into automated safety protocols—essentially creating a system that can trigger a "kill switch" or a wearable haptic alert before a collision even occurs.

Summary updated

BibTeX
@article{alduais2026trajgatformer,
  author = {Mohammed Alduais and Xinming Li and Qipei Mei},
  title = {{TrajGATFormer: A graph-based transformer approach for worker and obstacle trajectory prediction in off-site construction environments}},
  journal = {Adv. Eng. Informatics},
  year = {2026},
  doi = {10.1016/j.aei.2026.104583},
}
Figure