SwinUNet3D - A Hierarchical Architecture for Deep Traffic Prediction using Shifted Window Transformers

Alabi Bojesomo, Hasan Al-Marzouqi, Panos LiatsisarXiv 2022 · arXiv

Introduction

Traffic forecasting has transitioned from simple time-series analysis into a complex spatiotemporal "video prediction" task. Modern datasets represent urban traffic as a sequence of image-like frames where each pixel encodes critical information such as vehicle volume and speed. While Convolutional Neural Networks (CNNs) and U-Net architectures have traditionally dominated this field, they often struggle to capture long-range dependencies and the intricate inter-relationships between disparate time steps and data channels.

Transformers offer a solution for modeling these long-range interactions but are typically computationally prohibitive for high-resolution 3D data. SwinUNet3D addresses this by integrating the efficiency of Video Swin Transformers into a hierarchical U-Net framework, creating a model that is both computationally viable and highly effective at capturing 3D traffic dynamics.

Methodology

SwinUNet3D is a pure transformer-based hierarchical architecture that eschews traditional convolutional blocks in its primary stages. The model's design is centered around several key innovations:

3D Shifted Window Transformers

To avoid the quadratic computational cost of global attention, SwinUNet3D utilizes "shifted windows." Attention is computed within small, local 3D blocks. In subsequent layers, these windows are shifted, allowing information to propagate and mix across the entire spatiotemporal field.

Feature Mixing Layer

A critical component of the architecture is the specialized "feature mixing" layer. Before entering the transformer blocks, the model flattens the time and channel dimensions and processes them through a fully connected layer. This allows the network to learn the inter-relationships between features—such as the correlation between speed and volume—across time before spatial processing begins.

Asymmetric U-Net Structure

To manage the high parameter count inherent in transformer models, the authors implemented an asymmetric structure. The encoder is "heavy," featuring four transformer layers per block, while the decoder is "light," utilizing only one transformer layer per block. This design ensures the model remains trainable on standard hardware.

Architecture of SwinUNet3D (a) and the internal structure of the Swin-Transformer blocks (b).

Experimental Setup and Datasets

The model was evaluated using the Traffic Map Movie Forecasting Challenge 2021 (Traffic4cast2021) dataset. This benchmark includes traffic data from 10 diverse cities over a two-year period. The task involves taking 12 frames of traffic state (representing one hour) and predicting the subsequent 6 frames. The input features consist of 8 dynamic channels (volume and speed for four cardinal directions) and 9 static channels (road map geometry).

Results and Discussion

SwinUNet3D demonstrates superior performance compared to standard graph-based and convolution-based baselines provided by the Traffic4cast competition.

Performance comparison showing Swin-UNet3D achieving the lowest Core MSE against GCN and CNN-based U-Net baselines.

The results indicate that the transformer-based approach is better suited for capturing the complex dependencies in traffic data. Ablation studies further highlighted the importance of the Feature Mixing Layer; its inclusion reduced the Core MSE from 49.99 to 49.72, suggesting that understanding feature interactions over time is as vital as spatial localization.

Conclusion

SwinUNet3D proves that a U-Net architecture can function effectively for 3D spatiotemporal tasks without relying on traditional convolutional blocks. While the model is more resource-intensive than 2D CNNs and requires substantial data, its ability to model complex traffic patterns sets a new benchmark. Future research will focus on multi-objective loss formulations and hypercomplex networks to further refine token mixing and predictive accuracy.

Summary updated

Abstract

Traffic forecasting is an important element of mobility management, an important key that drives the logistics industry. Over the years, lots of work have been done in Traffic forecasting using time series as well as spatiotemporal dynamic forecasting. In this paper, we explore the use of vision transformer in a UNet setting. We completely remove all convolution-based building blocks in UNet, while using 3D shifted window transformer in both encoder and decoder branches. In addition, we experiment with the use of feature mixing just before patch encoding to control the inter-relationship of the feature while avoiding contraction of the depth dimension of our spatiotemporal input. The proposed network is tested on the data provided by Traffic Map Movie Forecasting Challenge 2021(Traffic4cast2021), held in the competition track of Neural Information Processing Systems (NeurIPS). Traffic4cast2021 task is to predict an hour (6 frames) of traffic conditions (volume and average speed)from one hour of given traffic state (12 frames averaged in 5 minutes time span). Source code is available online at https://github.com/bojesomo/Traffic4Cast2021-SwinUNet3D.
BibTeX
@misc{bojesomo2022swinunet,
  author = {Alabi Bojesomo and Hasan Al-Marzouqi and Panos Liatsis},
  title = {{SwinUNet3D - A Hierarchical Architecture for Deep Traffic Prediction using Shifted Window Transformers}},
  year = {2022},
  eprint = {2201.06390},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2201.06390},
}
Figure