Highly Discriminative Driver Distraction Detection Method Based on Swin Transformer

Ziyang Zhang, Lie Yang, Chen LvVehicles 2024 · DOI

TL;DR

This paper introduces a novel driver distraction detection method, ST-HDFL, that leverages the powerful feature extraction capabilities of Swin Transformers and a new "highly discriminative feature learning strategy." This strategy aims to improve the model's ability to distinguish between different distraction behaviors by reducing intra-class variations and increasing inter-class separation in the feature space, leading to state-of-the-art performance.

What Problem Does This Paper Address?

Driver distraction is a major cause of traffic accidents, making accurate and reliable detection crucial for improving road safety and advancing intelligent transportation systems. While deep learning methods have shown promise, existing approaches often struggle with feature discrimination. They primarily rely on classification loss, which can lead to overlapping features between different distraction classes, limiting overall accuracy. Furthermore, the limited availability of training data in this domain poses a significant challenge, leading to overfitting. This paper tackles these issues by proposing a method that enhances feature discriminability and addresses potential overfitting.

Methods

The core of the proposed ST-HDFL method lies in two main components:

  • Swin Transformer for Feature Extraction: The paper utilizes the Swin Transformer, a powerful vision transformer architecture known for its hierarchical feature representation and efficient window-based self-attention mechanism. This allows it to capture multi-scale features effectively, outperforming traditional CNNs in many visual tasks. The Swin Transformer is employed to extract rich feature representations from input images.
  • Highly Discriminative Feature Learning Strategy: This novel strategy is designed to improve the separability of features in the latent space. It comprises:
  • Sample Center (SC) Loss: This loss function aims to reduce the distance between feature vectors of samples belonging to the same class and their respective class center vectors. This effectively pulls samples of the same class closer together in the feature space.
  • Center Vector Shift Process: This process dynamically adjusts the class center vectors during training. By shifting each center vector away from the overall average of all center vectors, it actively increases the distance between different class centers, thereby enhancing inter-class discrimination.

Additionally, the paper employs data augmentation techniques based on image transformations (cropping and rotation) to mitigate overfitting issues arising from limited training data.

Datasets

The proposed method was evaluated on two publicly available datasets:

  • AUC-DDD (AUC Distracted Driver Detection) dataset: This dataset contains 17,308 images from 31 participants, divided into 10 classes of driving states.
  • State-Farm dataset: This dataset, sourced from a Kaggle competition, includes 22,424 labeled images (and 79,726 unlabeled), also divided into 10 classes, with image sizes of 640x480.

Key Insights and Ideas

  • Swin Transformer's Superiority: The paper strongly advocates for Swin Transformer's effectiveness in driver distraction detection, demonstrating its superior feature learning capabilities compared to classical CNNs and even the Vision Transformer (ViT) on the AUC-DDD dataset.
  • Beyond Classification Loss: The research highlights that optimizing solely with classification loss is insufficient for achieving highly discriminative features. The proposed SC loss and center vector shift strategy offer a more effective way to structure the feature space.
  • Visualizing Feature Space: The use of t-SNE to visualize feature distributions is a powerful tool for demonstrating how the proposed strategy effectively clusters same-class samples and separates different-class samples, providing strong qualitative evidence for its effectiveness.
  • Addressing Data Scarcity: The combination of Swin Transformer's robust feature learning and the proposed data augmentation strategy effectively tackles the challenge of limited training data, preventing overfitting and improving generalization.

Key Results

The ST-HDFL model achieved state-of-the-art performance on both datasets:

  • AUC-DDD Dataset: The method achieved an accuracy of 95.66%.
  • State-Farm Dataset: The method achieved an accuracy of 99.71%.

In comparative experiments, ST-HDFL outperformed several other state-of-the-art methods, including Drive-Net, MobileVGG, and Vanilla CNN. The confusion matrices also indicated high accuracy across all 10 distraction classes, with most classes achieving over 90% accuracy.

Limitations and Future Work

The authors acknowledge two primary limitations:

  • Computational Complexity: Swin Transformer, while powerful, has higher computational complexity compared to some classical CNN models, leading to longer processing times per image (4.81 ms for Swin Transformer vs. 0.52-2.85 ms for others).
  • Lack of Temporal Information: The current model processes images individually, neglecting the temporal continuity of driver distraction. This limits its ability to fully capture dynamic distraction behaviors.

For future work, the authors plan to:

  • Develop lighter models that maintain high accuracy to reduce computational load.
  • Design video-based driver distraction detection methods to leverage temporal features between adjacent frames.

Summary updated

Abstract

Driver distraction detection not only helps to improve road safety and prevent traffic accidents, but also promotes the development of intelligent transportation systems, which is of great significance for creating a safer and more efficient transportation environment. Since deep learning algorithms have very strong feature learning abilities, more and more deep learning-based driver distraction detection methods have emerged in recent years. However, the majority of existing deep learning-based methods are optimized only through the constraint of classification loss, making it difficult to obtain features with high discrimination, so the performance of these methods is very limited. In this paper, to improve the discrimination between features of different classes of samples, we propose a high-discrimination feature learning strategy and design a driver distraction detection model based on Swin Transformer and the highly discriminative feature learning strategy (ST-HDFL). Firstly, the features of input samples are extracted through the powerful feature learning ability of Swin Transformer. Then, the intra-class distance of samples of the same class in the feature space is reduced through the constraint of sample center distance loss (SC loss), and the inter-class distance of samples of different classes is increased through the center vector shift strategy, which can greatly improve the discrimination of different class samples in the feature space. Finally, we have conducted extensive experiments on two publicly available datasets, AUC-DD and State-Farm, to demonstrate the effectiveness of the proposed method. The experimental results show that our method can achieve better performance than many state-of-the-art methods, such as Drive-Net, MobileVGG, Vanilla CNN, and so on.
BibTeX
@inproceedings{zhang2024highly,
  author = {Ziyang Zhang and Lie Yang and Chen Lv},
  title = {{Highly Discriminative Driver Distraction Detection Method Based on Swin Transformer}},
  booktitle = {Vehicles},
  year = {2024},
  doi = {10.3390/vehicles6010006},
}
Figure