LiteFat: Lightweight Spatio-Temporal Graph Learning for Real-Time Driver Fatigue Detection

Jing Ren, Suyu Ma, Hong Jia, Xiwei Xu, Ivan Lee, Haytham Fayek, Xiaodong Li, Feng Xia

Introduction

Driver fatigue is a critical safety concern, contributing to nearly 30% of all fatal road accidents globally. While computer vision systems have shown promise in monitoring driver behavior—specifically identifying signs of drowsiness like yawning or prolonged eye closure—most existing solutions are computationally expensive. These "heavyweight" models often require high-end GPUs and significant memory, making them unsuitable for the low-power embedded processors typically found in automotive hardware.

Furthermore, many graph-based models rely on fixed, predefined facial structures. By manually defining which facial landmarks are connected, these systems may introduce human bias and fail to capture subtle, non-obvious motion patterns indicative of fatigue. To address these challenges, we present LiteFat, a lightweight, real-time driver fatigue detection model that utilizes an adaptive spatio-temporal graph neural network to achieve high accuracy with minimal computational overhead.

Methodology

The LiteFat framework is designed for efficiency and robustness, consisting of three integrated modules:

1. Facial Landmark and Feature Extraction

The system identifies 68 specific facial landmarks to capture geometric structure. However, geometry alone can miss vital contextual information like skin texture or subtle wrinkles. To compensate, LiteFat employs MobileNetV3-Small to extract deep-learning "contextual embeddings," providing a richer representation of the driver's state without the weight of a standard convolutional neural network.

2. Multimodal Feature Fusion

LiteFat merges the geometric coordinates of the facial landmarks with the visual features extracted from MobileNet. This fusion creates a comprehensive feature matrix that represents both the physical position of facial features and the visual nuances of the driver's expression.

3. Spatio-Temporal Graph Learning (STGL)

This core module treats the face as a dynamic graph where landmarks serve as nodes.

  • Adaptive Adjacency Matrix: Unlike traditional models with fixed connections, LiteFat learns the relationships between facial points during training. This allows the model to discover which connections are most relevant for fatigue detection dynamically.
  • GCN & TCN Architecture: The model uses Graph Convolutional Networks (GCN) to process spatial facial expressions and Temporal Convolutional Networks (TCN) to analyze how these expressions evolve over time. TCNs were selected over traditional Recurrent Neural Networks (like LSTMs) because they allow for better parallel processing and avoid common gradient issues, significantly increasing inference speed.

Experimental Results

The researchers evaluated LiteFat using the YawDD (Yawning Detection Dataset), which includes a diverse range of drivers (male/female, with/without glasses) and various camera angles. The model was tested against three primary actions: normal driving, talking, and yawning.

Performance and Efficiency

LiteFat demonstrates a superior balance of accuracy and resource management:

  • Accuracy: The model achieved a state-of-the-art 99.5% accuracy on the YawDD dataset.
  • Model Size: LiteFat utilizes only 1.32 million parameters. In comparison, the JHPFA-Net model uses 226 million parameters, making LiteFat approximately 170 times smaller.
  • Real-Time Readiness: The system recorded a forward pass time of 1.39 seconds and a throughput of 5.75 samples per second, proving its viability for real-time deployment on embedded systems.
  • Robustness: ROC curve analysis showed an Area Under the Curve (AUC) of 1.00 for yawning detection, indicating near-perfect classification between alert and fatigued states.

Key Insights

  • Adaptive Learning: Self-learned graph structures consistently outperform standard mathematical models (like the Laplacian matrix). By allowing the model to define its own facial point relationships, it captures fatigue cues that human-defined models might overlook.
  • Hybrid Advantage: Combining lightweight landmarks with MobileNetV3-Small embeddings provides the "best of both worlds"—geometric precision and textural context—at a fraction of the computational cost of traditional deep learning models.

Conclusion and Future Work

LiteFat represents a significant step toward practical, real-time driver monitoring. While the model excels at detecting yawning and talking, future research will focus on integrating broader modalities, such as heart rate data or steering wheel patterns, to create a more holistic safety system. Additionally, the team plans to deploy LiteFat onto physical robotic devices to test its performance under varying real-world lighting conditions and vehicle vibrations, ensuring the model's robustness in actual driving environments.

Summary updated

BibTeX
@inproceedings{ren2025litefat,
  author = {Jing Ren and Suyu Ma and Hong Jia and Xiwei Xu and Ivan Lee and Haytham Fayek and Xiaodong Li and Feng Xia},
  title = {{LiteFat: Lightweight Spatio-Temporal Graph Learning for Real-Time Driver Fatigue Detection}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2025},
  doi = {10.1109/iros60139.2025.11247563},
}
Figure