MDFG: Multi-Dimensional Fine-Grained Modeling for Fatigue Detection

Mei Wang, Xiaojie Zhu, Ruimin Hu, Dongliang Zhu, Liang Liao, Mang YeAAAI 2025 · DOI

Introduction

Fatigue detection is traditionally treated as a binary classification problem—distinguishing between "alert" and "fatigued" states. However, this coarse-grained approach fails to account for the continuous, transitional nature of fatigue. This paper introduces the Multi-Dimensional Fine-Grained (MDFG) modeling framework, which moves beyond binary labels to capture temporal resolution and distributional complexity through synthesized intermediate states and meta-learning.

The Problem with Coarse-Grained Labels

The primary challenge in fatigue detection lies in the limitations of Coarse-Grained Labels (CGL). These labels, often derived from subjective self-assessments like the Karolinska Sleepiness Scale, lack the temporal resolution to capture micro-variations occurring in seconds. Furthermore, they ignore the "intermediate states"—the transition period where a person is neither fully alert nor completely fatigued. Treating these states as two non-overlapping distributions leads to significant misclassification, especially for behavioral subtypes like squinting versus slow blinking.

The MDFG Framework

The MDFG framework addresses these gaps through a three-stage pipeline:

  1. Preprocessor & Trustworthy Extraction: The system extracts Eye Aspect Ratio (EAR) features and employs a "SmallLoss" mechanism. This identifies "trustworthy" samples by prioritizing those with lower loss during early training, effectively filtering noise from the initial coarse-grained data.
  2. Fine-Grained Label Generation:
  3. Categorization (FGC): Clustering and Gaussian Mixture Models (GMM) identify subtypes within alert and fatigued states, establishing "base classes."
    • Quantification (FGQ): To represent the fatigue spectrum, the framework synthesizes intermediate states by linearly combining Gaussian distributions from the alert and fatigued base classes. This creates a complete set of labels representing various fatigue levels (e.g., 0.1, 0.3, 0.5).
  4. Network Learning Stage: A meta-learning module is utilized to handle variability and class imbalance. By learning via pairwise comparisons (determining if one sample is more fatigued than another), the model gains a deeper understanding of underlying distribution shifts.
MDFG Framework Architecture: Overview of the Preprocessor, Fine-Grained Label Generation, and Network Learning stages.

Experimental Results

The framework was evaluated on two real-world datasets: reRLDD (284 alert, 194 fatigue instances) and reDROZY (183 alert, 322 fatigue instances). Unlike simulated data, these involve subjects experiencing genuine sleep deprivation.

MDFG demonstrated significant performance improvements over traditional binary methods. On average, accuracy increased by 10.0% on reRLDD and 12.1% on reDROZY. Ablation studies confirmed that both the fine-grained labeling and the meta-learning module contributed substantially to these gains.

Accuracy comparison: Performance of default, type-based, and level-based modeling on the reRLDD and reDROZY datasets.

Furthermore, the model exhibited high robustness. While competing state-of-the-art methods often showed drastic performance drops when switching between datasets, MDFG maintained a consistent F1 score with only a 6.1% variance.

Conclusion

MDFG successfully transitions fatigue detection from a binary "light switch" model to a nuanced "dimmer switch" model. By mathematically synthesizing transitional states and identifying behavioral subtypes, the framework achieves superior accuracy and robustness. Future work will explore integrating multimodal data, such as head posture and physiological signals (EEG/ECG), to further enhance detection capabilities across diverse environments.

Summary updated

BibTeX
@inproceedings{wang2025mdfg,
  author = {Mei Wang and Xiaojie Zhu and Ruimin Hu and Dongliang Zhu and Liang Liao and Mang Ye},
  title = {{MDFG: Multi-Dimensional Fine-Grained Modeling for Fatigue Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i12.33388},
}
Figure