Introduction
Developing intelligent in-cabin monitoring systems—capable of detecting driver distraction, fatigue, or emotional state—is a cornerstone of modern automotive safety. However, the development of these systems is hindered by a "data drought." Collecting large-scale, high-quality facial data within vehicles is prohibitively expensive, time-consuming, and raises significant privacy concerns regarding biometric information.
This research investigates the viability of using synthetic data (AI-generated images) as a solution to these hurdles. The study explores how state-of-the-art vision foundation models handle the transition from synthetic training environments to real-world inference across multiple tasks simultaneously.
Methodology
The researchers developed a Multi-Task Learning (MTL) framework designed to predict three key attributes: Gaze Plane, Age, and Facial Expression.
Architectural Comparison
The study compared two prominent architectures:
- ResNet-18: A classic convolutional neural network (CNN) valued for its efficiency and inductive bias toward local features.
- ViT-b_16 (Vision Transformer): A modern transformer-based model that treats images as sequences of patches, offering high performance on large-scale datasets.
Adaptation and Preprocessing
To adapt these ImageNet-pre-trained models for the automotive domain, three strategies were evaluated: Linear Probing (LP), Prefix Tuning (PT), and Full Fine-Tuning (FFT). Additionally, the researchers found that focusing the model's attention on the driver's face via automated cropping significantly improved performance compared to using full-cabin images.
Datasets and Training
The models were trained on three distinct synthetic datasets:
- SynthA: High volume (10,000 images) but low diversity.
- SynthB: Moderate volume (~3,000 images) with high demographic diversity (race, age, accessories).
- SynthC: Smallest volume (~1,900 images) captured in diverse non-vehicle environments.
A combined dataset of 14,919 samples was also used. For real-world validation, the UTKFace dataset served as an Out-of-Distribution (OOD) test set to measure the "sim-to-real" transferability.
Key Insights
The ResNet Advantage
Contrary to expectations, the older ResNet-18 architecture consistently outperformed the modern Vision Transformer. The authors suggest a "complexity mismatch": the high capacity of ViTs may lead them to overfit the specific artifacts of synthetic data, whereas the simpler ResNet architecture generalizes better to the underlying facial features.
Diversity Over Volume
While SynthA provided high training accuracy, it failed to generalize to real-world data (under 10% accuracy). In contrast, SynthC—the smallest and least "automotive" dataset—offered the best generalization. This indicates that environmental and demographic diversity in synthetic data is far more critical than raw image count.
Results and Conclusion
The optimal configuration was identified as a ResNet model using Full Fine-Tuning (FFT) on the Combined Dataset. On face-only validation sets, the model achieved high accuracies: 93.1% for Gaze, 95.5% for Age, and 86.0% for Facial Expression.
When tested on the real-world UTKFace dataset, the model achieved an average accuracy of approximately 50%. While this highlights a persistent "sim-to-real" gap, it confirms that models can indeed learn transferable human features from purely synthetic pixels. Future work will focus on closing this gap through advanced data augmentation and model compression for edge deployment.
Summary updated