Generating Synthetic Datasets with Deep Learning Models for Human Physical Fatigue Analysis

Arsalan Lambay, Ying Liu, Phillip L. Morgan, Ze JiMachines 2025 · DOI

TL;DR

This paper tackles the challenge of collecting sufficient, high-quality data for training machine learning models to detect human physical fatigue, a crucial aspect of Industry 5.0. They propose a novel approach using conditional Generative Adversarial Networks (cGANs) to generate realistic synthetic datasets that mimic real-world fatigue data, enabling more robust and cost-effective ML model development.

What Problem Does This Paper Address?

The increasing integration of collaborative robots in Industry 5.0 necessitates a focus on human-centric workplace design. A key challenge in this domain is accurately detecting and mitigating human physical fatigue, which can lead to reduced productivity and safety concerns. While machine learning (ML) models show promise for fatigue detection, their effectiveness hinges on large, high-quality datasets. However, collecting such data through sensors is often expensive, time-consuming, and can raise privacy concerns. This creates a data scarcity problem, hindering the development and deployment of advanced ML solutions for fatigue analysis. This paper aims to bridge this gap by generating synthetic data that can effectively train ML models for fatigue detection.

Methods

The core of this research lies in employing deep learning-based Synthetic Data Generation (SDG) techniques. Specifically, the authors utilize a conditional Generative Adversarial Network (cGAN) framework. This framework consists of two main components:

  • Generator: This model is responsible for creating synthetic data. It takes a noise vector and a conditional vector as input, which guides the generation process to produce data that mimics specific characteristics of real-world fatigue data.
  • Discriminator: This model acts as a critic, evaluating the authenticity of the generated data by trying to distinguish it from real data.

The cGAN is trained adversarially, where the generator learns to produce increasingly realistic data to fool the discriminator, and the discriminator improves its ability to detect fakes. The authors also mention exploring a tabular LSTM GAN as an alternative architecture.

Key methodological steps include:

  • Data Preprocessing: Cleaning, filtering, synchronizing, and normalizing sensor data.
  • Feature Extraction: Deriving meaningful features from raw sensor data (EMG, IMU) in both time and frequency domains.
  • Feature Selection and Dimension Reduction: Employing techniques like correlation analysis, wrapper methods (SVM, logistic regression), and embedded methods (LASSO, random forest) to select the most informative features, reducing computational complexity.
  • Model Training: Training the cGAN models using both real and synthetic data.
  • Evaluation: Quantitatively and qualitatively assessing the generated synthetic data's similarity to real data using PCA, density plots, and distribution stability analysis. The performance of ML models trained on synthetic data is then compared to models trained on real data.

Datasets

The study utilizes real-world sensor data collected from participants performing six different physical activities designed to mimic industrial exertion scenarios. These activities involve varying degrees of shoulder and hand movements at different flexion levels. The data modalities include:

  • Electromyography (EMG): Measuring muscle electrical activity.
  • Inertial Measurement Units (IMUs): Capturing motion and orientation.
  • Borg Rating of Perceived Exertion: A subjective measure of exertion.

The paper does not introduce a new benchmark dataset but rather uses existing data for training and evaluating their SDG approach.

Key Insights and Ideas

  • Synthetic Data as a Viable Alternative: The research strongly suggests that synthetic data generated by cGANs can effectively replicate the statistical properties and distributions of real-world human physical fatigue data, making it a valuable substitute for training ML models, especially when real data is scarce or sensitive.
  • Handling Data Heterogeneity: The proposed cGAN approach is adept at handling the complex and imbalanced nature of human fatigue data, which often includes tabular, categorical, and time-series components.
  • Improved ML Model Performance: Training ML models on synthetic data, as demonstrated in this study, can lead to improved precision, recall, and F1-scores compared to training solely on real data, particularly for robust classifiers like random forest and gradient boosting.
  • Importance of Feature Engineering: The study highlights the critical role of careful feature extraction and selection in preparing data for both SDG and subsequent ML model training, ensuring that the generated data captures essential fatigue-related characteristics.

Key Results

The study demonstrates that the generated synthetic datasets closely approximate the original datasets across various tasks and features, as evidenced by:

  • PCA Plots: Showing a similar spread and clustering of data points between original and synthetic datasets, with synthetic data exhibiting a slightly broader variance.
  • Density Plots: Revealing that the synthetic data distributions closely mirror the original data distributions for key features, with minor deviations.
  • Box Plots (Field Distribution Stability): Indicating similar central tendencies and spreads between synthetic and original data, with some differences in outliers.

Crucially, when ML models (specifically Random Forest and Gradient Boosting) were trained on the synthetic data and tested on real data, they showed improved performance compared to models trained solely on real data. For instance, the Random Forest classifier achieved an accuracy of 0.92 when trained on synthetic data and tested on real data, outperforming models trained solely on real data in some metrics. The paper reports high F1-scores for the top-performing classifiers, with Random Forest achieving an F1-score of 0.92 for both fatigued and non-fatigued states when trained on synthetic data.

Limitations and Future Work

The authors acknowledge several limitations:

  • Mode Collapse: A potential issue in GANs where the generator produces limited variations of data.
  • Data Label Dependency: The reliance on accurate labels in the original data for effective synthetic data generation.
  • Lack of Real-World Variability: The generated synthetic data may not fully capture the complete spectrum of real-world variability.

Future research directions include:

  • Exploring more advanced deep learning techniques for data augmentation and feature selection.
  • Investigating state-of-the-art approaches to address mode collapse and data label dependency.
  • Further experimentation with hyperparameters to enhance classifier performance.
  • Expanding investigations to include a wider range of variables, industrial scenarios, and more powerful classifiers to assess the generalizability and versatility of SDG techniques.

Summary updated

Abstract

There has been a growth of collaborative robots in Industry 5.0 due to the research in automation involving human-centric workplace design. It has had a substantial impact on industrial processes; however, physical exertion in human workers is still an issue, requiring solutions that combine technological innovation with human-centric development. By analysing real-world data, machine learning (ML) models can detect physical fatigue. However, sensor-based data collection is frequently used, which is often expensive and constrained. To overcome this gap, synthetic data generation (SDG) uses methods such as tabular generative adversarial networks (GANs) to produce statistically realistic datasets that improve machine learning model training while providing scalability and cost-effectiveness. This study presents an innovative approach utilising conditional GAN with auxiliary conditioning to generate synthetic datasets with essential features for detecting human physical fatigue in industrial scenarios. This approach allows us to enhance the SDG process by effectively handling the heterogeneous and imbalanced nature of human fatigue data, which includes tabular, categorical, and time-series data points. These generated datasets will be used to train specialised ML models, such as ensemble models, to learn from the original dataset from the extracted feature and then identify signs of physical fatigue. The trained ML model will undergo rigorous testing using authentic, real-world data to evaluate its sensitivity and specificity in recognising how closely generated data match with actual human physical fatigue within industrial settings. This research aims to provide researchers with an innovative method to tackle data-driven ML challenges of data scarcity and further enhance ML technology’s efficiency through training on SD. This study not only provides an approach to create complex realistic datasets but also helps in bridging the gap of Industry 5.0 data challenges for the purpose of innovations and worker well-being by improving detection capabilities.
BibTeX
@inproceedings{lambay2025generating,
  author = {Arsalan Lambay and Ying Liu and Phillip L. Morgan and Ze Ji},
  title = {{Generating Synthetic Datasets with Deep Learning Models for Human Physical Fatigue Analysis}},
  booktitle = {Machines},
  year = {2025},
  doi = {10.3390/machines13030235},
}
Figure