Model Training Through Synthetic Data Generation: Investigating the Impact on Human Physical Fatigue

Arsalan Lambay, Phillip L. Morgan, Ying Liu, Ze JiAHFE 2024 · DOI

TL;DR

This paper explores using Generative Adversarial Networks (GANs) to solve the "data drought" in human physical fatigue research. By generating high-fidelity synthetic tabular data that mimics real-world sensor outputs (like heart rate and motion), the researchers demonstrate that we can train robust machine learning models to detect worker exhaustion without the high cost and privacy hurdles of massive real-world data collection.

What Problem Does This Paper Address?

In the era of Industry 4.0 and 5.0, collaborative robots (cobots) are supposed to make life easier for factory workers. However, physical fatigue remains a massive hurdle, leading to injuries and decreased productivity. To fix this, we need Machine Learning (ML) models that can "see" fatigue coming using wearable sensors.

The bottleneck? Data. Collecting high-quality, annotated physiological data from humans is expensive, time-consuming, and fraught with privacy concerns. Real-world datasets are often small, imbalanced, or lack the diversity needed to train a "universal" fatigue detector. The authors propose Synthetic Data Generation (SDG) as a scalable, affordable bridge to fill this gap.

Methods

The researchers focused on Generative Adversarial Networks (GANs)—a class of AI where two neural networks (a Generator and a Discriminator) "compete" against each other. The Generator tries to create fake data that looks real, while the Discriminator tries to spot the fakes.

The study compared three specific GAN architectures to see which could best replicate the nuances of human fatigue:

  1. Time-Series DGAN: Designed for sequential data.
  2. Tabular ACTGAN: An evolution of CTGAN designed to handle complex, multimodal tabular datasets.
  3. Tabular LSTM-GAN: Utilizes Long Short-Term Memory units to better capture temporal dependencies within the data.

To evaluate the "realness" of the synthetic data, they used Principal Component Analysis (PCA) to visualize data distributions and calculated Field Correlation Stability (FCS) and Field Distribution Stability (FDS) scores.

Datasets

The study utilized an open-source repository of authentic real-world data (previously detailed in Lambay et al., 2022). This dataset focused on:

  • Inertial Measurement Units (IMU): Capturing movement and posture.
  • Heart Rate Sensors: Monitoring physiological strain.

This real-world data served as the "ground truth" used to train the GANs and as a benchmark to validate the quality of the generated synthetic sets.

Key Insights and Ideas

  • LSTM is King for Fatigue: The Tabular LSTM-GAN emerged as the clear winner. Because fatigue is a physiological process that develops over time, the LSTM’s ability to remember sequences allowed it to generate synthetic data that much more closely mirrored the "flow" of real human exhaustion.
  • The "Quality Score" Gap: There was a significant performance gap between models. While ACTGAN is often praised for handling diverse data types, it struggled to replicate the specific distributional patterns of fatigue sensors compared to the LSTM-based approach.
  • Hyperparameter Tuning is Non-Negotiable: The authors found that "fine-tuning" the GAN (adjusting internal settings like learning rates and dropout layers) wasn't just a minor optimization—it was essential for the synthetic data to achieve "pronounced concordance" with real-world characteristics.

Key Results

The study provided a hierarchical ranking of the models based on how well their synthetic data matched the real training data:

  • Top Performer: Tabular LSTM-GAN. It achieved the highest quality scores, with a Field Distribution Stability of 88 and Field Correlation Stability of 66. PCA plots showed its synthetic data points almost perfectly overlapping with real data.
  • Runner Up: ACTGAN. While it managed multimodal data well, it showed "discernible disparity" in distribution patterns, scoring lower (FDS: 46, FCS: 42).
  • Lowest Performer: Time-Series dGAN. This model exhibited the least efficacy and a notable disparity from the authentic examples.

Limitations and Future Work

While the results are promising, the authors acknowledge several areas for growth:

  • Variable Diversity: The current study relied heavily on IMU and heart rate data. Future research needs to incorporate biomechanical gait analysis and subjective questionnaires to create a more holistic "fatigue profile."
  • Downstream Validation: While the synthetic data looks real statistically, the next step is to prove that an ML model trained only on this synthetic data performs just as well as one trained on real data when deployed on a factory floor.
  • Feature Selection: The authors suggest combining GANs with techniques like Recursive Feature Elimination (RFE) to identify which specific sensor readings are the "smoking guns" for detecting early-onset fatigue.

Summary updated

Abstract

Collaborative robots, or cobots, are one of the Industry 4.0 technologies that have and continue to change many industrial procedures. However, amid this technological advancement, the persisting physical strain on human workers remains a significant concern. Even with the advent of cobots aimed at alleviating burdensome tasks, certain physical jobs continue to induce fatigue in human workers. Addressing this challenge necessitates the development of robust solutions that combine technological innovation with human-centric considerations. One critical aspect in mitigating physical fatigue in human workers involves the application of Machine Learning (ML) models. These models heavily depend on data obtained from real-world situations that accurately represent the complexities of physical strain. However, this kind of data is frequently limited and costly to gather using sensors, which hinders the development of an effective ML model. This scarcity underscores the need for alternative approaches, with Synthetic Data Generation (SDG) emerging as a viable solution to this problem. The production of synthetic data offers a new approach to address the lack of relevant data needed to train machine learning algorithms. By employing techniques like Tabular Generative Adversarial Networks (GANs), synthetic datasets can be created, simulating realistic human physical fatigue detection features. Tabular GANs have, for example, been shown to be effective in creating synthetic data that closely resembles the statistical characteristics and patterns of real-world datasets. Furthermore, tabular GANs present a scalable and affordable response to the problem of data scarcity. The research reported here presents a novel approach centred on employing the Tabular GAN methodology to create synthetic datasets encompassing key features pertinent to the detection of human physical fatigue. The results of this study are expected to contribute substantially to creating robust solutions to alleviate physical strain and enhance human workers' overall well-being in industrial settings. The goal is to create datasets that accurately represent the complexities found in real-world scenarios where physical fatigue notably influences human performance. These synthetically generated datasets will serve as the foundation for training specialized ML models designed explicitly for detecting the development of human physical fatigue. The trained ML model will undergo rigorous testing and validation using a substantial repository of authentic real-world data. The model's accuracy and reliability in detecting human physical fatigue will be assessed through this evaluation process. The ultimate objective is to achieve a level of accuracy that demonstrates the model's proficiency in identifying and predicting the onset of physical fatigue in human workers within industrial settings. This research endeavours to bridge the gap between Industry 4.0 innovations and human well-being by leveraging synthetic data generation techniques to enhance the accuracy and efficiency of ML models in detecting human physical fatigue.
BibTeX
@inproceedings{lambay2024model,
  author = {Arsalan Lambay and Phillip L. Morgan and Ying Liu and Ze Ji},
  title = {{Model Training Through Synthetic Data Generation: Investigating the Impact on Human Physical Fatigue}},
  booktitle = {AHFE},
  year = {2024},
  doi = {10.54941/ahfe1005349},
}
Figure