Biosignal Generation and Latent Variable Analysis with Recurrent Generative Adversarial Networks

Shota Harada, Hideaki Hayashi, Seiichi Uchida

TL;DR

Researchers developed a conditional recurrent Generative Adversarial Network (GAN) to generate realistic, multi-class time-series biosignals like ECGs and EEGs using a single model. By applying Canonical Correlation Analysis (CCA) to the GAN's latent space, they cracked open the generative "black box," allowing users to intentionally control specific physical characteristics of the generated signals—such as amplitude and frequency—without needing domain-specific mathematical models.

What Problem Does This Paper Address?

Machine learning models for healthcare rely heavily on large datasets of biosignals (like heart or brain activity) to detect abnormalities such as myocardial infarctions or epileptic seizures. However, medical data is notoriously scarce and privacy-restricted, making data augmentation crucial.

While GANs are great at generating synthetic data, applying them to time-series biosignals presents two major challenges. First, previous GAN approaches required training a completely separate model for every single class of data (e.g., one for "normal" heartbeats, one for "abnormal"), which is computationally expensive. Second, GANs are notorious black boxes. They map random noise to realistic data, but the relationship between the input noise and the output characteristics is unclear. If a researcher wants to generate an ECG with a slightly higher frequency or a larger amplitude, standard GANs offer no way to control these specific features.

Methods

The authors propose a conditional recurrent GAN combined with a novel latent variable analysis step:

  • Conditional LSTM-GAN: Both the Generator and Discriminator are built using Long Short-Term Memory (LSTM) networks to handle time-series data. To solve the multi-model problem, the network is conditional—it takes a class label alongside the random noise vector, allowing a single model to generate multiple classes of biosignals.
  • Unrolled GAN Training: To prevent mode collapse (where the generator outputs the exact same signal repeatedly), they use an "unrolled" training updating rule, which stabilizes the adversarial learning process.
  • Latent Space Control via CCA: This is the paper's standout technical contribution. After training, the authors extract physical features (like maximum amplitude or mean frequency) from the generated signals. They then run Canonical Correlation Analysis (CCA) to find the linear correlations between the input random noise vectors and these extracted output features. By identifying the "canonical loadings" (the specific directions in the noise space that correlate with specific physical traits), they can intentionally tweak the input noise to control the generated signal's characteristics.

Datasets

The models were trained and evaluated on three real-world time-series datasets from the UEA & UCR Time Series Classification Repository:

  1. ECG200: 200 samples of single heartbeats (length 96), classified as normal or myocardial infarction.
  2. TwoLeadECG: 1,162 samples of heartbeats (length 82), classified by which of two ECG leads recorded them.
  3. Epileptic Seizure Recognition Data Set (EEG): 11,500 samples of brain activity (length 178), classified as normal or recorded during a seizure.

Key Insights and Ideas

  • Smooth Class Transitions: By feeding the Generator a class label that is a linear interpolation between two actual classes (e.g., halfway between "normal" and "abnormal"), the model generates a signal that smoothly blends the physical characteristics of both.
  • Linear Tools Work on Non-linear Latent Spaces: Even though a GAN is a highly non-linear deep neural network, applying a linear method like CCA to the input-output pairs is sufficient to find meaningful, controllable directions in the latent space.
  • Domain-Agnostic Control: Traditionally, controlling specific features of a synthetic biosignal required complex, hand-crafted mathematical models (like coupled differential equations) built by domain experts. This CCA-GAN approach achieves similar controllability purely through machine learning, making it easily adaptable to entirely different types of time-series data.

Key Results

  • High-Quality Generation: The quality of the generated data was evaluated quantitatively using Dynamic Time Warping (DTW) distance. The proposed conditional GAN achieved similarity scores comparable to or better than baseline augmentation methods (noise addition, interpolation, Hidden Markov Models, and independent GANs).
  • Parameter Efficiency: By using a single conditional model instead of independent models for each class, the parameter count is reduced by a factor of $C$ (the number of classes) without sacrificing generation quality.
  • Successful Feature Manipulation: The authors successfully demonstrated that by multiplying the canonical loadings by a constant and adding them to the input noise, they could predictably force the Generator to output signals with higher maximum values or increased mean frequencies.

Limitations and Future Work

The authors are transparent about several limitations:

  • LSTM Low-Pass Effect: LSTMs naturally behave somewhat like low-pass filters, meaning the proposed architecture struggles to accurately reproduce very high-frequency components in the biosignals.
  • Training Blindspots: Standard GAN loss functions do not clearly indicate learning progress, and the model requires substantial time to train.
  • Limits of Controllability: The CCA method can only control characteristics that actually vary within the training dataset. If a feature is entirely static across the real data, the GAN won't learn a latent representation for it, and CCA cannot extract a control vector for it.

Summary updated

Abstract

The effectiveness of biosignal generation and data augmentation with biosignal generative models based on generative adversarial networks (GANs), which are a type of deep learning technique, was demonstrated in our previous paper. GAN-based generative models only learn the projection between a random distribution as input data and the distribution of training data. Therefore, the relationship between input and generated data is unclear, and the characteristics of the data generated from this model cannot be controlled. This study proposes a method for generating time-series data based on GANs and explores their ability to generate biosignals with certain classes and characteristics. Moreover, in the proposed method, latent variables are analyzed using canonical correlation analysis (CCA) to represent the relationship between input and generated data as canonical loadings. Using these loadings, we can control the characteristics of the data generated by the proposed method. The influence of class labels on generated data is analyzed by feeding the data interpolated between two class labels into the generator of the proposed GANs. The CCA of the latent variables is shown to be an effective method of controlling the generated data characteristics. We are able to model the distribution of the time-series data without requiring domain-dependent knowledge using the proposed method. Furthermore, it is possible to control the characteristics of these data by analyzing the model trained using the proposed method. To the best of our knowledge, this work is the first to generate biosignals using GANs while controlling the characteristics of the generated data.
BibTeX
@article{harada2019biosignal,
  author = {Shota Harada and Hideaki Hayashi and Seiichi Uchida},
  title = {{Biosignal Generation and Latent Variable Analysis with Recurrent Generative Adversarial Networks}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2934928},
}
Figure