Semi-unsupervised Learning of Human Activity using Deep Generative Models

Matthew Willetts, Aiden R. Doherty, Stephen J. Roberts, Chris C. HolmesarXiv 2018 · OpenReview

TL;DR

This paper introduces "semi-unsupervised learning," a challenging scenario where some data classes have few labels and others have none. They propose a novel Gaussian mixture deep generative model that outperforms existing methods on MNIST and demonstrates effective activity recognition from sensor data, paving the way for more robust machine learning in data-scarce domains.

What Problem Does This Paper Address?

In many real-world applications, especially in healthcare and medicine, labeled data is scarce and expensive to obtain, while unlabeled data is abundant. This paper tackles a specific, challenging regime called "semi-unsupervised learning." Here, the training data consists of some classes with very few labeled examples and other classes with no labeled examples at all. This is a step beyond traditional semi-supervised learning (where all classes have at least some labels) and transfer learning (which often assumes a clear source-target domain split). The authors highlight that this scenario is common in areas like activity recognition from sensor data, where rare events or conditions might only appear in unlabeled data. The core problem is how to effectively learn from this mixed data distribution to achieve good classification performance.

Methods

The authors propose a new deep generative model based on the variational autoencoder (VAE) framework, specifically building upon the M2 model by Kingma and Welling. The key innovation lies in how they model the latent space z. Instead of a single distribution for z across all classes (as in M2), their model uses a Gaussian mixture where the distribution of z is conditioned on the class label y. This allows different classes to have different "styles" or latent representations.

The model is trained using a combination of evidence lower bounds (ELBOs) for both labeled and unlabeled data. For unlabeled data, they use the standard ELBO for semi-supervised learning. For labeled data, they incorporate a term that encourages the model to correctly classify the labeled examples. The core idea is to jointly learn the class labels and discover underlying structure in the data, even for classes that were never seen with labels during training.

Datasets

  • MNIST: Used for evaluating the classification performance of the proposed model against the M2 model. The experiment involved semi-supervised learning on digits 0, 1, 2, 8, 9 (with 100 labels each) and treating digits 3, 4, 5, 6, 7 as entirely unsupervised.
  • CAPTURE-24: A human activity recognition dataset collected from wrist-worn accelerometers. This dataset was used to evaluate the model's ability to perform activity classification and discover structure in time-series data. For this experiment, a subset of participants' data was used as labeled data, while the rest was treated as unlabeled.

Key Insights and Ideas

  • The "Semi-Unsupervised" Gap: The paper clearly defines and addresses a practical data scarcity problem that is distinct from standard semi-supervised or zero-shot learning.
  • Conditional Latent Space: Conditioning the latent variable z on the class label y using a Gaussian mixture is crucial. This allows the model to learn class-specific latent representations, which is more flexible than a single latent distribution for all classes.
  • Handling Unseen Classes: The proposed model is designed to learn from data where some classes are entirely absent from the labeled set, a significant advantage for real-world scenarios with rare events.
  • Structure Discovery: Beyond classification, the model shows promise in discovering meaningful clusters or "structures" within unlabeled activity data, which can be interpreted as different activity states.

Key Results

  • MNIST Performance: The proposed Gaussian mixture deep generative model achieved a higher accuracy (0.94) on the MNIST semi-unsupervised classification task compared to the M2 model (0.82). The authors note that their model showed less confusion between unsupervised classes.
  • Activity Recognition: On the CAPTURE-24 dataset, the model demonstrated effective activity classification and structure discovery. Qualitative analysis showed that the discovered clusters corresponded to meaningful activity states like "standing" and "sitting." The Calinski-Harabasz score, a measure of cluster quality, rose significantly from approximately 160 for M2 to around 700 for their model.

Limitations and Future Work

  • Model Complexity: The authors acknowledge that using high-capacity neural networks in generative models can sometimes lead to issues like posterior collapse, where the model fails to learn informative latent representations for the class variable. Their Gaussian mixture approach aims to mitigate this.
  • Future Directions: The authors plan to extend their work by:
  • Applying the methods to larger datasets, such as the UK Biobank Activity Dataset.
  • Exploring more flexible and powerful parameterizations for the distributions within their model.
  • Extending the approach to handle more complex time-series analysis.

Summary updated

Abstract

We introduce 'semi-unsupervised learning', a problem regime related to transfer learning and zero-shot learning where, in the training data, some classes are sparsely labelled and others entirely unlabelled. Models able to learn from training data of this type are potentially of great use as many real-world datasets are like this. Here we demonstrate a new deep generative model for classification in this regime. Our model, a Gaussian mixture deep generative model, demonstrates superior semi-unsupervised classification performance on MNIST to model M2 from Kingma and Welling (2014). We apply the model to human accelerometer data, performing activity classification and structure discovery on windows of time series data.
BibTeX
@misc{willetts2018semi,
  author = {Matthew Willetts and Aiden R. Doherty and Stephen J. Roberts and Chris C. Holmes},
  title = {{Semi-unsupervised Learning of Human Activity using Deep Generative Models}},
  year = {2018},
  eprint = {1810.12176},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1810.12176},
}
Figure