TimeCaps: Capturing Time Series Data with Capsule Networks

Hirunima Jayasekara, Vinoj Jayasundara, Jathushan Rajasegaran, Sandaru Jayasekara, Suranga Seneviratne, Ranga RodrigoarXiv 2019 · OpenReview

TL;DR

This paper introduces TimeCaps, a novel capsule network architecture designed to capture temporal relationships in 1D time series data. By creating specialized capsule modules for temporal and channel dimensions, TimeCaps achieves state-of-the-art or competitive results in ECG beat classification and audio command recognition, while also enabling efficient signal reconstruction from a low-dimensional latent representation.

What Problem Does This Paper Address?

Traditional signal processing methods for time series data often require domain expertise for parameter selection or manual feature engineering, limiting their adaptability. While deep learning approaches like CNNs and RNNs have shown promise, they can lose crucial temporal information during pooling or struggle with limited/imbalanced datasets. Capsule Networks, successful in image tasks for preserving spatial relationships, were not inherently designed for 1D temporal data. This paper aims to bridge this gap by adapting capsule networks to effectively capture temporal dependencies in 1D signals, enabling robust classification, characterization, and reconstruction without extensive prior knowledge.

Methods

The core innovation is the TimeCaps architecture, which extends capsule networks to 1D time series. It comprises two parallel modules:

  • TimeCaps Cell A: Focuses on local temporal information by predicting future feature maps within a single frame. It uses convolutions to generate "votes" for subsequent capsules, which are then dynamically routed.
  • TimeCaps Cell B: Captures broader temporal relationships by predicting future and past values of signal segments. It segments the input and uses dynamic routing to learn these temporal dependencies.

These two cells are concatenated and fed into a classification layer. A key aspect is the dynamic routing mechanism, which allows capsules to learn part-whole relationships by agreeing on predictions. Crucially, TimeCaps also incorporates a decoder network that can reconstruct the original signal from a very low-dimensional latent representation (as low as 4 dimensions per signal), a significant improvement over existing methods. The paper also explores signal perturbation on the learned instantiation parameters to analyze their impact on reconstruction.

Datasets

  • MIT-BIH Arrhythmia Dataset: Used for Electrocardiogram (ECG) beat classification. The dataset contains 16 classes, but the authors used 13 classes due to low sample counts in the remaining ones.
  • Google Speech Commands Dataset: Used for audio command recognition, containing 30 classes of short audio commands.

Key Insights and Ideas

  • Capsule Networks are Adaptable to Temporal Data: The paper demonstrates that the core principles of capsule networks, particularly dynamic routing and learning instantiation parameters, can be effectively applied to 1D time series data to capture temporal relationships.
  • Dual Temporal Feature Extraction: Using two parallel capsule modules (Cell A for local temporal, Cell B for broader temporal) allows TimeCaps to learn complementary temporal features, leading to richer signal representations.
  • Low-Dimensional Reconstruction: TimeCaps can reconstruct 1D signals from extremely low-dimensional latent representations, significantly reducing computational complexity and storage requirements compared to existing methods.
  • Data-Driven Parameterization: The learned instantiation parameters provide a way to characterize and manipulate signals without relying on predefined models, offering flexibility for various signal processing tasks.
  • Robustness to Imbalanced Data: TimeCaps shows promise in handling datasets with imbalanced classes, a common challenge in real-world signal processing applications.

Key Results

  • ECG Classification: Achieved 96.96% accuracy on classifying 13 ECG beat categories from the MIT-BIH dataset, surpassing state-of-the-art results for this specific task.
  • Audio Classification: Achieved on-par classification results with state-of-the-art methods on the Google Speech Commands Dataset (30 classes).
  • Reconstruction: Demonstrated successful reconstruction of ECG signals from a latent representation as low as 4 dimensions per signal, with Mean Squared Error (MSE) as low as 0.004.
  • Signal Perturbation: Showed that specific instantiation parameters have a significant and predictable impact on the reconstructed signal's morphology, enabling data-driven signal analysis.

Limitations and Future Work

  • Computational Complexity: The paper acknowledges that capsule networks, in general, can be computationally intensive, especially with a large number of classes or high-dimensional input data (like raw audio).
  • Audio Input Challenges: For raw audio, the high sampling rate leads to a large number of capsules, potentially degrading the effectiveness of dynamic routing. The authors propose pooling layers to mitigate this but note that joint training of the encoder and decoder for raw audio is challenging.
  • Future Work:
    • Synthesizing ECG data samples to improve accuracy for rare beat types.
  • Exploring the addition of controlled noise to instantiation parameters for data augmentation or analysis.
  • Further investigating the impact of different numbers of instantiation parameters on reconstruction quality.

Summary updated

Abstract

Capsule networks excel in understanding spatial relationships in 2D data for vision related tasks. Even though they are not designed to capture 1D temporal relationships, with TimeCaps we demonstrate that given the ability, capsule networks excel in understanding temporal relationships. To this end, we generate capsules along the temporal and channel dimensions creating two temporal feature detectors which learn contrasting relationships. TimeCaps surpasses the state-of-the-art results by achieving 96.21% accuracy on identifying 13 Electrocardiogram (ECG) signal beat categories, while achieving on-par results on identifying 30 classes of short audio commands. Further, the instantiation parameters inherently learnt by the capsule networks allow us to completely parameterize 1D signals which opens various possibilities in signal processing.
BibTeX
@misc{jayasekara2019timecaps,
  author = {Hirunima Jayasekara and Vinoj Jayasundara and Jathushan Rajasegaran and Sandaru Jayasekara and Suranga Seneviratne and Ranga Rodrigo},
  title = {{TimeCaps: Capturing Time Series Data with Capsule Networks}},
  year = {2019},
  eprint = {1911.11800},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.11800},
}
Figure