Diffeomorphic Temporal Alignment Nets

Ron Shapira Weber, Matan Eyal, Nicki Skafte Detlefsen, Oren Shriki, Oren FreifeldNeurIPS 2019 · OpenReview

Time-series data—from heartbeats (ECG) to motion sensors—often suffers from nonlinear time warping. Even if two signals represent the same underlying process, they might be "stretched" or "compressed" differently over time. This misalignment makes statistical analysis difficult; for example, taking the average of ten misaligned heartbeats results in a blurry, uninformative signal rather than a clear ECG rhythm.

Traditional solutions like Dynamic Time Warping (DTW) or "congealing" algorithms have two major flaws. First, they are computationally complex, scaling poorly with the number of signals. Second, they lack generalization. They solve the alignment for a specific batch of data, meaning if a new signal arrives, the entire optimization process must be re-run.

To address this, researchers introduced Diffeomorphic Temporal Alignment Nets (DTAN), a deep learning framework that moves time-series alignment from a per-batch optimization problem to a learning problem. DTAN learns to automatically align misaligned time-series data using smooth, invertible transformations, allowing for near-instantaneous alignment of new signals in a single forward pass.

DTAN successfully aligns both training and unseen test signals, drastically reducing variance compared to misaligned inputs.

The DTAN Architecture

DTAN is built on several key components designed to ensure fast, reliable, and physically plausible alignments:

  • Temporal Transformer Nets (TTN): Inspired by Spatial Transformer Networks used in computer vision, DTAN uses a "localization network" (a Convolutional Neural Network) to analyze an input signal and predict the parameters of a temporal warp.
  • CPAB Warps: To ensure the warping is "well-behaved," the authors use Continuous Piecewise-Affine Based (CPAB) transformations. These are diffeomorphic, meaning they are differentiable and invertible. This is crucial for maintaining the integrity of the signal, ensuring time is never folded backward.
  • Unsupervised and Semi-supervised Learning: In the single-class case, the model is entirely unsupervised, learning to warp signals to minimize the variance of the entire batch. In the multi-class case, it is semi-supervised: it uses class labels during training to minimize within-class variance, but at test time, it can align a signal without knowing its label.
  • Recurrent DTAN (R-DTAN): The authors also propose a recurrent version that applies multiple warps sequentially. This increases the "expressiveness" of the alignment without increasing the number of trainable parameters.

Key Insights

The success of DTAN relies on a few core conceptual shifts. Traditional alignment is transductive—it only cares about the data currently in front of it. DTAN is inductive; it learns the underlying rule of how signals in a specific domain tend to be misaligned.

Furthermore, by restricting the warps to be diffeomorphic, the model avoids "cheating" (e.g., squashing the entire signal into a single point to artificially zero out the variance). The authors also use a specific CPA covariance matrix to penalize "jerky" or unrealistic warps, forcing the model to find smooth, physically plausible alignments. Remarkably, a single DTAN can learn to align multiple different classes of signals, working seamlessly on new signals regardless of their category.

Performance and Results

The model was evaluated on a mix of synthetic data and the UCR Time-Series Classification Archive, a massive benchmark containing 84 different real-world datasets.

  • Generalization: DTAN successfully aligned test data it had never seen before, a feat impossible for traditional methods like DBA (DTW Barycenter Averaging) or SoftDTW.
  • Speed: DTAN is incredibly fast at test time. Aligning a batch of 10,000 signals (length 512) took only 0.66 seconds on a standard GPU.
  • Accuracy: Using a Nearest Centroid Classifier (NCC), DTAN-aligned data outperformed Euclidean baselines on 93% of the UCR datasets and beat DTW-based methods in the majority of cases.
  • Preprocessing Boost: When used as a front-end for a standard CNN classifier (DTAN-CNN), it improved classification accuracy on 87% of the datasets compared to using a CNN alone.
Scatter plots demonstrating DTAN's superior test accuracy across multiple datasets compared to Euclidean, DBA, SoftDTW, and standard CNN baselines.

Limitations and Future Work

While highly effective, the current implementation of DTAN focuses on fixed-length univariate signals. Although the authors suggest using Recurrent Neural Networks (RNNs) to handle variable-length data, this was not the primary focus of their experiments. Additionally, unlike DTW, which can be used on any two signals immediately out of the box, DTAN requires a representative training set to learn the warping patterns of a specific domain.

Despite these limitations, DTAN represents a significant step forward in time-series analysis, making complex temporal alignment scalable, generalizable, and applicable to real-time data streams.

Summary updated

Abstract

Time-series analysis is confounded by nonlinear time warping of the data. Traditional methods for joint alignment do not generalize: after aligning a given signal ensemble, they lack a mechanism, that does not require solving a new optimization problem, to align previously-unseen signals. In the multi-class case, they must also first classify the test data before aligning it. Here we propose the Diffeomorphic Temporal alignment Net (DTAN), a learning-based method for time-series joint alignment. Via flexible temporal transformer layers, DTAN learns and applies an input-dependent nonlinear time warping to its input signal. Once learned, DTAN easily aligns previously-unseen signals by its inexpensive forward pass. In a single-class case, the method is unsupervised: the ground-truth alignments are unknown. In the multi-class case, it is semi-supervised in the sense that class labels (but not the ground-truth alignments) are used during learning; in test time, however, the class labels are unknown. As we show, DTAN not only outperforms existing joint-alignment methods in aligning training data but also generalizes well to test data. Our code is available at https://github.com/BGU-CS-VIL/dtan.
BibTeX
@inproceedings{weber2019diffeomorphic,
  author = {Ron Shapira Weber and Matan Eyal and Nicki Skafte Detlefsen and Oren Shriki and Oren Freifeld},
  title = {{Diffeomorphic Temporal Alignment Nets}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=8qZYcesGii},
}
Figure