Non-Parametric and Regularized Dynamical Wasserstein Barycenters for Time-Series Analysis

Kevin C. Cheng, Shuchin Aeron, Michael C. Hughes, Eric L. Miller

Introduction

Many real-world systems do not exhibit abrupt jumps between states but instead transition smoothly. For instance, a wearable sensor tracking a runner captures a clear "standing" state and a "running" state, but also a complex, transitional period in between. Traditional models like Hidden Markov Models (HMMs) treat these as discrete jumps, while standard state-space models often use "convex combinations" (linear mixtures) to handle transitions. However, linear mixtures often lead to "ghosting"—a distribution that suggests a person is simultaneously standing and running, rather than gradually accelerating.

This paper introduces a non-parametric framework for modeling such time series using Dynamical Wasserstein Barycenters (DWB). By leveraging optimal transport to "morph" one distribution into another, the authors capture complex, multi-modal data distributions with high fidelity.

Comparison of pure-state distributions, latent state transitions, and the resulting mixture models. Note how the Wasserstein Barycenter (d) avoids the 'ghosting' effect seen in the Convex Combination (c).

Methodology

The researchers propose a Non-Parametric and Regularized DWB approach to overcome the limitations of previous parametric models (e.g., Gaussian assumptions) and address identifiability issues. Key technical components include:

  • Quantile-Based Modeling: For univariate data, the Wasserstein-2 distance is equivalent to the $L_2$ distance between quantile functions. This allows complex distributions to be represented as discrete "quantile vectors," transforming a difficult distribution-matching problem into a manageable linear least-squares problem.
  • Dual Regularization: To solve the "inverse-scaling" problem—where multiple combinations of weights and distributions produce the same output—the authors introduce two penalties: Temporal Smoothness (ensuring latent weights change gradually) and Distribution Anchoring (preventing pure-state distributions from drifting too far from the global average).
  • Cyclic Descent Optimization: A block-coordinate descent algorithm alternates between updating the pure-state shapes and the time-varying weights.

Key Insights: The Window Size Tradeoff

A critical finding is the "U-shaped" error curve encountered when estimating distributions from a sliding window. If the window is too small, statistical noise dominates the estimate. Conversely, if the window is too large, it "blurs" the fast dynamics of the transition, losing temporal resolution. The authors demonstrate that the optimal window size is intrinsically linked to the speed of the system's transitions.

Window approximation error as a function of window size $n$ for various transition speeds $T$. The U-shaped curves demonstrate the tradeoff between statistical noise and dynamical blurring.

Experimental Results

The model was evaluated on simulated data, a proprietary "Beep Test" dataset, and the Microsoft Research (MSR) Human Activity Dataset. Key results include:

  • Quantitative Superiority: On the MSR dataset, the non-parametric model achieved a data-fit error ($e_y$) of 1.32, significantly outperforming the 3.76 achieved by Gaussian-based DWB models.
  • Qualitative Accuracy: The non-parametric approach correctly identified multi-modal peaks in exercise data (e.g., distinct phases within a single repetition) that Gaussian models typically smooth over.
  • Robustness: The introduced regularization successfully recovered ground-truth latent trajectories in simulated experiments where unregularized models failed due to mathematical ambiguity.

Conclusion and Future Work

While the current implementation is optimized for univariate time series, it provides a robust foundation for capturing non-Gaussian dynamics in human activity and other complex systems. Future work aims to explore Sinkhorn divergences to accelerate computation and investigate analytical methods for determining optimal window sizes based on signal properties. The transition from parametric to non-parametric DWB represents a significant step toward more expressive and identifiable time-series models.

Summary updated

BibTeX
@article{cheng2023non,
  author = {Kevin C. Cheng and Shuchin Aeron and Michael C. Hughes and Eric L. Miller},
  title = {{Non-Parametric and Regularized Dynamical Wasserstein Barycenters for Time-Series Analysis}},
  journal = {IEEE Trans. Signal Process.},
  year = {2023},
  url = {https://openreview.net/forum?id=tSsszhfBJR},
}
Figure