Abstract
This article proposes a Bayesian nonparametric method for forecasting, imputation, and clustering in sparsely observed, multivariate time series data. The method is appropriate for jointly modeling hundreds of time series with widely varying, non-stationary dynamics. Given a collection of $N$ time series, the Bayesian model first partitions them into independent clusters using a Chinese restaurant process prior. Within a cluster, all time series are modeled jointly using a novel “temporally-reweighted” extension of the Chinese restaurant process mixture. Markov chain Monte Carlo techniques are used to obtain samples from the posterior distribution, which are then used to form predictive inferences. We apply the technique to challenging forecasting and imputation tasks using seasonal flu data from the US Center for Disease Control and Prevention, demonstrating superior forecasting accuracy and competitive imputation accuracy as compared to multiple widely used baselines. We further show that the model discovers interpretable clusters in datasets with hundreds of time series, using macroeconomic data from the Gapminder Foundation.
Introduction
Real-world multivariate time series—such as regional disease outbreaks or global economic indicators—are notoriously difficult to model. They are often high-dimensional, non-stationary, and sparsely observed. Standard tools often fall short: parametric models like Vector Autoregression can become numerically unstable in high dimensions, while existing nonparametric models often assume data points are exchangeable, ignoring crucial temporal ordering. This research introduces the Temporally-Reweighted Chinese Restaurant Process (TRCRP) mixture, a domain-general framework designed to discover hidden patterns and provide robust forecasts in complex, non-stationary environments.
Methodology
The TRCRP mixture model evolves through two primary mechanisms: temporal reweighting and hierarchical structure learning. In a standard Chinese Restaurant Process (CRP), new data points join clusters based on existing occupancy. The TRCRP modifies this by introducing a cohesion function $G$, which reweights the probability of a time point $t$ joining cluster $k$ based on how similar the current local history is to historical patterns previously seen in that cluster.
To handle multiple time series, the model uses an "outer" CRP to partition series into groups that share the same temporal segmentation. Within a regime, the model assumes individual time series are independent given the regime (a Naive Bayes simplification), ensuring numerical stability in high-dimensional settings. Inference is conducted via Markov chain Monte Carlo (MCMC) to account for uncertainty in both clustering and forecasting.
Experimental Results
The model was evaluated on two primary datasets: weekly influenza-like-illness (ILI) rates from the CDC (augmented with Twitter and temperature data) and macroeconomic GDP data from the Gapminder Foundation.
Forecasting and Imputation
In the 2015 flu season, the Multivariate TRCRP consistently achieved the lowest Mean Absolute Error across all forecast horizons (1 to 10 weeks), significantly outperforming Facebook Prophet, Seasonal ARIMA, and HDP-HMM. For imputation, the model showed competitive accuracy with Amelia II, particularly when entire seasons of data were missing.
Interpretable Clustering
On the Gapminder data, the model automatically grouped 170 countries into meaningful clusters based on their 50-year GDP trajectories, distinguishing between stable growth in Western democracies and the economic volatility of post-Soviet states.
Discussion and Conclusion
Unlike models that hard-code seasonality, TRCRP is purely empirical, detecting regimes based on similarity to past data. This allows it to adapt to unusual events, such as mild flu seasons, while maintaining a "safe failure mode" by reverting to a broad prior in unprecedented scenarios. Future work will focus on automating the selection of the history window size $p$ and improving computational scalability for extremely large datasets.
Summary updated