mRSC: Multi-dimensional Robust Synthetic Control

Muhummad Amjad, Vishal Misra, Devavrat Shah, Dennis ShenProc. ACM Meas. Anal. Comput. Syst 2019

Abstract

Evaluating the impact of a policy or intervention often requires estimating a counterfactual—what would have happened in the absence of that intervention. Synthetic Control (SC) methods achieve this by constructing a weighted combination of "donor" units. However, standard SC and its successor, Robust Synthetic Control (RSC), struggle when the pre-intervention period is short, leading to overfitting and poor inference. We present mRSC (Multi-dimensional Robust Synthetic Control), a generalization that incorporates multiple metrics to estimate the synthetic control. By treating additional metrics as a way to effectively lengthen the training period, mRSC provides a consistent estimator even with limited historical data. Our analysis shows that prediction error decays at a rate faster than RSC by a factor of $\sqrt{K}$, where $K$ is the number of metrics. We demonstrate the efficacy of mRSC through synthetic experiments and real-world case studies in retail and sports analytics.

Introduction

In observational studies where randomized control trials are impossible, Synthetic Control (SC) has become a gold-standard tool for causal inference. By creating a "virtual" version of a target unit from a donor pool, researchers can estimate the effect of a specific intervention. While Robust Synthetic Control (RSC) improved upon the original framework by handling missing data and noise, both methods share a critical vulnerability: the "short-history" problem. When the pre-intervention period is brief, the model lacks sufficient temporal data to learn the latent relationships between units, resulting in unreliable counterfactuals.

mRSC addresses this by leveraging the multi-dimensional nature of modern datasets. Instead of relying on a single metric over time, mRSC utilizes $K$ related metrics to provide a richer context for the latent factor model, effectively multiplying the available data points for training.

The mRSC Methodology

The core innovation of mRSC is the transformation of a 3D data tensor (Units × Time × Metrics) into a structure suitable for robust estimation.

1. Tensor Flattening and De-noising

Rather than analyzing metrics in isolation, mRSC "flattens" the donor tensor into a wide matrix by concatenating the data for all $K$ metrics side-by-side. To remove observation noise, the algorithm applies Singular Value Decomposition (SVD) to this flattened matrix. By using hard thresholding to retain only the most significant singular values, the model extracts the underlying latent factors that drive behavior across all metrics and units.

2. Weighted Least Squares

Once the donor matrix is de-noised, the target unit’s pre-intervention data is regressed against it to find the optimal weights ($\beta$). Because the de-noising step captures the shared latent structure, these weights are more robust to noise than those produced by standard SC.

3. Rank Preservation Diagnostic

To ensure that the included metrics are actually helpful, the authors propose a diagnostic test. If adding a new metric significantly increases the rank of the combined matrix, it suggests the metrics are driven by different underlying factors. In such cases, combining them might introduce bias rather than reducing variance.

Theoretical Insights

The paper bridges the gap between econometrics and machine learning by proving that if data follows a general Latent Variable Model, the target unit is naturally a linear combination of donors across all metrics simultaneously.

A key theoretical result is the speedup in convergence. The authors prove that the post-intervention Mean Squared Error (MSE) vanishes at a rate scaling with $1/\sqrt{KT_0}$, where $T_0$ is the length of the pre-intervention period. This confirms that adding more relevant metrics ($K$) directly improves the generalization of the synthetic control, compensating for a small $T_0$.

Empirical Validation

Retail Forecasting (Walmart Data)

Using weekly sales data across 45 stores and 81 departments, the authors tested mRSC’s ability to forecast sales when training data is sparse. In scenarios where the pre-intervention period was only 10% of the total timeline, mRSC outperformed vanilla RSC by 5–7x in terms of test prediction error. This demonstrates that departmental metrics can serve as powerful "time-multipliers" for store-level predictions.

Cricket Match Prediction

The authors applied mRSC to a dataset of over 4,700 international cricket matches. By using two metrics—runs scored and wickets lost—they predicted the final score of ongoing games.

  • Stein’s Paradox: Interestingly, the study found that using a global pool of all teams as donors was more accurate than restricting the pool to the specific team playing. This suggests that a larger, more diverse donor pool provides a better latent structure for the model.
  • Accuracy: The model achieved a median Mean Absolute Percentage Error (MAPE) of just 2.7% for 5-over-ahead forecasts, significantly outperforming models that ignored the multi-dimensional context of wickets lost.

Conclusion

mRSC provides a robust framework for causal inference and time-series forecasting in data-constrained environments. By leveraging multi-dimensional metrics, it overcomes the limitations of short historical records. While the method requires domain expertise to select relevant metrics, the provided diagnostic test offers a rigorous way to validate those choices. Future applications could extend this framework to complex engineering systems or high-frequency financial data where multiple related sensors or indicators are available.

Summary updated

Abstract

When evaluating the impact of a policy on a metric of interest, it may not be possible to conduct a randomized control trial. In settings where only observational data is available, Synthetic Control (SC) methods provide a popular data-driven approach to estimate a "synthetic" control by combining measurements of "similar" units (donors). Recently, Robust SC (RSC) was proposed as a generalization of SC to overcome the challenges of missing data high levels of noise, while removing the reliance on domain knowledge for selecting donors. However, SC, RSC, and their variants, suffer from poor estimation when the pre-intervention period is too short. As the main contribution, we propose a generalization of unidimensional RSC to multi-dimensional RSC, mRSC. Our proposed mechanism incorporates multiple metrics to estimate a synthetic control, thus overcoming the challenge of poor inference from limited pre-intervention data. We show that the mRSC algorithm with $K$ metrics leads to a consistent estimator of the synthetic control for the target unit under any metric. Our finite-sample analysis suggests that the prediction error decays to zero at a rate faster than the RSC algorithm by a factor of $K$ and $\sqrt{K}$ for the training and testing periods (pre- and post-intervention), respectively. Additionally, we provide a diagnostic test that evaluates the utility of including additional metrics. Moreover, we introduce a mechanism to validate the performance of mRSC: time series prediction. That is, we propose a method to predict the future evolution of a time series based on limited data when the notion of time is relative and not absolute, i.e., we have access to a donor pool that has undergone the desired future evolution. Finally, we conduct experimentation to establish the efficacy of mRSC on synthetic data and two real-world case studies (retail and Cricket).
BibTeX
@article{amjad2019mrsc,
  author = {Muhummad Amjad and Vishal Misra and Devavrat Shah and Dennis Shen},
  title = {{mRSC: Multi-dimensional Robust Synthetic Control}},
  journal = {Proc. ACM Meas. Anal. Comput. Syst.},
  year = {2019},
}
Figure