Data-Driven Contextual Optimization with Gaussian Mixtures: Flow-Based Generalization, Robust Models, and Multistage Extensions

Yong-Seok Yoon, Grani A. Hanasusanto, Yijie WangarXiv 2025 · DOI · arXiv

Introduction

In modern operations research and machine learning, decision-makers increasingly rely on "side information" or context—such as weather forecasts, macroeconomic indicators, or social media trends—to predict uncertain parameters like renewable energy generation or stock returns. This paradigm, known as contextual optimization, aims to learn the conditional distribution of uncertainty given the context to drive optimal decision-making.

However, existing methodologies often struggle with a fundamental trade-off. Parametric methods (e.g., linear regression) are data-efficient but fail to capture complex, multimodal distributions. Conversely, non-parametric methods (e.g., kernel regression) offer high flexibility but succumb to the "curse of dimensionality." As the number of context variables increases, the data required to maintain accuracy grows exponentially, rendering these models ineffective in high-dimensional settings. This paper proposes a novel framework using Gaussian Mixture Models (GMMs) to bridge this gap, offering a "Goldilocks" solution that is both expressive and scalable.

The GMM Framework for Contextual Optimization

The core innovation of this research is modeling the joint distribution of context and uncertain parameters as a Gaussian Mixture Model. This approach provides several distinct advantages:

  1. Analytical Tractability: If a joint distribution is modeled as a GMM, its conditional distribution is also a GMM with closed-form parameters. This allows for efficient, mathematically rigorous downstream optimization.
  2. Overcoming Dimensionality: The authors theoretically demonstrate that the approximation error of their empirical GMM depends only linearly on parameter estimation errors. Unlike non-parametric schemes, achieving high accuracy requires only polynomially many samples, making the model robust in high-dimensional context spaces.

Normalizing Flows (GMM-NF)

To handle real-world data that may not naturally follow a Gaussian distribution, the framework incorporates Normalizing Flows. An autoregressive normalizing flow is trained to map complex data into a latent space where it follows a GMM. The conditional distribution is computed in this tractable latent space and then mapped back to the original space, combining the non-linear feature extraction of neural networks with the rigorous probability math of GMMs.

Distributionally Robust Optimization (DRO)

To mitigate overfitting and handle model misspecification—specifically the difficulty of choosing the correct number of mixture components ($K$)—the authors employ a Wasserstein-based DRO formulation. This approach constructs an ambiguity set around the empirical distribution, ensuring the model remains robust even when the underlying data structure is uncertain.

Extending to Multistage Stochastic Optimization

Traditional approaches to multistage problems with Markovian uncertainty often rely on scenario trees that grow exponentially over time. By replacing standard kernel-based transition weights with GMM-based weights, the authors develop a data-driven dynamic programming scheme. This method’s sample complexity scales linearly with the time horizon, providing a powerful tool for long-horizon, high-dimensional problems like energy grid management.

Empirical Validation

The framework was evaluated across three diverse domains:

  • Synthetic Inventory Management (Newsvendor): In tests with up to 20 context variables, the GMM-NF model maintained low out-of-sample loss while non-parametric benchmarks collapsed due to the curse of dimensionality.
  • Portfolio Optimization: Using real-world S&P 500 data (2017–2023) and five macroeconomic indices, the GMM-NF model achieved higher Sharpe ratios and lower out-of-sample risks than six established benchmarks, including Optimal Transport models.
  • Multistage Wind Energy Optimization: Integrated into a Stochastic Dual Dynamic Programming (SDDP) framework, the GMM approach outperformed kernel-based methods in profit generation for wind energy assets in Ohio and North Carolina.

Conclusion

By leveraging the structural properties of Gaussian Mixture Models and the flexibility of Normalizing Flows, this framework provides a scalable and robust solution for contextual optimization. It effectively bypasses the curse of dimensionality while maintaining the expressiveness needed for complex, multimodal data. While training normalizing flows introduces some computational overhead, the resulting gains in decision quality and sample efficiency offer significant practical value for high-stakes decision-making under uncertainty.

Summary updated

Abstract

Contextual optimization enhances decision quality by leveraging side information to improve predictions of uncertain parameters. However, existing approaches face significant challenges when dealing with multimodal or mixtures of distributions. The inherent complexity of such structures often precludes an explicit functional relationship between the contextual information and the uncertain parameters, limiting the direct applicability of parametric models. Conversely, while non-parametric models offer greater representational flexibility, they are plagued by the "curse of dimensionality," leading to unsatisfactory performance in high-dimensional problems. To address these challenges, this paper proposes a novel contextual optimization framework based on Gaussian Mixture Models (GMMs). This model naturally bridges the gap between parametric and non-parametric approaches, inheriting the favorable sample complexity of parametric models while retaining the expressiveness of non-parametric schemes. By employing normalizing flows, we further relax the GM assumption and extend our framework to arbitrary distributions. Finally, inspired by the structural properties of GMMs, we design a novel GMM-based solution scheme for multistage stochastic optimization problems with Markovian uncertainty. This method exhibits significantly better sample complexity compared to traditional approaches, offering a powerful methodology for solving long-horizon, high-dimensional multistage problems. We demonstrate the effectiveness of our framework through extensive numerical experiments on a series of operations management problems. The results show that our proposed approach consistently outperforms state-of-the-art methods, underscoring its practical value for complex decision-making problems under uncertainty.
BibTeX
@misc{yoon2025data,
  author = {Yong-Seok Yoon and Grani A. Hanasusanto and Yijie Wang},
  title = {{Data-Driven Contextual Optimization with Gaussian Mixtures: Flow-Based Generalization, Robust Models, and Multistage Extensions}},
  year = {2025},
  eprint = {2509.14557},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.14557},
  howpublished = {arXiv},
}
Figure