Deep Generative Demand Learning for Newsvendor and Pricing

Shijin Gong, Huihang Liu, Xinyu Zhang

TL;DR

This paper introduces a novel framework for solving the "newsvendor problem"—the classic challenge of balancing stock levels—when demand is influenced by both contextual features and variable pricing. By using Conditional Deep Generative Models (cDGMs), the authors move beyond simple point predictions to learn the entire demand distribution, enabling joint optimization of pricing and inventory with strong theoretical guarantees and superior performance in complex, real-world scenarios.

What Problem Does This Paper Address?

The classic newsvendor problem asks: "How much should I stock to maximize profit given uncertain demand?" In modern retail, this is complicated by two factors:

  1. Contextual Features: Demand depends on external factors like weather, holidays, or demographics.
  2. Price Elasticity: Demand is not just uncertain; it is decision-dependent. Changing the price changes the demand distribution.

Traditional data-driven methods often struggle here. Simple models (like Sample Average Approximation) ignore features, while deterministic deep learning models (which predict mean demand) tend to systematically overestimate profits because they fail to capture the full "shape" of uncertainty. The authors target the Feature-based Newsvendor Pricing problem, where a manager must simultaneously decide the optimal price and the optimal stock level without making rigid assumptions about how demand behaves.

Methods

The core innovation is the use of Conditional Deep Generative Models (cDGMs), specifically conditional Variational Autoencoders (cVAEs) and conditional Generative Adversarial Networks (cGANs).

Instead of training a neural network to output a single number (expected demand), the authors train a Generator $G(x, p, \eta)$. This generator takes contextual features ($x$), a candidate price ($p$), and a random noise vector ($\eta$) to produce "synthetic" demand samples.

  • Algorithm 1 (Inventory Only): For a fixed price, the model generates thousands of demand samples. The optimal inventory is then simply the specific quantile of these samples (determined by the cost of overstocking vs. understocking).
  • Algorithm 2 (Joint Pricing & Inventory): The model discretizes the possible price range and runs Algorithm 1 in parallel across a price grid. It then selects the price-inventory pair that yields the highest estimated expected profit.

The authors provide a rigorous theoretical foundation, proving that as the amount of historical data and the number of generated samples increase, their decisions converge to the true mathematical optimum.

Datasets

The paper evaluates the approach using three distinct sources:

  1. Synthetic Simulations: Four different Data Generating Processes (DGPs) ranging from simple linear relationships to complex, non-linear interactions and heteroscedastic noise (where uncertainty changes based on the input).
  2. Textual Feature Simulation: A specialized setup where demand is influenced by unstructured text (e.g., product descriptions or reviews), processed via word embeddings.
  3. Real-World Case Study: The Kaggle Food Demand Forecasting dataset, which contains 450,730 records of meal deliveries across 77 fulfillment centers.

Key Insights and Ideas

  • The "Deterministic Trap": The authors prove (Proposition 1) that using a standard "mean-prediction" model will almost always overestimate profit. This is a result of Jensen's Inequality; because the profit function is concave, the profit of the average demand is higher than the average profit of uncertain demand.
  • Generative Models as Simulators: By learning the distribution rather than a point estimate, the cDGM acts as a "digital twin" of the market. You can ask it "what-if" questions about prices that have never been tested before.
  • Efficiency through Parallelism: Because neural networks are optimized for matrix operations, the model can evaluate hundreds of potential prices simultaneously, making the joint optimization computationally feasible for real-time use.
  • No Structural Assumptions: Unlike older "Residual-based Estimation" (RBE) methods, this approach doesn't assume demand is "Linear + Noise." It can capture complex, "decision-dependent" uncertainty where the variance of demand might explode at certain price points.

Key Results

  • Superior Profitability: In complex simulations (DGP 'd'), the cDGM achieved a profit difference from the "oracle" (perfect knowledge) of only 2.64, compared to 32.02 for standard SAA and 14.55 for Kernel methods.
  • Joint Optimization Success: In joint pricing-inventory tests, the cDGM consistently outperformed all baselines. In the most difficult continuous price simulation, it achieved a profit of 103.54, while the next best baseline (Kernel) trailed at 78.95.
  • Real-World Robustness: On the Kaggle dataset, the cDGM consistently achieved the highest profits across various cost scenarios (different purchase costs and salvage values) for multiple meal IDs, proving it generalizes well to noisy, real-world supply chain data.

Limitations and Future Work

  • Price Discretization: The current joint optimization relies on a discrete price grid. While they prove this is effective, it could be computationally expensive for extremely high-precision pricing.
  • Single-Period Focus: The paper focuses on the "single-period" newsvendor. Real-world retail often involves multi-period dynamics (backorders, perishable inventory over time).
  • Scalability to Many Products: The authors acknowledge that extending this to multidimensional problems (optimizing 1,000 products at once where prices affect each other) would lead to a "curse of dimensionality" in the optimization step.
  • Future Directions: The authors suggest applying this generative framework to broader Conditional Stochastic Optimization problems beyond just retail, such as energy grid management or financial portfolio balancing.

Summary updated

Abstract

We consider data-driven inventory and pricing decisions in the feature-based newsvendor problem, where demand is influenced by both price and contextual features and is modeled without any structural assumptions. The unknown demand distribution results in a challenging conditional stochastic optimization problem, further complicated by decision-dependent uncertainty and the integration of features. Inspired by recent advances in deep generative learning, we propose a novel approach leveraging conditional deep generative models (cDGMs) to address these challenges. cDGMs learn the demand distribution and generate probabilistic demand forecasts conditioned on price and features. This generative approach enables accurate profit estimation and supports the design of algorithms for two key objectives: (1) optimizing inventory for arbitrary prices, and (2) jointly determining optimal pricing and inventory levels. We provide theoretical guarantees for our approach, including the consistency of profit estimation and convergence of our decisions to the optimal solution. Extensive simulations-ranging from simple to complex scenarios, including one involving textual features-and a real-world case study demonstrate the effectiveness of our approach. Our method opens a new paradigm in management science and operations research, is adaptable to extensions of the newsvendor and pricing problems, and holds potential for solving other conditional stochastic optimization problems.
BibTeX
@misc{gong2024deep,
  author = {Shijin Gong and Huihang Liu and Xinyu Zhang},
  title = {{Deep Generative Demand Learning for Newsvendor and Pricing}},
  year = {2024},
  eprint = {2411.08631},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.08631},
}

Loading related content...

Figure