When an advertising platform bids in a real-time auction, it needs more than a guess of the clearing price. It needs the whole distribution of possible prices, called the "bid landscape," so it can trade off the chance of winning against the amount it overpays. Two facts make this hard. First, prices pooled across traffic are strongly multimodal and spiky. Second, the data are censored: when the platform loses, it only learns that the price was above its own bid. The best neural answer to multimodality, Scalable Bid Landscape Forecasting in Real-time Bidding, puts a mixture of Gaussians at the output of one shared network. That makes every single prediction multimodal, and it breaks the fast one-dimensional search that production bid shading relies on.
We propose to test a different explanation. Much of the multimodality may come from pooling many distinct market regimes, such as cities, ad slots, or hours, and not from any one auction's own uncertainty. If so, it should be modeled in how inputs are routed rather than in the output density. We build a sparse mixture-of-experts forecaster, meaning a network with several parallel sub-networks ("experts") and a small gating model that sends each auction request to one of them. Each expert outputs a single log-normal price distribution. The forecast is therefore multimodal across traffic but unimodal for any given request, which keeps the surplus-maximizing bid findable by the golden section search used in An Efficient Deep Distribution Network for Bid Shading in First-Price Auctions. The second ingredient is specific to censored data. Experts that receive mostly lost auctions get almost no information about where prices actually fall, so we balance experts by how many uncensored observations they see, not by how many requests they see.
Core Claim
We make two claims. The first is that a top-1 routed mixture of log-normal experts, trained with the censored likelihood, will match an output-mixture network (MCNet) on held-out log-likelihood. It will also beat MCNet on first-price surplus, because the bid can be optimized exactly and cheaply. The second is that under heavy censoring, ordinary load balancing lets experts specialize on traffic the bidder rarely wins, where their price distributions are poorly determined. Balancing on uncensored counts fixes this.
The first claim is falsifiable, and either outcome teaches something. If MCNet keeps a clear log-likelihood advantage even against well-routed experts, then the multimodality is genuinely within-context. In that case unimodal-per-request shading is the wrong design, whatever the backbone.
Concretely, for request features $x$, top-1 gate $g(x) \in \{1,\dots,K\}$, and expert-specific location and scale networks $\mu_k, \sigma_k$, the model is
$$p(z \mid x) = \mathrm{LogNormal}\big(z;\ \mu_{g(x)}(x),\ \sigma_{g(x)}(x)\big),$$
where $z$ is the clearing price. The model is trained on the censored log-likelihood used by MCNet: log-density for won auctions, and log-probability that $z$ exceeds the submitted bid $b$ for lost ones. The gate is non-differentiable, so we train it with a softmax relaxation during early training and straight top-1 selection afterward.
For balancing, we reuse the bias trick of Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts. That paper adds a per-expert bias to the routing scores to choose experts, without adding a loss term. It updated the bias using raw token load and cut the maximum load violation from 0.72 to 0.04 in language modeling. We keep the mechanism but change what is counted. The bias for expert $k$ is updated as
$$\beta_k \leftarrow \beta_k + \eta\,\mathrm{sign}\big(\bar{w} - w_k\big),$$
where $w_k$ is the number of won (uncensored) auctions routed to expert $k$ in the batch, $\bar{w}$ is its mean over experts, and $\eta$ is a step size.
Research Program
The work has three parts.
The model. The architecture follows the shared-plus-routed pattern. Scene-wise Adaptive Network for Dynamic Cold-start Scenes Optimization in CTR Prediction used a shared expert group plus adaptive experts for heterogeneous recommendation scenes. It reported 0.1741 ms per-sample latency, which shows such models fit real-time bidding budgets. It predicted clicks, not censored prices. We borrow its structure: a shared trunk for market-wide price patterns and routed experts for regime-specific shifts.
Per-request expert outputs. Following Let the Experts Speak: Improving Survival Prediction & Calibration via Mixture-of-Experts Heads, each expert outputs a distribution specific to the request rather than a fixed prototype. That paper found fixed prototypes hurt calibration in censored clinical survival analysis. We take its lesson and move it from time-to-event hazards to price landscapes, with a latency constraint and a downstream bidding objective it did not consider.
The analysis. The central measurement is a decomposition. On held-out data we compare four models:
- a single log-normal network;
- MCNet's output mixture;
- routed log-normal experts;
- routed experts that each output a small mixture.
Their log-likelihood gaps split the modeling gain into a between-regime part (captured by routing) and a within-regime part (captured only by output mixtures). We report this split separately for each campaign, and across expert counts $K$ from 2 to 64. The expert-count sweep checks the known sensitivity of survival mixture-of-experts models to a mismatch between the number of experts and the number of true clusters.
Baselines
Naive reference. A single dense log-normal network trained with censored likelihood and optimized by golden section search. This is the Deep Distribution Network design. Its reported gains, 9.65% offline and 14.3% online surplus lift, are on proprietary Verizon Media data and cannot be reproduced. We therefore re-implement it on our data. It is the natural reference because our model reduces to it when $K=1$.
Strongest density model. MCNet, the neural network that outputs Gaussian mixture parameters under the censored loss. It directly represents the alternative hypothesis of within-context multimodality. Its paper reports relative improvements only: over 30% ANLP (average negative log probability of the true price) versus censored regression, and more than 10% versus survival trees. No absolute iPinYou ANLP for MCNet was found on Lacuna, so we re-run it.
Strongest reported absolute likelihood. Deep Landscape Forecasting for Real-time Bidding Advertising (DLF), a recurrent network over discretized price steps. It reports ANLP 4.774 and C-index 0.911 on overall iPinYou, against 5.148 and 0.858 for the Survival Tree Model of Functional Bid Landscape Forecasting for Display Advertising. Its reported inference time is 22 ms per sample.
Strongest surplus method. MINMAX from Simultaneous Optimization of Bid Shading and Internal Auction for Demand-Side Platforms, which optimizes surplus end to end. It reports a 7.42% surplus lift on iPinYou. It assumes uncensored logs, so we run it both on full logs, which is unfair in its favor, and on won-only logs.
Fairness conditions. All neural models get the same features and the same number of active parameters per request. Each gets the same hyperparameter search budget of 30 trials on a validation split, the same number of seeds, and the same censoring simulation.
Win criteria. Beating MCNet means equal or lower ANLP together with higher simulated first-price surplus at lower latency. Beating DLF means ANLP within its 4.774 at under a millisecond of latency. Beating MINMAX means higher surplus in the censored setting.
Ablations. We isolate the key ingredients three ways. Raw-load bias balancing versus uncensored-count balancing tests the censoring-aware idea. A classical auxiliary balancing loss versus bias balancing tests whether the interference gradients reported for language models also appear under censored likelihood. Learned routing versus hard routing on campaign and region identifiers tests whether learning the routing matters at all.
Why not just…?
Why not just use MCNet with a wider trunk? This is the main threat. An output mixture can represent everything our model can, and more. The decomposition experiment settles it directly. If routed log-normal experts leave a large likelihood gap to MCNet at matched active parameters, we conclude the multimodality is within-context and report that as the result. Even then, MCNet's multimodal density makes the surplus curve multimodal, so its bids require grid search rather than golden section search. Surplus per millisecond therefore remains a fair axis on which it can lose.
Why not just train one model per campaign or region, or feed segment identifiers to a dense net? On iPinYou's nine campaigns, routing by known metadata may capture most between-regime structure. Functional Bid Landscape Forecasting for Display Advertising found large differences across cities, which supports this worry. We include both hard routing on metadata and identifier embeddings as baselines. If learned routing does not beat them, the honest conclusion is that metadata segmentation plus censored log-normal heads is enough. Learned routing would then be justified only on high-feature-count data such as YOYI.
Why not just use survival trees or DLF? Survival trees partition categorical features naturally, but MCNet's authors report that trees need aggressive feature trimming at industrial scale, and DLF already beats them on iPinYou. DLF is the likelihood leader, but its 22 ms per-sample inference and its discretized output are poor fits for per-request bid optimization. We compare against it on likelihood and on latency, not on likelihood alone.
Proposed Experiments
Data and censoring. We use iPinYou (Real-Time Bidding Benchmarking with iPinYou Dataset) as the main benchmark and YOYI as the large-scale check. Censoring is simulated by replaying bidding policies over logged winning prices, so the true price is known even for "lost" rows. We vary the policy to produce win rates from about 50% down to under 10%, the regime where information-starved experts should appear.
Metrics. We report ANLP and C-index on all rows and on the censored subset. Calibration is measured as the gap between predicted and empirical win probability at each bid decile. Simulated first-price surplus is computed as value minus bid when the bid exceeds the market price, using one fixed value proxy for every method. We also report per-request latency and per-expert counts of uncensored observations.
Decisive checks. There are three:
- the likelihood decomposition between routed log-normal experts and MCNet;
- whether uncensored-count balancing reduces the ANLP degradation from win rate 50% to 10% relative to raw-load balancing, with gains concentrated on experts that previously received few won auctions;
- whether the routed model's surplus beats MCNet and MINMAX in the censored setting while staying under 1 ms.
A null result on the first check with a positive result on the second would still give a clear finding: censoring, not multimodality, is what sparse routing must be designed around.
References
- Scalable Bid Landscape Forecasting in Real-Time Bidding
- An Efficient Deep Distribution Network for Bid Shading in First-Price Auctions
- Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
- Scene-wise Adaptive Network for Dynamic Cold-start Scenes Optimization in CTR Prediction
- Let the Experts Speak: Improving Survival Prediction & Calibration via Mixture-of-Experts Heads
- Deep Landscape Forecasting for Real-time Bidding Advertising
- Functional Bid Landscape Forecasting for Display Advertising
- Simultaneous Optimization of Bid Shading and Internal Auction for Demand-Side Platforms
- Real-Time Bidding Benchmarking with iPinYou Dataset