Learning Breakpoint Threshold Policies for Storage Arbitrage Under Real Price Uncertainty

Energy storage operators decide every hour whether to buy electricity, sell it, or wait. The best-understood models of real hardware assume the operator knows every future price in advance, which is the "deterministic price assumption." Those models include conversion efficiency that changes with power level and fixed costs for switching a device on. The best-understood models of price uncertainty do the reverse: they treat the hardware as an idealized box with one constant efficiency and no switching cost. Each simplification is made to keep the math solvable, and each makes the resulting schedule look better than it will be in practice.

We propose a single model that keeps both the uncertainty and the hardware detail. It relies on one structural fact. Under perfect foresight, a storage device with piecewise-linear efficiency prefers to operate exactly at the corners of its efficiency curve, and it follows a threshold rule: charge or discharge only when the price crosses a level. We take that shape as the family of policies to search over. The policy's price thresholds are then fitted directly on historical price paths to maximize a mix of average profit and worst-case profit (Conditional Value-at-Risk, the average over the worst fraction of days). This objective is flat almost everywhere as a function of the thresholds and jumps wherever a threshold crosses an observed price. We optimize it by gradually removing random-noise smoothing.

Core Claim

The perfect-foresight structure found by Price arbitrage using variable-efficiency energy storage can serve as the policy class once prices are uncertain. That paper models efficiency with piecewise-affine curves and fixed on/off costs inside a mixed-integer program, and proves the optimum is a monotone threshold policy that sits at efficiency breakpoints. It leaves stochastic prices explicitly open.

We make the following claim. A policy that picks a breakpoint power level through price thresholds, indexed by state-of-charge band and hour of day, and that is trained directly on Conditional Value-at-Risk of daily profit over real price paths, should reach expected profit within a few percent of scenario-based stochastic programming on held-out years. It should reach better tail profit than robust and chance-constrained models that ignore efficiency structure. It should also decide in milliseconds.

The non-smooth optimization direction supplies the training method. It does not supply the headline result.

Relation to Prior Work

Three papers motivate the gap. Energy Storage Arbitrage Under Price Uncertainty: Market Risks and Opportunities builds robust and chance-constrained arbitrage models on NYISO prices and traces out risk–profit frontiers. It uses a constant round-trip efficiency and omits operating costs, so its frontiers overstate what real hardware earns. Benchmarking Flexible Electric Loads Scheduling Algorithms under Market Price Uncertainty shows that planning against average prices is unreliable. It also shows that two-stage stochastic programs cost about 74 seconds per decision, and that a simple heuristic comes close on cost. Both points push toward cheap, structured policies. Optimal Threshold Control for Energy Arbitrage with Degradable Battery Storage proves a two-threshold optimal policy under stochastic prices with convex degradation. That proof needs known price-transition probabilities and convex costs. Our setting has non-convex fixed costs and no known price model, and we optimize a risk measure rather than expected value.

Three other papers supply the method. Real-Time Optimisation for Online Learning in Auctions convolves a step-shaped revenue function with a kernel so that gradient ascent can work on it. We apply the same idea to threshold parameters. Variable Smoothing for Weakly Convex Composite Functions shows that shrinking the smoothing parameter on an epoch schedule gives $O(\epsilon^{-3})$ stationarity, compared with $O(\epsilon^{-4})$ for plain subgradient methods. We borrow the schedule. Our objective is not weakly convex, so we do not claim the guarantee carries over. Risk-Sensitive Submodular Optimization optimizes Conditional Value-at-Risk by alternating between the decision and an auxiliary tail-cutoff variable, with smoothed gradients. It shows that expected-value optimizers can collapse to zero tail value. We reuse the cutoff-variable trick. Our profit function is not submodular, so we lose its approximation guarantee.

One paper constrains the design. Differentiability and Regularization of Parametric Convex Value Functions in Stochastic Multistage Optimization smooths dynamic-programming value functions and gets fast gradients. It depends on convexity and discrete noise, and fixed on/off costs break convexity. That is why we search over policies rather than smooth a value function.

Research Program

The policy $\pi_\theta$ maps the current price, hour, and state-of-charge band to one of the efficiency breakpoint power levels, or to idle. It does so by comparing the price with thresholds $\theta$. Charging thresholds sit below discharging thresholds, which keeps the policy monotone as the perfect-foresight structure suggests. With 24 hours, a handful of state-of-charge bands, and three to five breakpoints per direction, $\theta$ has roughly 100–300 entries.

Let $R_d(\theta)$ denote the realized profit on historical day $d$ when the policy is simulated with the full piecewise efficiency and fixed costs. The training objective is

$$F(\theta)=(1-\lambda)\,\frac{1}{D}\sum_{d} R_d(\theta)+\lambda\,\max_{\tau}\Big\{\tau-\frac{1}{\alpha D}\sum_d \max(0,\tau-R_d(\theta))\Big\},$$

where $D$ is the number of training days, $\alpha$ is the tail fraction, $\lambda\in[0,1]$ trades average profit against tail profit, and $\tau$ is the auxiliary cutoff whose optimum is the $\alpha$-quantile of daily profit.

$F$ is piecewise constant in $\theta$, so we ascend its Gaussian smoothing

$$F_{\sigma}(\theta)=\mathbb{E}_{u\sim N(0,I)}\big[F(\theta+\sigma u)\big],$$

with the noise scale $\sigma$ shrinking by epoch. Gradients come from antithetic zeroth-order estimates, or from a soft-threshold relaxation of the simulator. We compare both.

We also test one extension. We add a cycle-depth degradation cost of the kind Degradation Aware Optimization for Energy Storage Systems argues is necessary to prevent near-zero-margin cycling. We then ask whether the learned thresholds widen as expected.

Baselines

The naive reference is a fixed-percentile rule: charge at full power below the 20th training-price percentile and discharge above the 80th. Hardware and prices are identical to ours. Beating it is necessary but not interesting.

The deterministic reference is Flamm et al.'s piecewise-affine mixed-integer program, re-solved hourly against a point price forecast in a rolling loop. It uses the same forecaster that the stochastic baselines receive. This isolates the value of modeling uncertainty at all.

We re-implement the robust and chance-constrained models of Energy Storage Arbitrage Under Price Uncertainty in two versions: with their constant efficiency, and with our piecewise hardware added. Both use their 2014–2021 calibration and 2022–2023 test split. They report a non-negative-profit-day ratio of 0.94 for polyhedral sets versus 0.57 for Chance-Normal at $\Gamma=0.8$. Their hardware is idealized, so these numbers are a reference point, not a target. Beating them means a higher non-negative-day ratio at equal or better expected profit on the same test days with the same hardware.

The strongest competitor is a two-stage stochastic mixed-integer program with piecewise efficiency, re-solved in a rolling horizon over sampled price scenarios, in the style of SO2 in B-FELSA. It gets the same scenario generator, a scenario count tuned on 2021, and a per-step time cap. In B-FELSA it was 60% cheaper than direct charging, but the perfect-information oracle was still about 85% better, and it took about 74 seconds per step. Beating it means matching its expected profit within 3%, while improving tail profit at $\alpha=0.1$ and cutting decision time by two orders of magnitude.

We also include dynamic programming on discretized state of charge, with a Markov price model fitted to training data. This is Petrik and Wu's approach extended to breakpoint actions, and it is the most dangerous structured baseline for a single device.

Two ablations isolate the proposal's key ingredients. The first lets thresholds select any power level on a fine grid instead of only breakpoints, which tests whether the structural restriction matters. The second sets $\lambda=0$, which tests whether training directly on tail risk matters. All learned methods get the same number of simulator calls for tuning.

Why not just…?

Why not just re-solve the deterministic hardware model every hour? Rolling re-optimization against a point forecast is cheap and uses the exact hardware. However, B-FELSA found that the deterministic model had the highest unmet demand, 1.63%, when planning for averages. That paper studies flexible loads rather than arbitrage, so the rolling baseline could still be competitive on expected profit here. Our prediction is that it loses mainly on the tail metric. If it matches us on both profit and tail, the uncertainty half of the claim fails.

Why not just run the scenario stochastic program with piecewise efficiency? This is the real threat on profit, and for one device it may win on average. Our case rests on two points. First, it optimizes expected value, and Risk-Sensitive Submodular Optimization shows expected-value optimizers can leave the tail unprotected. Second, it is slow. We report the full profit–tail–latency trade-off rather than claiming dominance.

Why not just do dynamic programming with a fitted price chain? For one device with one-dimensional state of charge, exact dynamic programming is cheap. So our advantage has to come from two places. First, the fitted chain may be misspecified in volatile years like 2022. Second, a whole-day Conditional Value-at-Risk does not decompose stage by stage without enlarging the state. If this baseline matches our tail profit out of sample, we would narrow the contribution to cases with several devices or several markets, where the state grows.

Why not just use derivative-free search such as random search or evolution strategies? Our estimator is itself a smoothed zeroth-order method, so the real question is whether the shrinking schedule and the cutoff variable help. We run a covariance-adapting evolution strategy and plain random search with equal simulator budgets. We report both, and credit the policy class rather than the optimizer if they tie.

Proposed Experiments

The main testbed uses NYISO NYC hourly prices with the 2014–2021 / 2022–2023 split. Storage is modeled as a 100 kW power-to-gas unit with the efficiency curve and fixed costs from Flamm et al., and as a lithium-ion battery with a three-segment efficiency curve. The perfect-information optimum on each test day gives a gap metric in B-FELSA's style.

We measure expected daily profit, Conditional Value-at-Risk at $\alpha\in\{0.05,0.1,0.2\}$, the non-negative-profit-day ratio, the gap to perfect information, and decision latency. Robustness checks retrain on ISO-NE prices with degradation added, where Petrik and Wu report that degradation-blind control loses money over five years. There is no reported result on Lacuna for piecewise-efficiency arbitrage under stochastic prices on these datasets, so every hardware-aware baseline number will be produced here.

The decisive checks are these. On 2022, the volatile test year, the learned breakpoint policy should beat both the rolling deterministic model and the constant-efficiency robust model on tail profit. It should stay within 3% of the scenario program's expected profit. The fine-grid ablation should do worse out of sample, which would show that the breakpoint restriction regularizes rather than merely speeds things up. The $\lambda=0$ ablation should lose tail profit, as Wilder's results predict. Failure of the first check refutes the proposal. Failure of the second or third means the gain comes from ordinary policy search rather than from the proposed ingredients.