A fast-charging controller for a battery has to be rewarded for preserving the cell's health. The damage it is supposed to avoid happens inside the cell, where it cannot be seen during operation. Examples are a thickening film on the cathode, lithium metal plating onto the anode, and cracked particles. A deployed controller only sees terminal voltage, current and temperature. Every reward used in practice is therefore a proxy computed from those signals. A controller that optimizes hard against a proxy will find wherever the proxy is wrong.
This proposal treats the reward as a short program to be synthesized and tested against the optimizer that will exploit it. A language model proposes the structure of a health-proxy program that reads only terminal-signal features. A symbolic search fills in its numeric constants under physically motivated constraints. Each candidate is scored on how much true degradation a charging policy causes after being optimized against it. A physics simulator reveals that degradation, but the deployed program never reads it.
The motivating failure is cathode electrolyte interphase (CEI) densification, meaning the film on the cathode surface becoming denser and choking ion entry. Study on Electrochemical Performance and Magnesium Storage Mechanism of Na3V2(PO4)3@C Cathode in Mg(TFSI)2/DME Electrolyte reports that over 1000 cycles the charge-transfer resistance fell from 179.3 Ω to 47.55 Ω, while the slow diffusion impedance worsened because of CEI densification. A lumped resistance-growth penalty would score that cell as getting healthier while it was choking.
Core Claim
Reward programs restricted to observable features can be made hard to exploit, and this robustness comes from two ingredients. The first is structure that separates fast and slow transport timescales. The second is a selection criterion that measures true damage under adversarial optimization, not how well the proxy fits passive data.
The timescale argument is simple. A surface film of thickness $L_f$ and effective ionic diffusivity $D_f$ has a characteristic transport time
$$\tau_f \approx \frac{L_f^2}{D_f},$$
which is the same quadratic scaling that governs bulk solid diffusion (The application of nanotechnology in the field of lithium-ion battery). Densification lowers $D_f$ and can raise $L_f$, so $\tau_f$ moves out of the fast window where charge-transfer resistance is read and into the slow, diffusion-like part of a current-pulse relaxation.
A proxy that integrates only fast polarization will under-penalize exactly this regime. A charging policy optimized against such a proxy will push into it. We predict that the gap between proxy-optimal and truly-optimal policies is largest when degradation is dominated by interfacial films, and smallest when it is ohmic.
Relation to Prior Work
Three works supply the method. Eureka: Human-Level Reward Design via Coding Large Language Models has a coding model write reward functions from environment source code and mutate them using training statistics. It beat human rewards on 83% of 29 robotics tasks with a 52% average normalized improvement. It assumes an unhackable fitness metric and generates unconstrained Python. We keep its evolutionary loop but replace both of those assumptions.
Learning to Infer Program Sketches has a neural model output a program skeleton with holes, which an enumerative search fills. It reached 90.0% on AlgoLisp, against 88.6% for neural generation alone and 7.3% for search alone. We reuse this division of labor: the language model writes structure, and search fits constants.
Programmatic Reward Design by Example fills human-written reward sketches from demonstrations, under symbolic constraints that block reward hacking. It works only in grid worlds and needs expert demonstrations. Neither condition holds for degrading cells, so we replace demonstrations with simulator-revealed damage and keep its constraint mechanism.
The battery-control work motivates the gap. Ageing-Aware Deep Reinforcement Learning for Adaptive Fast Charging of Li-ion Battery Considering Coupled Degradation mechanisms trains a continuous-action reinforcement learning agent (DDPG) on a PyBaMM electrochemical model with four coupled degradation mechanisms. Its reward is hand-weighted and uses internal simulator quantities that a real battery management system cannot read.
Safe Learning-Based Optimization of Model Predictive Control: Application to Battery Fast-Charging and Learning Model Predictive Control Parameters via Bayesian Optimization for Battery Fast Charging tune predictive controllers with Bayesian optimization on equivalent-circuit models, which are simple resistor-capacitor approximations of the cell. They enforce voltage and temperature limits but do not model interphase films.
The second of these is a constraining result. It shows the optimizer distorts model parameters, such as the internal resistance spline, to compensate for mismatch. This is exactly the behavior that makes any learned proxy suspect under optimization pressure.
Aging matrix visualizes complexity of battery aging across hundreds of cycling protocols shows that non-invasive differential-voltage fitting recovers mechanistic health quantities across 359 cells. It supplies our feature vocabulary and a possible real-data check.
Research Program
The simulator is a PyBaMM model in the Doyle-Fuller-Newman family, a standard physics-based porous-electrode model. We use the LG M50T parameterization and the four coupled degradation mechanisms of the DDPG study. The model includes SEI growth (the corresponding film on the anode) and the lithium-plating side reactions characterized in Study on Lithium-Ion Battery Degradation Caused by Side Reactions in Fast-Charging Process.
We add one ingredient of our own: a cathode film whose resistance enters as a slow, diffusion-limited element. Its thickness grows with cumulative high-potential exposure and its diffusivity falls with cycling. We will state plainly that this term is a surrogate motivated by the magnesium observation, not a validated CEI model.
We sample cell parameters, including film growth rates and diffusivities, to create training and held-out cell populations. The held-out cells include a population whose degradation is film-dominated.
The reward language has a small set of primitives computed from terminal signals:
- fast and slow components extracted from short rest periods after current pulses;
- shifts in differential-voltage peaks;
- coulombic efficiency;
- temperature rise;
- a cycle-count indicator for which degradation stage the cell is in.
Each primitive is a deterministic interpreter call, in the spirit of Binding Language Models in Symbolic Languages, which uses the language model only to write the program. The model is never queried inside the control loop.
The model proposes sketches that combine primitives with holes. Constraints of the PRDBE kind restrict the holes. For example, the penalty must be non-decreasing in slow-relaxation amplitude, and stage-dependent weights must stay ordered. Enumeration and local numeric search fill the holes.
The selection criterion is what departs from Eureka. For each filled candidate, we optimize a charging policy against it on the training cells, under the same charging-time incentive and safety limits as everywhere else. We then score the candidate by the true simulated capacity loss, lithium-inventory loss and film growth that this policy causes.
We use two policy classes. One is the two-parameter current-decay family used in the DDPG study, which is cheap to optimize by Bayesian optimization. The other is the DDPG lifecycle policy, used for final evaluation. The language model sees which latent mechanism the policy exploited, translated into text, and revises the sketch.
Baselines
The naive reference is a fixed multi-stage protocol tuned once for the nominal cell, alongside a hand-weighted observable proxy that penalizes capacity fade and total resistance growth. This is what a practitioner would deploy. Our method must beat it on true degradation at equal charging time. We expect it to be beaten mainly on film-dominated cells.
The strongest competitor on the hacking question is a learned neural proxy, a small network or Gaussian-process regressor from the same terminal-signal features to the same simulator-latent damage. It receives the same simulator trajectories and the same tuning budget. We also allow it adversarial data augmentation, retraining on trajectories from policies optimized against it, so it gets the same access to the optimizer that our program search gets.
The strongest existing synthesis method is Eureka-style unconstrained reward generation. It uses the same language model, the same number of candidates and iterations, and the same adversarial fitness. Its reported 83%-of-tasks and 52% results apply to robotics; there is no reported result found on Lacuna for any reward-synthesis method on battery charging.
From control, we run safe Bayesian-optimized predictive control on the same simulator with observable voltage and temperature constraints. Its reported figures, up to 410 seconds faster to 80% state of charge and a 15% versus 40% constraint-violation rate, were obtained on an equivalent-circuit Samsung cell, so they are context rather than a number to beat directly.
Two ablations isolate the key ingredients. The first removes the slow-relaxation primitives, testing timescale separation. The second scores candidates by passive regression fit to latent damage instead of adversarial fitness, testing the selection criterion.
Beating a baseline means a smaller proxy-hacking gap at matched charging time across held-out cell populations, with confidence intervals over random seeds. The gap is the true damage of the proxy-optimal policy minus that of the oracle policy optimized on true damage.
Why not just…?
Why not just optimize the charging protocol directly on simulated degradation and ship it? This is a real threat. If the simulator-optimal fixed protocol stays near-optimal across held-out cells, an observable reward adds nothing. We run exactly this baseline on held-out and film-dominated populations. If its excess damage is within noise of our closed-loop policy, we will report that and narrow the claim to cell-specific adaptation late in life. That is where the DDPG study finds the maximum safe charging speed drops sharply.
Why not just learn a neural proxy from observables? A regressor can fit passive data well. The question is whether it stays faithful once a policy optimizes against it. The parameter-distortion finding in the Bayesian-optimized predictive control work suggests that flexible fitted objects bend under optimization pressure. Our head-to-head under adversarial retraining settles it. If the neural proxy matches our programs, the contribution reduces to the adversarial selection criterion, and the ablation that swaps in passive fitness will show whether that criterion alone carries the benefit.
Why not just use conservative predictive control with voltage and temperature limits? Terminal-voltage limits do not observe film choking. The predictive-control papers show conservative settings are safe only by being slow. We measure where conservative control sits on the charging-time versus true-damage frontier, relative to our method.
Why not just use unconstrained Eureka? Given the same adversarial fitness, it might discover timescale separation on its own. Comparing it at equal budget tells us whether the language restriction and the constraints matter, or only the fitness does.
Decisive Checks
The central prediction is that the proxy-hacking gap of the resistance-only and no-slow-primitive variants grows with the share of degradation due to the cathode film, while the full program's gap stays flat. If no such interaction appears, the timescale argument is wrong for this setting.
A second check concerns the stage indicator. Synthesized programs should rediscover stage dependence, with plating weight rising late in life, matching the three-stage lifecycle reported in the DDPG study. If they do not, the stage primitive is unnecessary.
If the aging-matrix dataset is accessible, a third check applies. We would ask whether programs fitted only in simulation, evaluated on early-cycle terminal data, rank its protocols by lifetime, which spans 44 to 994 equivalent full cycles. A Spearman rank correlation reliably above that of the resistance-only proxy would be the first evidence that the method transfers beyond its simulator. If the data is not accessible, the claim stays a simulation result, and the paper will say so.
References
- Study on Electrochemical Performance and Magnesium Storage Mechanism of Na3V2(PO4)3@C Cathode in Mg(TFSI)2/DME Electrolyte
- The application of nanotechnology in the field of lithium-ion battery
- Eureka: Human-Level Reward Design via Coding Large Language Models
- Learning to Infer Program Sketches
- Programmatic Reward Design by Example
- Ageing-Aware Deep Reinforcement Learning for Adaptive Fast Charging of Li-ion Battery Considering Coupled Degradation mechanisms
- Safe Learning-Based Optimization of Model Predictive Control: Application to Battery Fast-Charging
- Learning Model Predictive Control Parameters via Bayesian Optimization for Battery Fast Charging
- Aging matrix visualizes complexity of battery aging across hundreds of cycling protocols
- Study on Lithium-Ion Battery Degradation Caused by Side Reactions in Fast-Charging Process
- Binding Language Models in Symbolic Languages