Battery electrolytes last longer when the salt's anions, not the solvent molecules, crowd around each lithium or zinc ion. The group of molecules bound to an ion is called its solvation shell. When anions dominate that shell, they are the first thing to break down at the electrode, and they leave behind a dense, inorganic protective layer. Electrolyte chemists already design for this. What they cannot cheaply predict is how the shell behaves over time: how long a solvent molecule stays bound, how often an anion swaps in, and how quickly an ion sheds its shell before plating onto the electrode. Those rates come only from expensive molecular dynamics (MD) simulations, which step every atom forward under thermal noise, run one formulation at a time. Machine-learning tools for electrolytes predict equilibrium snapshots or single numbers. They do not produce the time-dependent, intrinsically random sequence of shell changes.
We propose to learn a stochastic, formulation-conditioned simulator of solvation-shell dynamics. Given a composition, it generates random trajectories of an ion's coordination state whose statistics, including lagged time correlations, match MD. The central technical claim is about how to train such a simulator so that its long rollouts stay faithful. Fitting one step at a time by likelihood lets errors compound over long rollouts. A deterministic simulator erases the noise that sets the kinetics. Training it adversarially against a discriminator is unstable. We instead adapt score-matching imitation learning, which compares noised state distributions without a discriminator, to match distributions of lagged state pairs. Lagged pairs carry the kinetic information that plain state distributions miss.
Core Claim
Treat the learned simulator as a policy that chooses the ion's next shell state, and treat MD trajectories as expert demonstrations. Under that view, the known trade-offs of imitation learning map directly onto surrogate modeling of molecular kinetics.
One-step maximum-likelihood fitting is behavioral cloning, which means supervised regression on individual state-action pairs. Its error grows quadratically with horizon, a point made across IQ-Learn, SMILING and the value-flow analysis. Here that error shows up as drift of the coordination-number distribution over long rollouts, and residence times distort as a result. A deterministic surrogate fails in the way Beyond Myopic World Models and SLAC describe for control: it rolls out the mean shell, a state the real system rarely occupies. Adversarial trajectory matching inherits the oscillation and seed sensitivity documented for adversarial imitation in BeTAIL and Multi-Agent GAIL.
SMILING supplies the fix. It replaces the discriminator with a least-squares denoising score regressor and proves that imitation error grows linearly in horizon. We will show that a lagged-pair version of that objective yields surrogates whose residence times, shell-population fractions and desolvation first-passage times match held-out MD, including on formulations the surrogate never saw. We will also show that it trains more reliably across random seeds than an adversarial alternative.
Research Program
The state is a compact description of one cation's first shell at each saved MD frame. It records the number of solvent, diluent and anion oxygen or nitrogen contacts; whether each anion is also bound to other cations; and a few continuous shell-geometry features. The shell-population classes follow directly from these counts: a solvent-separated ion pair (no anion contact), a contact ion pair (CIP, one anion bound to the ion), or an aggregate (AGG, anions bridging several cations). The conditioning vector describes the formulation: salt, solvent, diluent identities and molar ratios, and temperature. It is embedded with a pretrained molecular encoder in the spirit of Uni-ELF, which predicts equilibrium radial distribution functions from composition.
We differ from Uni-ELF on purpose. It regresses static structure, while we generate the random time series. Our target, the protocol-conditioned kinetics, is exactly what Open Electrolyte Databank argues static properties cannot capture.
Training runs in two stages. The first is a one-step likelihood warm start. The second is lagged-pair score matching. Let $x^{(k)}_t = (s_t, s_{t+k})$ be a pair of states separated by lag $k$. Let $p_E^{(k)}$ and $p_\theta^{(k)}$ be its distributions under MD and under the surrogate $\pi_\theta$. Let $\sigma$ index Gaussian noise levels. The surrogate minimizes
$$\mathcal{L}(\theta) = \sum_{k \in K} \mathbb{E}_{\sigma}\, \mathbb{E}_{\tilde{x} \sim p_{\theta,\sigma}^{(k)}} \left\| \nabla \log p_{E,\sigma}^{(k)}(\tilde{x}) - \nabla \log p_{\theta,\sigma}^{(k)}(\tilde{x}) \right\|^2 ,$$
where $K$ is a set of lags spanning picoseconds to nanoseconds and $p_{\cdot,\sigma}$ denotes a distribution convolved with noise of scale $\sigma$. Both scores are fit by denoising regression, never by a classifier. As in SMILING, the score difference becomes a per-step reward, and the surrogate is improved by soft actor-critic on its own rollouts. The "environment" is the surrogate's own state update, so the difficulty IQ-Learn flags is avoided by construction: an implicit reward that is only valid under the training dynamics does not arise here. With $K=\{0\}$, the objective reduces to SMILING's state matching. That reduction gives the key ablation. State matching alone can reproduce equilibrium shell populations while getting the rates wrong, and lagged pairs are what pin down the time correlations.
For data, we will run our own MD on two chemistries drawn from the file. The first is the dimethoxyethane / fluorinated-ether diluent / LiFSI family, where diluent dipoles reshape the shell, as in the dipole–dipole paper. We sweep diluent identity and ratio. The second is aqueous zinc sulfate with and without the ZnSiF6 additive from the zinc interphase paper. Each formulation gets a few independent replicas. Splits hold out whole diluent species and extrapolated ratios, echoing the solvent-split protocol that RILOOD showed exposes spurious shortcuts (MNSolv RMSE 0.7662 for CIGIN versus 0.6784 for RILOOD). A stretch goal adds an applied field near a model electrode to probe interfacial desolvation.
Learning Free Energy Pathways through RL of Adaptive Steered MD is the closest prior use of reinforcement learning inside molecular simulation. It uses PPO to steer a single molecule toward low-barrier paths with an engineered reward. We use no steering and no hand-designed reward. We learn a generative model of unbiased multi-body shell kinetics from demonstrations.
Baselines
The do-nothing reference is direct MD on each test formulation. It defines ground truth and the cost the surrogate must beat. We report wall-clock time per microsecond of equivalent shell trajectory, and a surrogate that is not substantially faster than MD has no reason to exist. We also include a static predictor that outputs only equilibrium CIP/AGG fractions from composition, trained Uni-ELF-style. It cannot produce rates and shows what equilibrium-only models capture.
The strongest domain competitor is a Markov state model (MSM), which counts transitions between discretized shell states at a fixed lag. We run it in two forms. The oracle form is built from MD on the test formulation itself, so it is an upper bound on what discretized kinetics can give. The deployable form is a regressor that maps formulation features to MSM transition matrices, trained on the same training formulations. Beating the deployable MSM means lower error on residence times and first-passage times on held-out formulations. The oracle MSM is a ceiling we aim to approach, not exceed.
The strongest ML competitor is the one-step maximum-likelihood stochastic surrogate, a neural transition model with the same architecture and warm start but no rollout-level training. Another baseline is a deterministic mean-predicting version of that surrogate.
The adversarial competitor replaces our score reward with a discriminator on lagged pairs, trained in the manner of DAC or GAIL. It gets the same state representation, rollout budget and hyperparameter search size as our method: an equal number of configurations, each run for five seeds. For robotics context only, SMILING reports reaching expert performance on Humanoid-Walk with 5,000 expert states versus 25,000 for DAC. IQ-Learn reports a Humanoid score of 5227.1 versus 644.5 for ValueDICE. We found no reported result on Lacuna for any learned surrogate of solvation-shell kinetics, so every number in our tables will be new.
The ablations are state-only matching ($K=\{0\}$), removing the likelihood warm start, and replacing RL with pathwise gradients through rollouts.
Why not just…?
Why not just build a Markov state model per formulation? For a formulation that has already been simulated, an MSM may be as good as anything we build, and we expect the oracle MSM to be very hard to beat. The case for our method is limited to formulations without MD. Even there, MSM-matrix regression is a real threat, because it is simple and physically interpretable. Our held-out-diluent experiment settles the question directly. If regressed MSMs match our surrogate on residence and first-passage times, we will report that the rollout-matching machinery is unnecessary for bulk kinetics. We would then keep the method only if it wins on continuous-geometry statistics or on the non-Markovian, field-driven interfacial setting, where fixed-lag discretization is weakest.
Why not just fit a stochastic one-step model by maximum likelihood? This is behavioral cloning. The imitation-learning literature cited above predicts quadratic error growth, and in our setting the prediction is concrete: coordination distributions should drift as rollout length increases. We test it by plotting shell-population and autocorrelation error against rollout length for both models. If the one-step model does not drift on these short-memory systems, that is a genuine negative result for our motivation, and we will say so.
Why not just use adversarial trajectory matching, which is standard for generative sequence models? It optimizes the same target, and a well-tuned discriminator might work. The evidence from BeTAIL, Multi-Agent GAIL and SMILING's Cheetah-Run instability suggests it will be seed-sensitive. We measure this directly under equal tuning budgets through the fraction of diverged runs, the across-seed spread in residence-time error, and oscillation in the reward signal during training.
Proposed Experiments
The primary metrics are four. We measure relative error in mean residence time of solvent and anion in the first shell. We measure the Wasserstein distance on CIP/AGG/solvent-separated fractions and on coordination-number autocorrelation functions. We measure error in the mean first-passage time to full desolvation. We measure speedup over MD. All four are reported on in-distribution and held-out-diluent formulations.
The first decisive check is the horizon curve. Our method should show roughly flat error as rollouts lengthen, while one-step likelihood fitting should degrade. SMILING's instance-dependent bound warns that high-variance dynamics loosen the guarantee, so we report how the gap scales with measured transition variance, zinc hydration versus the lithium ether systems, rather than assuming it vanishes.
The second check is the lag ablation. State-only matching should get populations right and rates wrong, and lagged pairs should fix the rates. If they do not, the core claim fails.
The third check is stability: fewer diverged seeds than the adversarial baseline at matched budget.
The fourth check is downstream relevance. We ask whether surrogate-predicted desolvation rates rank the dipole-tuned formulations the same way their reported rate capability does. That paper reports 76% capacity retention after 1,500 cycles at 10C for its best formulation. We will not claim more than a rank correlation from so few experimental points.
References
- IQ-Learn: Inverse soft-Q Learning for Imitation
- Diffusing States and Matching Scores: A New Framework for Imitation Learning
- Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanism
- Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction
- Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
- BeTAIL: Behavior Transformer Adversarial Imitation Learning From Human Racing Gameplay
- Multi-Agent Generative Adversarial Imitation Learning
- Uni-ELF: A Multi-Level Representation Learning Framework for Electrolyte Formulation Design
- Open Electrolyte Databank: Unlocking Molecular-Mixture Intelligence for Battery Electrolyte Discovery through a Standardized, Multimodal Foundation
- Solvation structure modulation via dipole–dipole interactions for high-rate lithium metal batteries exceeding 400 Wh kg−1
- Ultra‐Stable Aqueous Zinc Anodes: Enabling High‐Performance Zinc‐Ion Batteries via a ZnSiF6‐Derived Protective Interphase
- Relational Invariant Learning for Robust Solvation Free Energy Prediction
- Learning Free Energy Pathways through Reinforcement Learning of Adaptive Steered Molecular Dynamics