A Hybrid State Space-Attention Architecture for Forecasting the Trajectories of Battery Degradation

Shrey Patel, Bhargav Srinivas Rao, Samet AyhanarXiv 2026 · arXiv

TL;DR

DecayFormer is a compact hybrid model that predicts a battery's full health trajectory from just its first 100 cycles. By combining selective recurrence with state-queried attention and a physics-inspired "master curve" head, it achieves state-of-the-art accuracy across multiple chemistries using 94% fewer parameters than previous leading models.

What Problem Does This Paper Address?

Predicting how long a battery will last is a multi-month (or year) bottleneck in R&D. "Early-life forecasting" aims to take data from the first few days of cycling and predict the entire State-of-Health (SOH) trajectory.

The challenge is that battery degradation is non-linear. Most cells fade slowly for a long time before hitting a "knee"—a point of accelerated failure. Current state-of-the-art (SOTA) models have become increasingly bloated, often relying on Large Language Model (LLM) embeddings and extensive hyperparameter tuning for specific datasets. These "black box" approaches often struggle to generalize to new battery chemistries (like Sodium-ion or Zinc-ion) or unseen operating conditions where the "knee" might occur much earlier or later than in the training data.

Methods

The authors introduce DecayFormer, a model that prioritizes architectural efficiency and physical constraints over raw scale. Its design rests on three pillars:

  1. Hybrid State-Space Attention: The model uses a "Selective Recurrence" layer (inspired by Mamba) where each channel propagates a single scalar state. This is paired with a "State-Queried Attention" mechanism. Unlike standard Transformers where tokens query each other, here the recurrent state acts as the query. This allows the model to summarize long sequences efficiently while still being able to "look back" at specific early-cycle signatures.
  2. SOC-Interval Statistics: Instead of just looking at raw capacity, the model analyzes specific State-of-Charge (SOC) intervals. This captures subtle "signatures" in the voltage-current curves that signal internal degradation (like lithium inventory loss) long before they show up as capacity fade.
  3. The Master Curve Head: This is the model’s "physics-informed" secret sauce. Instead of predicting SOH directly, the head splits the task into two:
  4. Lifetime Scalar ($\ell$): A single number representing the "rate" of aging (how fast the clock ticks).
  5. Master Curve ($s$): A monotonically non-increasing profile representing the "shape" of degradation. By forcing the SOH to be non-increasing, the model avoids physically impossible predictions where a battery's health spontaneously improves.

Datasets

The model was evaluated on the BatteryLife benchmark, which is currently the most diverse public dataset for this task. It includes:

  • Li-ion: Standard Lithium-ion cells.
  • CALB: Industrial-grade Lithium-ion.
  • Na-ion: Emerging Sodium-ion chemistry.
  • Zn-ion: Zinc-ion coin cells.

The evaluation used 22 "aging-condition-exclusive" folds, meaning the model was tested on batteries operating under conditions (temperature, discharge rates) it never saw during training.

Key Insights and Ideas

  • Recurrence and Attention are Complementary: The authors show that recurrence is great at summarizing the "history" of a battery, while attention is necessary to retrieve specific early-life features that correlate with end-of-life behavior.
  • The "Shift" is the Hard Part: Through an error analysis, the authors found that most forecasting errors come from predicting when the battery will fail (the lifetime scalar), not the shape of how it fails. On Li-ion, correcting just the predicted lifetime scalar removed 56% of the total error.
  • Efficiency as a Feature: DecayFormer uses only 0.79M parameters, compared to the 12.9M–14.9M parameters of the previous SOTA, BatteryMFormer. This makes it viable for deployment on edge devices or battery management systems (BMS).

Key Results

DecayFormer achieved impressive Mean Absolute Percentage Error (MAPE) across diverse chemistries without any per-fold hyperparameter tuning:

  • Li-ion: 2.25% MAPE (Best in class).
  • Na-ion: 0.95% MAPE (Best in class).
  • CALB: 2.76% MAPE.
  • Zn-ion: 7.19% MAPE.

Notably, the model is extremely fast. It can forecast the full trajectory of a cell in 1.2 ms on a laptop GPU and 32 ms on a standard CPU, making it orders of magnitude faster than the physical testing it replaces.

Limitations and Future Work

  • The Zn-ion Anomaly: The model struggled with Zinc-ion batteries. Interestingly, Zn-ion cells often gain capacity in their early life before fading. Because DecayFormer’s master curve head is mathematically constrained to be "non-increasing," it literally cannot represent this early-life capacity boost.
  • Lifetime Estimation: Since the majority of the remaining error lies in the lifetime scalar ($\ell$), future work will likely focus on better ways to estimate this "shift factor" from early-cycle voltage signatures.
  • Broader Benchmarking: The authors suggest testing the architecture against other emerging frameworks like PhyMamba to further validate the hybrid state-space approach.

Summary updated

Abstract

Early-life battery degradation trajectory forecasting aims to predict a battery cell's full state-of-health (SOH) trajectory from its initial charge-discharge cycles, potentially reducing cell qualification and evaluation time from months to days. Despite recent advances, accurate forecasting remains challenging for emerging battery chemistries, data-scarce domains, and cells exhibiting degradation behaviors beyond those observed during training. Existing state-of-the-art approaches increasingly rely on model scaling, large language model-derived aging-condition embeddings, and extensive fold-specific hyperparameter tuning, raising concerns about computational efficiency, reproducibility, and generalization. This work introduces DecayFormer, a compact hybrid state-space and attention architecture designed for early-life degradation trajectory forecasting. DecayFormer combines cycle-level selective recurrence, using a single scalar state per channel, with an attention mechanism queried by the recurrent state. It further incorporates state-of-charge (SOC) interval statistics and a master curve head that jointly estimates a scalar lifetime and a monotonically non-increasing SOH profile parameterized by normalized lifetime. We evaluate DecayFormer on the BatteryLife benchmark using all 22 aging-condition-exclusive folds, with a single run per fold and one fixed configuration across all folds, without fold-specific tuning. DecayFormer achieves mean test MAPE values of 2.25% on Li-ion, 2.76% on CALB, 0.95% on Na-ion, and 7.19% on Zn-ion. These results demonstrate the potential of compact sequence models to deliver accurate early-life degradation forecasts across diverse battery chemistries and aging conditions, without relying on extensive model scaling or per-fold optimization.
BibTeX
@misc{patel2026hybrid,
  author = {Shrey Patel and Bhargav Srinivas Rao and Samet Ayhan},
  title = {{A Hybrid State Space-Attention Architecture for Forecasting the Trajectories of Battery Degradation}},
  year = {2026},
  eprint = {2611.19116},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19116},
}
Figure