Dynamic Fractal Mamba: A Neural Renormalization Group Flow for Scale-Invariant Sequence Modeling

Shenglei Fang, Xianfang Sun, You ZhouICML 2026 · OpenReview

Introduction: The "Scale-Locked" Problem

Most modern sequence models—including Transformers and recent State Space Models (SSMs) like Mamba—are fundamentally "scale-locked." Trained on fixed sequence lengths or image resolutions, they struggle to generalize when presented with substantially longer horizons or higher resolutions during inference.

While hierarchical architectures like U-Nets or Swin Transformers attempt to capture multi-scale information, they typically rely on disjoint, scale-specific parameters. This lack of cross-scale consistency means the model fails to learn the underlying "physics" of the data, instead memorizing different approximations for different zoom levels. To achieve true scale generalization, models require recursive parameter sharing: the principle that the same underlying dynamics should apply whether analyzing microscopic details or macroscopic trends.

Methodology: Physics-Inspired Sequence Modeling

To address this, researchers introduce Dynamic Fractal Mamba (DF-Mamba), an architecture that operationalizes the concept of "Renormalization Group (RG) flow" from statistical physics into deep learning.

DF-Mamba relies on four core mechanisms:

  1. Recursive RG-Flow Architecture: Rather than using a flat stack of blocks, DF-Mamba employs a recursive operator. It processes the input at one scale, coarse-grains the representation, and then recursively calls itself to process the coarser version.
  2. Exponential Time-Step Scaling: To capture long-range dependencies, the discretization step ($\Delta$) of the SSM is scaled exponentially with the recursion depth ($2^k$). This effectively "dilates" time as the model moves to coarser representations, allowing it to see further without increasing computational cost.
  3. Learned Coarse-Graining: A lightweight MLP acts as a "block-spin" transformation to aggregate adjacent tokens. Crucially, this encoder is shared across all levels, enforcing a scale-invariant rule for information compression.
  4. Physics-Informed Constraints: Recursive training is stabilized using two auxiliary losses:
  5. Reconstruction Loss ($\mathcal{L}_{recon}$): Ensures the coarse-graining process retains vital information.
  6. Consistency Loss ($\mathcal{L}_{consist}$): Forces the upsampled global view to match the local view, enforcing the fixed-point property of scale invariance.

Despite its recursive nature, DF-Mamba maintains an overall computational complexity of $O(L)$. Because the sequence length halves at each step, the total computational work follows a geometric series that converges to roughly twice the cost of a single-scale model.

Key Insights: The Fractal Nature of Data

DF-Mamba operates on the assumption that many scientific and natural datasets are statistically self-similar, or fractal. By mimicking RG flow, the model aligns its internal processing with this inherent structure.

By reusing the same weights across scales, the model is forced to learn a "universal operator" rather than overfitting to a specific resolution. This parameter sharing acts as a powerful inductive bias, enabling the model's most striking capability: zero-shot scale extrapolation. A model trained exclusively on small local crops (e.g., 128x128) can be directly applied to full-resolution images (512x512) and actually improve in accuracy, as it can finally process the global structures it was inherently designed to handle.

Empirical Results

DF-Mamba was evaluated across three distinct domains requiring multi-scale reasoning, consistently outperforming baselines while remaining highly parameter-efficient:

  • Computational Pathology: Designed to stress-test zero-shot scale extrapolation, DF-Mamba achieved 77.78% accuracy on full-resolution histopathology images despite being trained only on smaller crops. In contrast, Vision Transformers (ViT) and Vision Mamba (ViM) exhibited performance stagnation or failure under identical zero-shot conditions. Furthermore, DF-Mamba required only 0.89M parameters, compared to 1.92M for ViM, 2.3M for ViT, and 11.18M for ResNet.
  • Spatial Transcriptomics: On the 10x Genomics human breast cancer Visium dataset, DF-Mamba outperformed the previous state-of-the-art (DiffusionST) by 14% in Adjusted Rand Index (ARI) and nearly 28% in Normalized Mutual Information (NMI).
  • Long-Range Time-Series Forecasting: Across ETT, Electricity, and Weather datasets, DF-Mamba consistently outperformed Transformers and flat Mamba baselines, particularly excelling on longer prediction horizons (336 and 720 steps).

Limitations and Future Directions

While highly effective for multi-scale hierarchies, DF-Mamba has limitations. The authors note that the model can sometimes "over-smooth" subtle, hyper-local textural anomalies, occasionally struggling to distinguish fine-grained differences (e.g., "InSitu" cancer vs. "Normal" tissue). Additionally, for data that is predominantly single-scale or very short, the recursive overhead may not yield significant performance gains.

Future work will explore cross-scale contrastive learning to help the model better preserve high-frequency residuals, with the ultimate goal of scaling the architecture into multi-modal scientific foundation models.

Summary updated

Abstract

Sequence models typically operate at a fixed temporal or spatial scale and struggle to generalize to substantially longer horizons or higher resolutions without retraining. Existing hierarchical architectures expand receptive fields but rely on scale-specific parameters and lack mechanisms to enforce consistent dynamics across scales. We propose Dynamic Fractal Mamba (DF-Mamba), a recursive state-space model that applies a single shared operator across multiple scales. By sharing parameters across recursion depths and exponentially scaling the effective time step, DF-Mamba achieves an exponentially expanding receptive field while preserving linear computational complexity. A learned content-aware coarse-graining module aggregates representations across scales. Auxiliary reconstruction and cross-scale consistency objectives stabilize recursive training. We evaluate DF-Mamba on long-range time-series forecasting, spatial transcriptomics, and computational pathology. Across all tasks, DF-Mamba consistently outperforms Transformers and flat Mamba baselines while using fewer parameters and maintaining linear-time scalability. Importantly, models trained on short sequences or low-resolution inputs generalize in a zero-shot manner to substantially larger temporal and spatial scales unseen during training. These results demonstrate that recursive parameter sharing provides an effective inductive bias for learning scale-consistent and efficient sequence representations.
BibTeX
@inproceedings{fang2026dynamic,
  author = {Shenglei Fang and Xianfang Sun and You Zhou},
  title = {{Dynamic Fractal Mamba: A Neural Renormalization Group Flow for Scale-Invariant Sequence Modeling}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=L8a9GRfoly},
}
Figure