BDNet: Fast-slow thinking is all you need

Chenzhi Zheng, Zhuoran ZhengarXiv 2026 · arXiv

TL;DR

BDNet introduces a "fast-slow" thinking paradigm for image classification by transforming per-image "fast" features relative to "slow" statistical references accumulated during training. By replacing complex pairwise attention with a simple, linear-complexity Baseline Deformation Operator (BDO), the model achieves competitive accuracy while demonstrating significantly superior robustness to noise and distribution shifts.

What Problem Does This Paper Address?

Modern vision backbones—whether based on Convolutions, Transformers, or State-Space Models (SSMs)—primarily compete on their spatial aggregation rules. However, the authors of BDNet identify a different question: Should an individual image's features be interpreted in isolation, or relative to a stable reference that represents the broader dataset statistics?

Standard models often struggle with robustness; a small amount of Gaussian noise or a shift in brightness can tank performance. Furthermore, the $O(N^2)$ complexity of self-attention remains a bottleneck for high-resolution processing. BDNet aims to build a compact, $O(N)$ complexity backbone that uses "slow" statistical memory to anchor "fast" visual features, making the representation more resilient to perturbations.

Methods

The core innovation is the Baseline Deformation Operator (BDO). Unlike attention, which compares every token to every other token, BDO operates on the residual between a feature and a reference.

  1. Fast Features ($f$): These are the standard hidden states recomputed for every image.
  2. Slow References ($s$): These are stage-specific vectors that accumulate the mean of training outputs via an Exponential Moving Average (EMA). Crucially, these references are frozen during inference, acting as a fixed "coordinate frame" for the model.
  3. The Operator: The BDO subtracts the slow reference from the fast feature, applies a learned low-rank channel projection ($D \to r \to D$, where $r=D/4$), and adds the result back via a residual connection scaled by a learned parameter $\gamma$.
The BDO calculates a learned correction based on the deviation from a slow statistical reference.

The overall architecture is a standard four-stage pyramid (20.70M parameters) using Pre-LayerNorm residual blocks. Because BDO acts independently on tokens, spatial aggregation is handled by the surrounding convolutional stem and downsampling layers.

The BDNet block integrates the BDO correction alongside a standard MLP.

Datasets

  • Training/Validation: ImageNet-1K (ILSVRC2012).
  • Robustness Evaluation: The authors applied controlled perturbations to the ImageNet validation set, including:
  • Gaussian Noise: $\sigma \in \{0.05, 0.10, 0.20\}$
    • Contrast: Factors $\in \{0.75, 0.50, 0.25\}$
    • Brightness: Additive shifts $\in \{-0.20, -0.10, +0.10, +0.20\}$

Key Insights and Ideas

  • Deviation as Information: The central hypothesis is that the difference between an image and the "average" image is more informative and robust than the raw feature itself. If a feature aligns perfectly with the reference, the BDO correction vanishes.
  • Linear Inference Behavior: At inference time, because the reference $s$ is frozen, the BDO effectively becomes a constant channel offset plus a linear projection. This makes it computationally much lighter than dynamic attention mechanisms.
  • Decoupling Accuracy and Robustness: The study shows that while a model with attention (MHA) might achieve slightly higher "clean" accuracy, it degrades much faster under noise. BDNet prioritizes a "graceful degradation" profile.
  • Statistical Persistence: While BatchNorm also uses running statistics, BDNet uses them to define a displacement in a learned low-dimensional subspace, rather than just for normalization.

Key Results

BDNet (20.7M params) achieves a 80.2% Top-1 accuracy on clean ImageNet-1K. While this is slightly lower than larger models like ConvNeXt-T (82.1%), its strength lies in robustness:

  • Noise Resilience: Under heavy Gaussian noise ($\sigma=0.20$), BDNet retains 52.1% accuracy, significantly outperforming a "Zero-reference" version of itself (44.2%) and a budget-matched Attention model (46.4%).
  • Efficiency: The model maintains a linear complexity relative to the number of tokens, making it a viable candidate for high-resolution tasks where $O(N^2)$ attention fails.
  • Ablation Success: Removing the "slow" reference (setting $\delta = f$) resulted in a 2.2% drop in clean accuracy and a massive 6.3% drop in mean noise robustness, proving the reference is doing the heavy lifting.

Limitations and Future Work

  • Clean Accuracy Ceiling: The authors acknowledge that BDNet's clean accuracy is "modest" compared to the strongest published baselines like VMamba or ConvNeXt. It is positioned as a robustness-first architecture rather than a pure SOTA-chaser.
  • Task Scope: The paper focuses exclusively on image classification. It remains to be seen how this "fast-slow" mechanism translates to dense prediction tasks like object detection or semantic segmentation.
  • Reference Complexity: Currently, the "slow" state is a simple mean. Future work could explore more complex statistical references (e.g., class-conditional references or higher-order moments) to further improve representation power.

Summary updated

Abstract

We present BDNet, a hierarchical image-classification backbone built around fast image features and slow statistical references. Its baseline deformation operator (BDO) transforms the deviation of a feature from a stage-specific reference through a low-rank projection and a learned channel scale. Rather than computing pairwise token similarity, the branch operates directly on baseline-relative residuals with complexity linear in the token count. Each of four reference buffers accumulates detached stage statistics during training and remains frozen at inference. A four-stage pyramid with depths $(2,2,6,2)$ and widths $(96,192,384,768)$ yields a 20.70M-parameter classifier at $224\times224$ resolution. We organize an ImageNet-1K study around clean classification, eight controlled ablations, and Gaussian-noise, contrast, and brightness perturbations. The central hypothesis is that a slow reference can improve the robustness of a compact representation while retaining a simple tensor implementation. We distinguish this training-time statistical mechanism from input-dependent attention and make its linear inference behavior explicit.
BibTeX
@misc{zheng2026bdnet,
  author = {Chenzhi Zheng and Zhuoran Zheng},
  title = {{BDNet: Fast-slow thinking is all you need}},
  year = {2026},
  eprint = {2610.41485},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.41485},
}
Figure