GTR: Gated Token Recurrence for Efficient Dense Prediction

Zhe Feng, Longfei Liu, Wei Liu, Kai Chen, Jiangang Kong, Wei Zhou, Yifeng Qian, Dexiong Chen, Xuanlong Yu, Xi ShenarXiv 2026 · arXiv

TL;DR

Gated Token Recurrence (GTR) is a novel, softmax-free recurrent vision backbone designed to break the quadratic computational bottleneck of Transformers in high-resolution dense prediction tasks. By combining gated linear attention, multi-directional spatial scanning, and a highly simplified distillation strategy, GTR achieves state-of-the-art accuracy across six diverse vision tasks while delivering massive throughput gains and low latency on edge hardware like the NVIDIA DRIVE AGX Thor.

What Problem Does This Paper Address?

Modern computer vision relies heavily on self-attention-based backbones (like Vision Transformers) for dense prediction tasks such as object detection, semantic segmentation, and depth estimation. However, standard global softmax attention suffers from a critical flaw: its computational and memory complexity scales quadratically ($O(L^2)$) with the number of image tokens ($L$).

As input resolutions increase—which is essential for safety-critical applications like autonomous driving—processing these images becomes prohibitively expensive, especially on resource-constrained edge devices. While linear attention mechanisms offer a theoretical alternative, building a general-purpose linear backbone that performs consistently across diverse tasks without sacrificing practical hardware efficiency has remained an open challenge.

Methods

The authors introduce Gated Token Recurrence (GTR), a single-scale, linear-complexity vision backbone. It completely discards softmax attention in favor of a homogeneous recurrent architecture built on three core pillars:

  1. Key-Only Gated Linear Attention (GLA): Replaces traditional self-attention with a recurrent formulation that maintains a constant-size state matrix relative to sequence length, reducing complexity to $O(L)$.
  2. Directional Scanning: Because recurrence is inherently causal (directional), a single pass only captures context from one side. GTR solves this by cycling through four distinct scanning directions across its 12 blocks: left-to-right, right-to-left, top-to-bottom, and bottom-to-top.
  3. Spatial SwiGLU: To complement global recurrence with local feature mixing, GTR integrates a 3×3 depthwise convolution into the value branch of a standard SwiGLU block. This design allows the network to completely bypass explicit positional embeddings.
Overview of the GTR architecture showing the block design, scanning directions, and Spatial SwiGLU module.

Additionally, GTR introduces a remarkably simple Final-Output Representation Alignment distillation recipe. Instead of using complex multi-stage alignments or masked token predictions, GTR is trained by directly regressing the final layer's patch tokens against a frozen, detection-specialized DINOv3 teacher using a simple squared $\ell_2$ loss.

To bridge the gap between theoretical efficiency and real-world deployment, the authors also developed a specialized chunkwise CUDA operator that parallelizes chunk summaries and optimizes memory traffic for TensorRT deployment.

Datasets

GTR was thoroughly evaluated across a wide range of standard benchmarks:

  • Backbone Distillation: Image pool combining ImageNet-1K and COCO train2017.
  • Object Detection & Instance Segmentation: Pre-trained on Objects365 and fine-tuned/evaluated on COCO val2017.
  • Human Pose Estimation: COCO val2017.
  • Oriented Object Detection: DOTA-v1.0.
  • Semantic Segmentation: Cityscapes.
  • Monocular Depth Estimation: Evaluated zero-shot on NYU Depth V2 after training on an 8-source mixture (including ARKitScenes, SUN RGB-D, DIODE, Hypersim, TartanAir, Virtual KITTI 2, KITTI, and pseudo-labeled ImageNet).

Key Insights and Ideas

  • Simplicity Wins in Distillation: Unlike prior cross-architecture distillation methods that require matching intermediate layers or using masked inputs, simply aligning the final patch tokens via $\ell_2$ loss is sufficient to transfer powerful representations from a softmax-attention teacher to a linear recurrent student.
  • Pure Linear Attention Suffices: Many recent architectures rely on "hybrid" designs that retain softmax attention in a few layers to maintain accuracy. GTR proves that a pure, homogeneous linear attention backbone can match or exceed hybrid performance while avoiding the latency spikes associated with softmax layers.
  • Memory-Traffic is the Real Bottleneck: On edge hardware like automotive chips, peak compute (FLOPs) is rarely the limiting factor; memory bandwidth is. GTR’s custom chunkwise operator focuses on minimizing global-memory traffic, unlocking dramatic real-world speedups that match its theoretical linear complexity.
Figure 1: GTR: a shared backbone design for efficient dense prediction. Left: Task-specific models built on the same GTR backbone design produce predictions across semantic segmentation, human pose estimation, monocular depth estimation, object detection, and instance segmentation. Right: Peak GPU memory and inference throughput on RTX 4090 as input resolution increases, comparing GTR-S with RF-DETR-S (Robinson et al., 2026), EdgeCrafter-S (Liu et al., 2026), and YOLO26-S (Jocher et al., 2026). GTR-S shows slower peak-memory growth and higher throughput at increasing resolutions.

Key Results

  • Object Detection: GTR-L achieved 58.9 box AP on COCO val2017 with a median batch-one latency of just 1.908 ms on an RTX 4090, outperforming comparable softmax baselines in both accuracy and speed.
  • Resolution Scaling: When scaling input resolutions up to $1280^2$, GTR-S exhibited significantly slower peak-memory growth and achieved up to 81× faster processing speeds compared to softmax-based alternatives.
  • Kernel Efficiency: The specialized CUDA operator achieved up to a 4.0× speedup over the FlashLinearAttention (FLA) v0.5.0 baseline at 1.6K tokens.
  • Edge Deployment: When deployed via TensorRT on an automotive-grade NVIDIA DRIVE AGX Thor chip, GTR variants achieved real-time latencies ranging between 2.282 ms and 8.769 ms across all evaluated tasks.

Limitations and Future Work

While GTR demonstrates exceptional performance on dense, pixel-level prediction tasks, the authors note a few areas for improvement:

  • Teacher Dependency: The backbone relies heavily on a high-quality, pre-trained softmax teacher (DINOv3) for its initial distillation phase. Direct, stable pre-training of such recurrent architectures from scratch on massive datasets remains an open research question.
  • Task Scope: The evaluation is strictly focused on dense downstream vision tasks; its performance on global image-level classification or multi-modal vision-language tasks was not explored.

Summary updated

Abstract

Self-attention-based vision backbones perform well on dense prediction, but the quadratic computational cost of global softmax attention limits their efficiency as image resolution increases. We introduce Gated Token Recurrence (GTR), a softmax-free recurrent vision backbone that combines gated linear attention, alternating spatial scan directions, and spatially enhanced SwiGLU blocks. GTR is distilled from a detection-specialized DINOv3 teacher using only final-layer patch-token alignment through a linear projection and squared $\ell_2$ loss, without masked-token prediction or intermediate-layer supervision. With Objects365 detector pre-training, GTR-L achieves 58.9 box AP on COCO \texttt{val2017} with 1.908\,ms median batch-one latency under compiled FP16 execution on an RTX~4090. The same backbone also transfers to instance segmentation, pose estimation, oriented detection, semantic segmentation, and monocular depth estimation. In an isolated kernel benchmark, our specialized chunkwise CUDA operator is $4.0\times$ faster than FLA v0.5.0 at 1.6K tokens on RTX~4090. TensorRT deployment on DRIVE AGX Thor achieves 2.282--8.769\,ms median batch-one latency across the evaluated models. These results show that recurrent token mixing can provide an efficient alternative to global softmax attention for high-resolution dense prediction and edge deployment. Project page: https://intellindust-ai-lab.github.io/projects/GTR/
BibTeX
@misc{feng2026gtr,
  author = {Zhe Feng and Longfei Liu and Wei Liu and Kai Chen and Jiangang Kong and Wei Zhou and Yifeng Qian and Dexiong Chen and Xuanlong Yu and Xi Shen},
  title = {{GTR: Gated Token Recurrence for Efficient Dense Prediction}},
  year = {2026},
  eprint = {2609.26590},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.26590},
}
Figure