No article text was recorded for this version.
sparse attention limitations
Version 1 of 138 papers38 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Only 5% Attention Is All You Need: Efficient Long-range Document-level Neural Machine Translation
- Blog: Customizing the Inductive Biases of Softmax Attention using Structured Matrices
- Blog: SparQ Attention: Bandwidth-Efficient LLM Inference
- Blog: Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer
- Blog: ProxyAttn: Guided Sparse Attention via Representative Heads
- Blog: Context Matters: Query-aware Dynamic Long Sequence Modeling of Gigapixel Images
- Blog: Long Document Re-ranking with Modular Re-ranker
- Blog: Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling
- Blog: Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers
- Blog: InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
- Blog: Multi Resolution Analysis (MRA) for Approximate Self-Attention
- Blog: Deformable VisTR: Spatio Temporal Deformable Attention for Video Instance Segmentation
- Blog: You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling
- Blog: SMamba: Sparse Mamba for Event-based Object Detection
- Blog: Linear-Time Self Attention with Codeword Histogram for Efficient Recommendation
- Blog: Cluster-Former: Clustering-based Sparse Transformer for Question Answering
- Blog: Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning
- Blog: Luna: Linear Unified Nested Attention
- Blog: Efficient Content-Based Sparse Attention with Routing Transformers
- Blog: Core Context Aware Attention for Long Context Language Modeling
- Blog: Dodo: Dynamic Contextual Compression for Decoder-only LMs
- Blog: Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
- Blog: Sparser Block-Sparse Attention via Token Permutation
- Blog: Efficient Representation Learning via Adaptive Context Pooling
- Blog: Porous Lattice Transformer Encoder for Chinese NER
- Blog: Diffuser: Efficient Transformers with Multi-Hop Attention Diffusion for Long Sequences
- Blog: General-purpose, long-context autoregressive modeling with Perceiver AR
- Blog: In-context Autoencoder for Context Compression in a Large Language Model
- Blog: DuSA: Fast and Accurate Dual-Stage Sparse Attention Mechanism Accelerating Both Training and Inference
- Blog: Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers
- Blog: Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting
- Blog: Scaling Attention via Feature Sparsity
- Blog: A Provable Expressiveness Hierarchy in Hybrid Linear-Full Attention
- Blog: Poolingformer: Long Document Modeling with Pooling Attention
- Blog: SCA: Selective Compression Attention for Efficiently Extending the Context Window of Large Language Models
- Blog: Multi Resolution Analysis (MRA) for Approximate Self-Attention
- Blog: Scalable Hierarchical Self-Attention with Learnable Hierarchy for Long-Range Interactions
- Blog: On Limitation of Transformer for Learning HMMs
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_sparse_attention_limitations_42423de9,
title = {sparse attention limitations},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/sparse-attention-limitations/rdr_42423de99e62400caadc85addcc6ea48},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_sparse_attention_limitations_42423de9_v1,
title = {sparse attention limitations},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/sparse-attention-limitations/rdr_42423de99e62400caadc85addcc6ea48/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}