No article text was recorded for this version.
adaptive attention sparsity
Version 1 of 1205 papers271 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
- Blog: FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
- Blog: Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
- Blog: RaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference
- Blog: SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
- Blog: Adaptive Attention for Sparse-based Long-sequence Transformer
- Blog: AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference
- Blog: Training-free and Adaptive Sparse Attention for Efficient Long Video Generation
- Blog: Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
- Blog: Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification
- Blog: Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
- Blog: DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
- Blog: Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition
- Blog: MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
- Blog: Dynamic N: M Fine-grained Structured Sparse Attention Mechanism
- Blog: Sharing Key Semantics in Transformer Makes Efficient Image Restoration
- Blog: ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification and KV Cache Compression
- Blog: ProxyAttn: Guided Sparse Attention via Representative Heads
- Blog: TileSparse: Arithmetic-Intensity-Aware Sparse Attention for Compute-Bound LLM Decoding
- Blog: SCRAM: Spatially Coherent Randomized Attention Maps
- Blog: Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning
- Blog: Unifying KV Cache Compression for Large Language Models with LeanKV
- Blog: XAttention: Block Sparse Attention with Antidiagonal Scoring
- Blog: SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling
- Blog: Twilight: Adaptive Attention Sparsity with Hierarchical Top-p Pruning
- Blog: VSA: Faster Video Diffusion with Trainable Sparse Attention
- Blog: VMoBA: Mixture-of-Block Attention for Video Diffusion Models
- Blog: Twilight: Adaptive Attention Sparsity with Hierarchical Top-Pruning
- Blog: SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
- Blog: Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost
- Blog: Multi-Kernel Correlation-Attention Vision Transformer for Enhanced Contextual Understanding and Multi-Scale Integration
- Blog: VMoBA: Mixture-of-Block Attention for Video Diffusion Models
- Blog: SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
- Blog: AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference
- Blog: Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
- Blog: Rethinking the Self-Attention in Vision Transformers
- Blog: MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training
- Blog: Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
- Blog: SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
- Blog: Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
- Blog: Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
- Blog: SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection
- Blog: Learning When Not to Attend Globally
- Blog: FieldFormer: Self-Supervised Reconstruction of Physical Fields via Tensor Attention Prior
- Blog: Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding
- Blog: Sequential Attention with Keyword Mask Model for Community-based Question Answering
- Blog: Adaptively Sparse Transformers
- Blog: Bidirectional Sparse Attention for Faster Video Diffusion Training
- Blog: Cross Paradigm Representation and Alignment Transformer for Image Deraining
- Blog: Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries
- Blog: ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models
- Blog: SeerAttention: Self-distilled Attention Gating for Efficient Long-context Prefilling
- Blog: A length adaptive algorithm-hardware co-design of transformer on FPGA through sparse attention and dynamic pipelining
- Blog: Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers
- Blog: Accelerating Transformer Pre-Training with 2: 4 Sparsity
- Blog: Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing
- Blog: DeAltHDR: Learning HDR Video Reconstruction from Degraded Alternating Exposure Sequences
- Blog: OmniSparse: Training-Aware Fine-Grained Sparse Attention for Long-Video MLLMs
- Blog: ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
- Blog: Only 5\% Attention Is All You Need: Efficient Long-range Document-level Neural Machine Translation
Showing 60 of 205 papers.
History & citation
Current · Permanent link
- Context Dependent Sparse Attention for Efficient Transformers merged into this direction (85% overlap)
- Adaptive Sparse Attention via Dynamic Budgeting merged into this direction (68% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_adaptive_attention_sparsity_6a5f3647,
title = {adaptive attention sparsity},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/adaptive-attention-sparsity/rdr_6a5f3647c371420286d17fccc6c35e9e},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_adaptive_attention_sparsity_6a5f3647_v1,
title = {adaptive attention sparsity},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/adaptive-attention-sparsity/rdr_6a5f3647c371420286d17fccc6c35e9e/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}