No article text was recorded for this version.
hardware-aware sparse attention
Version 1 of 1142 papers187 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Efficiently Dispatching Flash Attention For Partially Filled Attention Masks
- Blog: Fast Video Generation with Sliding Tile Attention
- Blog: Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
- Blog: ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design
- Blog: Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light
- Blog: FlatFusion: Delving into Details of Sparse Transformer-based Camera-LiDAR Fusion for Autonomous Driving
- Blog: ELFATT: Efficient Linear Fast Attention for Vision Transformers
- Blog: HilbertA: Hilbert-Curve–Aligned Sparse Attention for 2D Structured Data
- Blog: Blockwise Self-Attention for Long Document Understanding
- Blog: VORTA: Efficient Video Diffusion via Routing Sparse Attention
- Blog: Transformer Acceleration with Dynamic Sparse Attention
- Blog: UniGist: Towards General and Hardware-aligned Sequence-level Long Context Compression
- Blog: FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
- Blog: Structural Deep Encoding for Table Question Answering
- Blog: InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference
- Blog: GFormer: Accelerating Large Language Models with Optimized Transformers on Gaudi Processors
- Blog: HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation
- Blog: HashAttention: Semantic Sparsity for Faster Inference
- Blog: Fast Attention Over Long Sequences With Dynamic Sparse Flash Attention
- Blog: Sparse Attention Adaptation for Long Reasoning
- Blog: Flash Sparse Attention: An Alternative Efficient Implementation of Native Sparse Attention Kernel
- Blog: QUILL: An Algorithm-Architecture Co-Design for Cache-Local Deformable Attention
- Blog: DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance
- Blog: DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generation
- Blog: Optimizing Mixture of Block Attention
- Blog: MonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention
- Blog: A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts
- Blog: UniGist: Towards General and Hardware-aligned Sequence-level Long Context Compression
- Blog: Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
- Blog: Hilbert-Guided Sparse Local Attention
- Blog: DuSA: Fast and Accurate Dual-Stage Sparse Attention Mechanism Accelerating Both Training and Inference
- Blog: A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts
- Blog: NOSA: Native and Offloadable Sparse Attention
- Blog: Flash3D: Super-scaling Point Transformers through Joint Hardware-Geometry Locality
- Blog: Multi-Head LatentMoE and Head Parallel: Communication-Efficient and Deterministic MoE Parallelism
- Blog: Adaptive Computation Pruning for the Forgetting Transformer
- Blog: Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention
- Blog: Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation
- Blog: Big Bird: Transformers for Longer Sequences
- Blog: DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
- Blog: SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention
- Blog: Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers
- Blog: Fast Video Generation with Sliding Tile Attention
- Blog: NOSA: Native and Offloadable Sparse Attention
- Blog: Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation
- Blog: Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference
- Blog: Sparser Block-Sparse Attention via Token Permutation
- Blog: FlashMask: Efficient and Rich Mask Extension of FlashAttention
- Blog: A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
- Blog: RaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference
- Blog: Faster Causal Attention Over Large Sequences Through Sparse Flash Attention
- Blog: BiFormer: Vision Transformer with Bi-Level Routing Attention
- Blog: SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
- Blog: SparseD: Sparse Attention for Diffusion Language Models
- Blog: PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models
- Blog: PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
- Blog: Self-Selected Attention Span for Accelerating Large Language Model Inference
- Blog: Sparse Modular Activation for Efficient Sequence Modeling
- Blog: VideoNSA: Native Sparse Attention Scales Video Understanding
- Blog: Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
Showing 60 of 142 papers.
History & citation
Current · Permanent link
- This direction split from Hardware-Aware Sparse Attention for Long-Context Transformers (84% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_hardware_aware_sparse_attention_8d382bd0,
title = {hardware-aware sparse attention},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/hardware-aware-sparse-attention/rdr_8d382bd0277a4789be6d40d206434092},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_hardware_aware_sparse_attention_8d382bd0_v1,
title = {hardware-aware sparse attention},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/hardware-aware-sparse-attention/rdr_8d382bd0277a4789be6d40d206434092/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}