No article text was recorded for this version.
hybrid local-global attention (hybrid global-local attention)
Version 1 of 1278 papers335 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Long-Short Transformer: Efficient Transformers for Language and Vision
- Blog: RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling
- Blog: Liger: Linearizing Large Language Models to Gated Recurrent Structures
- Blog: Probabilistic Time Series Modeling with Decomposable Denoising Diffusion Model
- Blog: ConvBERT: Improving BERT with Span-based Dynamic Convolution
- Blog: SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling
- Blog: LoLCATs: On Low-Rank Linearizing of Large Language Models
- Blog: Linear Log-Normal Attention with Unbiased Concentration
- Blog: Investigating Efficiently Extending Transformers for Long Input Summarization
- Blog: Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer
- Blog: Simple linear attention language models balance the recall-throughput tradeoff
- Blog: MVA: Linear Attention with High-order Query-Keys Integration and Multi-level Vocabulary Decomposition
- Blog: Focus Your Attention (with Adaptive IIR Filters)
- Blog: Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision
- Blog: PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention
- Blog: Lightweight Self-Attentive Sequential Recommendation
- Blog: Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification
- Blog: SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
- Blog: Unifying Linear-Time Attention via Latent Probabilistic Modelling
- Blog: EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes
- Blog: Gemma 2: Improving Open Language Models at a Practical Size
- Blog: GFormer: Accelerating Large Language Models with Optimized Transformers on Gaudi Processors
- Blog: Small Language Model Makes an Effective Long Text Extractor
- Blog: Liger: Linearizing Large Language Models to Gated Recurrent Structures
- Blog: Cluster-Former: Clustering-based Sparse Transformer for Question Answering
- Blog: The Devil in Linear Transformer
- Blog: Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions
- Blog: DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization
- Blog: Hymba: A Hybrid-head Architecture for Small Language Models
- Blog: One-Minute Video Generation with Test-Time Training
- Blog: Fast Conformer With Linearly Scalable Attention For Efficient Speech Recognition
- Blog: Beyond Paragraphs: NLP for Long Sequences
- Blog: Efficient Content-Based Sparse Attention with Routing Transformers
- Blog: Efficient Attention via Control Variates
- Blog: Core Context Aware Attention for Long Context Language Modeling
- Blog: Rope to Nope and Back Again: A New Hybrid Attention Strategy
- Blog: Towards Efficiently Learning Monotonic Alignments for Attention-based End-to-End Speech Recognition
- Blog: MiMo-V2-Flash Technical Report
- Blog: Machine Reading Comprehension Model Based on Fusion of Mixed Attention
- Blog: Transformer Quality in Linear Time
- Blog: Linear Log-Normal Attention with Unbiased Concentration
- Blog: Leveraging Local and Global Patterns for Self-Attention Networks
- Blog: Small Language Model Makes an Effective Long Text Extractor
- Blog: LoLCATs: On Low-Rank Linearizing of Large Language Models
- Blog: Linear Attention via Orthogonal Memory
- Blog: Sliding Window Attention Adaptation
- Blog: Longformer: The Long-Document Transformer
- Blog: Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
- Blog: Long-Short Transformer: Efficient Transformers for Language and Vision
- Blog: WebFormer: The Web-page Transformer for Structure Information Extraction
- Blog: Usage of specific attention improves change point detection
- Blog: Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
- Blog: Effective Distillation to Hybrid xLSTM Architectures
- Blog: 2D-WinSpatt-Net: A Dual Spatial Self-Attention Vision Transformer Boosts Classification of Tetanus Severity for Patients Wearing ECG Sensors in Low- and Middle-Income Countries
- Blog: Multi-Task Time Series Forecasting With Shared Attention
- Blog: ELFATT: Efficient Linear Fast Attention for Vision Transformers
- Blog: Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making
- Blog: DeltaLLM: A Training-Free Framework Exploiting Temporal Sparsity for Efficient Edge LLM Inference
- Blog: TopicBERT for Energy Efficient Document Classification
- Blog: Factorization Vision Transformer: Modeling Long-Range Dependency With Local Window Cost
Showing 60 of 278 papers.
History & citation
Current · Permanent link
- This direction split from Hybrid Architectures for Efficient Long Context Modeling (79% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_hybrid_local_global_attention_hybrid_glo_f83a57e6,
title = {hybrid local-global attention (hybrid global-local attention)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/hybrid-local-global-attention-hybrid-global-local-attention/rdr_f83a57e64474435baec07d7f9d1c3b90},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_hybrid_local_global_attention_hybrid_glo_f83a57e6_v1,
title = {hybrid local-global attention (hybrid global-local attention)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/hybrid-local-global-attention-hybrid-global-local-attention/rdr_f83a57e64474435baec07d7f9d1c3b90/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}