No article text was recorded for this version.
sliding window attention
Version 1 of 137 papers43 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Improving Long Context Document-Level Machine Translation
- Blog: Long Document Summarization with Top-Down and Bottom-Up Representation Inference
- Blog: Long-Span Summarization via Local Attention and Content Selection
- Blog: Local Self-Attention over Long Text for Efficient Document Retrieval
- Blog: MaskViT: Masked Visual Pre-Training for Video Prediction
- Blog: Effective Approaches to Attention-based Neural Machine Translation
- Blog: TAP-Net: Transport-and-Pack using Reinforcement Learning
- Blog: Generating Wikipedia by Summarizing Long Sequences
- Blog: Word Sense Disambiguation with Knowledge-Enhanced and Local Self-Attention-based Extractive Sense Comprehension
- Blog: Cascaded Text Generation with Markov Transformers
- Blog: Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide Analysis
- Blog: TriG-NER: Triplet-Grid Framework for Discontinuous Named Entity Recognition
- Blog: MotionDreamer: One-to-Many Motion Synthesis with Localized Generative Masked Transformer
- Blog: Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN
- Blog: An Online Sequence-to-Sequence Model Using Partial Conditioning
- Blog: Spatio-Temporal Fusion Based Convolutional Sequence Learning for Lip Reading
- Blog: Scaling Up ESM2 Architectures for Long Protein Sequences Analysis: Long and Quantized Approaches
- Blog: LongCoder: A Long-Range Pre-trained Language Model for Code Completion
- Blog: HTS-AT: A hierarchical token-semantic audio transformer for sound classification and detection
- Blog: Inference-Friendly Models With MixAttention
- Blog: Do Long-Range Language Models Actually Use Long-Range Context?
- Blog: Universal Length Generalization with Turing Programs
- Blog: Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
- Blog: CA3D: Convolutional-Attentional 3D Nets for Efficient Video Activity Recognition on the Edge
- Blog: Bytes Are All You Need: Transformers Operating Directly On File Bytes
- Blog: Quantifying the Contextualization of Word Representations with Semantic Class Probing
- Blog: Hierarchical Neural Network Approaches for Long Document Classification
- Blog: Efficient Time Series Processing for Transformers and State-Space Models through Token Merging
- Blog: Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model
- Blog: MuLD: The Multitask Long Document Benchmark
- Blog: MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling
- Blog: Distilling Linguistic Context for Language Model Compression
- Blog: When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants
- Blog: An Exploratory Study on Long Dialogue Summarization: What Works and What's Next
- Blog: Attentive cross-modal paratope prediction
- Blog: Learn Your Tokens: Word-Pooled Tokenization for Language Modeling
- Blog: Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
History & citation
Current · Permanent link
- This direction split from Sliding Window Attention for Long Context Transformers (81% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_sliding_window_attention_e379c674,
title = {sliding window attention},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/sliding-window-attention/rdr_e379c674e9d74e5f81539254034f7f1b},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 1, frozen text):
@misc{lacuna_sliding_window_attention_e379c674_v1,
title = {sliding window attention},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/sliding-window-attention/rdr_e379c674e9d74e5f81539254034f7f1b/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}