No article text was recorded for this version.
attention sink preservation
Version 2 of 249 papers63 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Efficient Streaming Language Models with Attention Sinks
- Blog: SinkTrack: Attention Sink based Context Anchoring for Large Language Models
- Blog: FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
- Blog: Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration
- Blog: LongHeads: Multi-Head Attention is Secretly a Long Context Processor
- Blog: Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention
- Blog: IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
- Blog: A Frustratingly Easy Improvement for Position Embeddings via Random Padding
- Blog: LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models
- Blog: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
- Blog: How Do Humans Write Code? Large Models Do It the Same Way Too
- Blog: Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
- Blog: E^2-LLM: Efficient and Extreme Length Extension of Large Language Models
- Blog: Exploring Context Window of Large Language Models via Decomposed Positional Vectors
- Blog: KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
- Blog: Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
- Blog: RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
- Blog: Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
- Blog: SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
- Blog: LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
- Blog: Improving Input-label Mapping with Demonstration Replay for In-context Learning
- Blog: An Understanding-oriented Robust Machine Reading Comprehension Model
- Blog: FCM: Forgetful Causal Masking Makes Causal Language Models Better Zero-Shot Learners
- Blog: Spectral Filters, Dark Signals, and Attention Sinks
- Blog: Frayed RoPE and Long Inputs: A Geometric Perspective
- Blog: D-LLM: A Token Adaptive Computing Resource Allocation Strategy for Large Language Models
- Blog: ClusterAttn: KV Cache Compression under Intrinsic Attention Clustering
- Blog: LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models
- Blog: The Fall of ROME: Understanding the Collapse of LLMs in Model Editing
- Blog: Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey
- Blog: All You Need is One: Capsule Prompt Tuning with a Single Vector
- Blog: Improving Input-label Mapping with Demonstration Replay for In-context Learning
- Blog: Vista-llama: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokens
- Blog: Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
- Blog: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
- Blog: E2-LLM: Efficient and Extreme Length Extension of Large Language Models
- Blog: Revealing Long-context Potential of Attention Heads via Frequency Kernels
- Blog: What Layers When: Learning to Skip Compute in LLMs with Residual Gates
- Blog: Sliding Window Attention Adaptation
- Blog: OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
- Blog: When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
- Blog: Outlier Dimensions that Disrupt Transformers are Driven by Frequency
- Blog: Star Attention: Efficient LLM Inference over Long Sequences
- Blog: Entropy-Aware Dynamic KV Cache Sparsification for Autoregressive Image Generation and Editing
- Blog: RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning
- Blog: A Survey on Large Language Model Acceleration based on KV Cache Management
- Blog: Overcoming Long Context Limitations of State Space Models via Context Dependent Sparse Attention
- Blog: Effective Distillation to Hybrid xLSTM Architectures
- Blog: Exploring the Role of BERT Token Representations to Explain Sentence Probing Results
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_attention_sink_preservation_54f45733,
title = {attention sink preservation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-sink-preservation/rdr_54f457334891474bb92e16fd24b4303a},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_attention_sink_preservation_54f45733_v2,
title = {attention sink preservation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-sink-preservation/rdr_54f457334891474bb92e16fd24b4303a/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}