No article text was recorded for this version.
attention sink phenomenon
Version 1 of 146 papers64 conceptsfrozen 2026-10-04
Papers in this version
- Blog: When Attention Sink Emerges in Language Models: An Empirical View
- Blog: Vision Transformers Don't Need Trained Registers
- Blog: StableMask: Refining Causal Masking in Decoder-only Transformer
- Blog: Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs
- Blog: From Attention to Activation: Unraveling the Enigmas of Large Language Models
- Blog: Systematic Outliers in Large Language Models
- Blog: Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention
- Blog: SLASH the Sink: Sharpening Structural Attention Inside LLMs
- Blog: Outlier-Efficient Hopfield Layers for Large Transformer-Based Models
- Blog: Efficient Streaming Language Models with Attention Sinks
- Blog: Attention Flows for General Transformers
- Blog: Softmax-free Linear Transformers
- Blog: Fast and Low-Cost Genomic Foundation Models via Outlier Removal
- Blog: You Need Better Attention Priors
- Blog: DualVC 2: Dynamic Masked Convolution for Unified Streaming and Non-Streaming Voice Conversion
- Blog: SkyLadder: Better and Faster Pretraining via Context Window Scheduling
- Blog: Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps
- Blog: Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning
- Blog: Compositional De-Attention Networks
- Blog: Exchangeable Deep Neural Networks for Set-to-Set Matching and Learning
- Blog: Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
- Blog: SinkTrack: Attention Sink based Context Anchoring for Large Language Models
- Blog: From Attention to Activation: Unravelling the Enigmas of Large Language Models
- Blog: Spectral Filters, Dark Signals, and Attention Sinks
- Blog: Your Transformer May Not be as Powerful as You Expect
- Blog: Analyzing the Structure of Attention in a Transformer Language Model
- Blog: Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
- Blog: What are you sinking? A geometric approach on attention sink
- Blog: DropKey for Vision Transformer
- Blog: Flowformer: Linearizing Transformers with Conservation Flows
- Blog: Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts
- Blog: Null-Shot Prompting: Rethinking Prompting Large Language Models With Hallucination
- Blog: PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
- Blog: Look Both Ways and No Sink: Converting LLMs into Text Encoders without Training
- Blog: On the Existence and Behavior of Secondary Attention Sinks
- Blog: One Token Is Enough: Improving Diffusion Language Models with a Sink Token
- Blog: LOTFormer: Doubly-Stochastic Linear Attention via Low-Rank Optimal Transport
- Blog: Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers
- Blog: Compositional Video Generation as Flow Equalization
- Blog: The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity
- Blog: A Fixed-Point Approach for Causal Generative Modeling
- Blog: Flaming-hot Initiation with Regular Execution Sampling for Large Language Models
- Blog: Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
- Blog: Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference
- Blog: Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework
- Blog: Learning What's Easy: Fully Differentiable Neural Easy-First Taggers
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_attention_sink_phenomenon_5af20fd3,
title = {attention sink phenomenon},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-sink-phenomenon/rdr_5af20fd340644f25ada0171302dd1d14},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_attention_sink_phenomenon_5af20fd3_v1,
title = {attention sink phenomenon},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-sink-phenomenon/rdr_5af20fd340644f25ada0171302dd1d14/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}