No article text was recorded for this version.
logarithmic attention scaling
Version 1 of 118 papers26 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation
- Blog: SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling
- Blog: BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model
- Blog: Scaling Laws of RoPE-based Extrapolation
- Blog: Selective Attention: Enhancing Transformer through Principled Context Control
- Blog: KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation
- Blog: Overcoming a Theoretical Limitation of Self-Attention
- Blog: Critical attention scaling in long-context transformers
- Blog: High-Dimensional Analysis of Single-Layer Attention for Sparse-Token Classification
- Blog: Softmax is not Enough (for Sharp Size Generalisation)
- Blog: EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
- Blog: Small Language Model Makes an Effective Long Text Extractor
- Blog: Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex
- Blog: Let's (not) just put things in Context: Test-time Training for Long-context LLMs
- Blog: Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation
- Blog: Scale-invariant attention
- Blog: Rethinking Softmax: Self-Attention with Polynomial Activations
- Blog: Transformers Implement First-Order Logic with Majority Quantifiers
History & citation
Current · Permanent link
- This direction split from Logarithmic Scaling for Stable Long Context Attention (100% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_logarithmic_attention_scaling_2873fc27,
title = {logarithmic attention scaling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/logarithmic-attention-scaling/rdr_2873fc2795024625929ea28e1646d443},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_logarithmic_attention_scaling_2873fc27_v1,
title = {logarithmic attention scaling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/logarithmic-attention-scaling/rdr_2873fc2795024625929ea28e1646d443/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}