No article text was recorded for this version.
length generalization failure
Version 1 of 171 papers85 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Sequence Length is a Domain: Length-based Overfitting in Transformer Models
- Blog: Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers
- Blog: Understanding How Positional Encodings Work in Transformer Model
- Blog: Context-aware Biases for Length Extrapolation
- Blog: Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities
- Blog: Sequence Length Independent Norm-Based Generalization Bounds for Transformers
- Blog: Sequence Length Independent Norm-Based Generalization Bounds for Transformers
- Blog: Born a Transformer - Always a Transformer?
- Blog: Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis
- Blog: A Convex Relaxation Approach to Generalization Analysis for Parallel Positively Homogeneous Networks
- Blog: TRA: Better Length Generalisation with Threshold Relative Attention
- Blog: Generalizing Reasoning Problems to Longer Lengths
- Blog: How Do Position Encodings Affect Length Generalization? Case Studies On In-Context Function Learning
- Blog: Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
- Blog: How Many Different Outputs Can a Transformer Generate?
- Blog: Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation
- Blog: Randomized Positional Encodings Boost Length Generalization of Transformers
- Blog: Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study
- Blog: Softplus Attention with Re-weighting Boosts Length Extrapolation in Large Language Models
- Blog: Adaptive Recurrence as Algorithmic Time for Length Generalization in Addition
- Blog: From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers
- Blog: Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study
- Blog: Learning Long-Horizon Action Dependencies in Sampling-Based Bilevel Planning
- Blog: A Theoretical Study of (Hyper) Self-Attention through the Lens of Interactions: Representation, Training, Generalization
- Blog: Long-Context Generalization with Sparse Attention
- Blog: The Role of Logic and Automata in Understanding Transformers
- Blog: Unveiling Transformers with LEGO: a synthetic reasoning task
- Blog: DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
- Blog: A Length-Extrapolatable Transformer
- Blog: Graph neural networks extrapolate out-of-distribution for shortest paths
- Blog: softmax is not enough (for sharp out-of-distribution)
- Blog: Extrapolation by Association: Length Generalization Transfer In Transformers
- Blog: LVCSR with Transformer Language Models
- Blog: In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention
- Blog: Towards Neural Functional Program Evaluation
- Blog: Universal Length Generalization with Turing Programs
- Blog: Towards a Theoretical Understanding of In-context Learning: Stability and Non-I.I.D Generalisation
- Blog: Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning
- Blog: Exploring Length Generalization For Transformer-based Speech Enhancement
- Blog: Long-Short Alignment for Effective Long-Context Modeling in LLMs
- Blog: A Formal Framework for Understanding Length Generalization in Transformers
- Blog: Recurrent Transformers Trade-off Parallelism for Length Generalization on Regular Languages
- Blog: StableMask: Refining Causal Masking in Decoder-only Transformer
- Blog: On the Long Range Abilities of Transformers
- Blog: Position Coupling: Leveraging Task Structure for Improved Length Generalization of Transformers
- Blog: The Impact of Positional Encoding on Length Generalization in Transformers
- Blog: Length Generalization Bounds for Transformers
- Blog: CodeSSM: Towards State Space Models for Code Understanding
- Blog: Quantitative Bounds for Length Generalization in Transformers
- Blog: When can transformers reason with abstract symbols?
- Blog: Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
- Blog: Principled Understanding of Generalization for Generative Transformer Models in Arithmetic Reasoning Tasks
- Blog: Incremental Transformer Neural Processes
- Blog: Transformers in Time Series: A Survey
- Blog: Transformers Can Do Arithmetic with the Right Embeddings
- Blog: Visual Reasoning: from State to Transformation
- Blog: Learning Adaptive Control Flow in Transformers for Improved Systematic Generalization
- Blog: Exploring Transformer Extrapolation
- Blog: Measuring Systematic Generalization in Neural Proof Generation with Transformers
- Blog: Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping
Showing 60 of 71 papers.
History & citation
Current · Permanent link
- This direction split from Length Generalization in Transformer Architectures (85% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_length_generalization_failure_0fb56f49,
title = {length generalization failure},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/length-generalization-failure/rdr_0fb56f49cc6d43b588681830c1853e87},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 1, frozen text):
@misc{lacuna_length_generalization_failure_0fb56f49_v1,
title = {length generalization failure},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/length-generalization-failure/rdr_0fb56f49cc6d43b588681830c1853e87/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}