No article text was recorded for this version.
positional encoding dependency
Version 2 of 255 papers57 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Understanding How Positional Encodings Work in Transformer Model
- Blog: Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
- Blog: Impact of Positional Encoding: Clean and Adversarial Rademacher Complexity for Transformers under In-Context Regression
- Blog: What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding
- Blog: VReBERT: A Simple and Flexible Transformer for Visual Relationship Detection
- Blog: Sigmoid Attention as a Better Substrate for Learned KV Cache Eviction
- Blog: Extending Context Window of Large Language Models from a Distributional Perspective
- Blog: PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
- Blog: Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation
- Blog: Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization
- Blog: Design Choices for Extending the Context Length of Visual Language Models
- Blog: The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval
- Blog: Position Augmentation: Reducing RoPE Extrapolation Cliffs via Random Position Scaling During Training
- Blog: Context Length Extension via Generalized Extrapolation Scale
- Blog: From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers
- Blog: Leveraging Diverse Modeling Contexts with Collaborating Learning for Neural Machine Translation
- Blog: LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
- Blog: Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities
- Blog: ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation
- Blog: MrRoPE: Mixed-radix Rotary Position Embedding
- Blog: On the token distance modeling ability of higher RoPE attention dimension
- Blog: Frayed RoPE and Long Inputs: A Geometric Perspective
- Blog: CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
- Blog: Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs
- Blog: Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
- Blog: On the token distance modeling ability of higher RoPE attention dimension
- Blog: Resonance RoPE: Improving Context Length Generalization of Large Language Models
- Blog: Improved MLP Point Cloud Processing with High-Dimensional Positional Encoding
- Blog: Vista-llama: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokens
- Blog: PPE: Positional Preservation Embedding for Token Compression in Multimodal Large Language Models
- Blog: Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding
- Blog: A Frustratingly Easy Improvement for Position Embeddings via Random Padding
- Blog: VideoRoPE: What Makes for Good Video Rotary Position Embedding?
- Blog: Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
- Blog: PolarQuant: Leveraging Polar Transformation for Key Cache Quantization and Decoding Acceleration
- Blog: CLEX: Continuous Length Extrapolation for Large Language Models
- Blog: Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
- Blog: Revealing Long-context Potential of Attention Heads via Frequency Kernels
- Blog: Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models
- Blog: Demystifying the Slash Pattern in Attention: The Role of RoPE
- Blog: PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference Overhead
- Blog: Multi-matrix Factorization Attention
- Blog: Mitigating Posterior Salience Attenuation in Long-Context LLMs with Positional Contrastive Decoding
- Blog: Induction Heads Interpolate N-Grams
- Blog: Linearized Relative Positional Encoding
- Blog: Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
- Blog: Critical attention scaling in long-context transformers
- Blog: Physics of Language Models: Part 1, Learning Hierarchical Language Structures
- Blog: Not All Answers Are Contextually Persuadable: Inference Dynamics in Large Language Models under Contextual Influence
- Blog: Tighter Bounds on the Expressivity of Transformer Encoders
- Blog: EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMs
- Blog: Rethinking Addressing in Language Models via Contextualized Equivariant Positional Encoding
- Blog: A Contextual-Aware Position Encoding for Sequential Recommendation
- Blog: Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
- Blog: On Initializing Transformers with Pre-trained Embeddings
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_positional_encoding_dependency_9ea390b5,
title = {positional encoding dependency},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/positional-encoding-dependency/rdr_9ea390b56fdd4c9fb5f97329001f438f},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_positional_encoding_dependency_9ea390b5_v2,
title = {positional encoding dependency},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/positional-encoding-dependency/rdr_9ea390b56fdd4c9fb5f97329001f438f/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}