No article text was recorded for this version.
query-key normalization
Version 2 of 2198 papers243 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks
- Blog: The Devil in Linear Transformer
- Blog: The Devil in Linear Transformer
- Blog: Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks
- Blog: Representation Mixing for TTS Synthesis
- Blog: Normalized and Geometry-Aware Self-Attention Network for Image Captioning
- Blog: EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models
- Blog: Chameleon: Mixed-Modal Early-Fusion Foundation Models
- Blog: Linear Transformers with Learnable Kernel Functions are Better In-Context Models
- Blog: Swin Transformer V2: Scaling Up Capacity and Resolution
- Blog: Query-Key Normalization for Transformers
- Blog: Self-supervised Learning of Contextualized Local Visual Embeddings
- Blog: DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD
- Blog: Tensorformer: Normalized Matrix Attention Transformer for High-Quality Point Cloud Reconstruction
- Blog: Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method
- Blog: STIV: Scalable Text and Image Conditioned Video Generation
- Blog: FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Model
- Blog: Lumina-Next : Making Lumina-T2X Stronger and Faster with Next-DiT
- Blog: Small-scale proxies for large-scale Transformer training instabilities
- Blog: Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
- Blog: Normalized and Geometry-Aware Self-Attention Network for Image Captioning
- Blog: An Initial Attempt of Combining Visual Selective Attention with Deep Reinforcement Learning
- Blog: Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems
- Blog: LAWCAT: Efficient Distillation from Quadratic to Linear Attention with Convolution across Tokens for Long Context Modeling
- Blog: DINT Transformer
- Blog: Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
- Blog: Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
- Blog: Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
- Blog: Boosting Vanilla Lightweight Vision Transformers via Re-parameterization
- Blog: Linearizing Vision Transformer with Test-Time Training
- Blog: LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems
- Blog: MixA: A Mixed Attention approach with Stable Lightweight Linear Attention to enhance Efficiency of Vision Transformers at the Edge
- Blog: Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
- Blog: TaylorShift: Shifting the Complexity of Self-Attention from Squared to Linear (and Back) using Taylor-Softmax
- Blog: Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method
- Blog: Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
- Blog: Rethinking Softmax: Self-Attention with Polynomial Activations
- Blog: Multi-Label Visual Feature Learning with Attentional Aggregation
- Blog: The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity
- Blog: Demystify Mamba in Vision: A Linear Attention Perspective
- Blog: CVTGAD: Simplified Transformer with Cross-View Attention for Unsupervised Graph-level Anomaly Detection
- Blog: Linearizing Large Language Models
- Blog: Softmax-Free Linear Transformers
- Blog: Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation
- Blog: Softmax-free Linear Transformers
- Blog: Contextual Learning in Fourier Complex Field for VHR Remote Sensing Images
- Blog: Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation
- Blog: Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
- Blog: MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs
- Blog: RestoRect: Degraded Image Restoration via Latent Rectified Flow & Feature Distillation
- Blog: Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
- Blog: Graph external attention enhanced transformer
- Blog: Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
- Blog: PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels
- Blog: ReGLA: Refining Gated Linear Attention
- Blog: XGen-7B Technical Report
- Blog: Open-sci-ref-0.01: open and reproducible reference baselines for language model and dataset comparison
- Blog: Differential Transformer
- Blog: Scaled ReLU Matters for Training Vision Transformers
- Blog: Multi-Token Attention
Showing 60 of 198 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_query_key_normalization_7d174621,
title = {query-key normalization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/query-key-normalization/rdr_7d17462153b84c1eb19ddfd8e215f8b8},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_query_key_normalization_7d174621_v2,
title = {query-key normalization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/query-key-normalization/rdr_7d17462153b84c1eb19ddfd8e215f8b8/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}