No article text was recorded for this version.
low-rank attention bottleneck
Version 1 of 127 papers32 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Low-Rank Bottleneck in Multi-head Attention Models
- Blog: MHLA: Restoring Expressivity of Linear Attention via Token-Level Multi-Head
- Blog: Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis
- Blog: More Identifiable yet Equally Performant Transformers for Text Classification
- Blog: Customizing the Inductive Biases of Softmax Attention using Structured Matrices
- Blog: Improving Transformers with Dynamically Composable Multi-Head Attention
- Blog: Bidirectional Generative Pre-training for Improving Healthcare Time-series Representation Learning
- Blog: Tensor Decomposition Based Attention Module for Spiking Neural Networks
- Blog: Multi Resolution Analysis (MRA) for Approximate Self-Attention
- Blog: A Simple and Effective Positional Encoding for Transformers
- Blog: Quality over Quantity in Attention Layers: When Adding More Heads Hurts
- Blog: LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning
- Blog: Learning Hard Retrieval Decoder Attention for Transformers
- Blog: Improving Transformers with Probabilistic Attention Keys
- Blog: Adder Attention for Vision Transformer
- Blog: Luna: Linear Unified Nested Attention
- Blog: Decoupled Side Information Fusion for Sequential Recommendation
- Blog: Combiner: Full Attention Transformer with Sparse Computation Cost
- Blog: Bayes optimal learning of attention-indexed models
- Blog: FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting
- Blog: Weight decay induces low-rank attention layers
- Blog: Higher Order Transformers: Efficient Attention Mechanism for Tensor Structured Data
- Blog: Improving Transformers with Probabilistic Attention Keys
- Blog: Higher Order Transformers With Kronecker-Structured Attention
- Blog: Efficient Folded Attention for Medical Image Reconstruction and Segmentation
- Blog: Paramixer: Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product Self-Attention
- Blog: Multi Resolution Analysis (MRA) for Approximate Self-Attention
History & citation
Current · Permanent link
- This direction split from The Low-Rank Bottleneck in Efficient Attention (70% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_low_rank_attention_bottleneck_bb8cb9b3,
title = {low-rank attention bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/low-rank-attention-bottleneck/rdr_bb8cb9b3378f42efb9a7045d3dda39d5},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_low_rank_attention_bottleneck_bb8cb9b3_v1,
title = {low-rank attention bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/low-rank-attention-bottleneck/rdr_bb8cb9b3378f42efb9a7045d3dda39d5/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}