No article text was recorded for this version.
input embedding rank bottleneck
Version 1 of 19 papers16 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Which transformer architecture fits my data? A vocabulary bottleneck in self-attention
- Blog: LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
- Blog: Which transformer architecture fits my data? A vocabulary bottleneck in self-attention
- Blog: A case where a spindly two-layer linear network decisively outperforms any neural network with a fully connected input layer
- Blog: Bayes-optimal Learning of Deep Random Networks of Extensive-width
- Blog: Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
- Blog: Bayes-optimal learning of an extensive-width neural network from quadratically many samples
- Blog: Redesigning Embedding Layers for Queries, Keys, and Values in Cross-Covariance Image Transformers
- Blog: Bayes-optimal Learning of Deep Random Networks of Extensive-width
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_input_embedding_rank_bottleneck_d31b27f1,
title = {input embedding rank bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/input-embedding-rank-bottleneck/rdr_d31b27f1fbf6412991dc40407f866453},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_input_embedding_rank_bottleneck_d31b27f1_v1,
title = {input embedding rank bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/input-embedding-rank-bottleneck/rdr_d31b27f1fbf6412991dc40407f866453/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}