No article text was recorded for this version.
softmax bottleneck (softmax rank bottleneck)
Version 2 of 224 papers45 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
- Blog: Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck
- Blog: Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation
- Blog: Sigsoftmax: Reanalysis of the Softmax Bottleneck
- Blog: The Softmax Bottleneck Does Not Limit the Probabilities of the Most Likely Tokens
- Blog: Closing the Curious Case of Neural Text Degeneration
- Blog: Why do Nearest Neighbor Language Models Work?
- Blog: Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond
- Blog: Tensorized Embedding Layers
- Blog: Stolen Probability: A Structural Weakness of Neural Language Models
- Blog: On Retrieval Augmentation and the Limitations of Language Model Training
- Blog: On the Softmax Bottleneck of Recurrent Language Models
- Blog: Knowledge Distillation For Recurrent Neural Network Language Modeling With Trust Regularization
- Blog: Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions
- Blog: Deep Residual Output Layers for Neural Language Generation
- Blog: Improving Neural Language Modeling via Adversarial Training
- Blog: On Efficiently Representing Regular Languages as RNNs
- Blog: The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?
- Blog: HistAlign: Improving Context Dependency in Language Generation by Aligning with History
- Blog: Joint Syntax Representation Learning and Visual Cue Translation for Video Captioning
- Blog: Vec2Sent: Probing Sentence Embeddings with Natural Language Generation
- Blog: Direct Output Connection for a High-Rank Language Model
- Blog: Logits of API-Protected LLMs Leak Proprietary Information
- Blog: Mixtape: Breaking the Softmax Bottleneck Efficiently
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_softmax_bottleneck_softmax_rank_bottlene_22a794f2,
title = {softmax bottleneck (softmax rank bottleneck)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/softmax-bottleneck-softmax-rank-bottleneck/rdr_22a794f2de4c49a7b0cbc0819c71b3ae},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 2, frozen text):
@misc{lacuna_softmax_bottleneck_softmax_rank_bottlene_22a794f2_v2,
title = {softmax bottleneck (softmax rank bottleneck)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/softmax-bottleneck-softmax-rank-bottleneck/rdr_22a794f2de4c49a7b0cbc0819c71b3ae/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}