No article text was recorded for this version.
causal attention bottleneck
Version 1 of 184 papers91 conceptsfrozen 2026-10-04
Papers in this version
- Blog: The Impossibility of Inverse Permutation Learning in Transformer Models
- Blog: Examining Scaling and Transfer of Language Model Architectures for Machine Translation
- Blog: LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
- Blog: The Mystery of the Pathological Path-star Task for Language Models
- Blog: CausalLM is not optimal for in-context learning
- Blog: GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph
- Blog: Weakly-Supervised Generation and Grounding of Visual Descriptions with Conditional Generative Models
- Blog: HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
- Blog: VidLaDA: Bidirectional Diffusion Large Language Models for Efficient Video Understanding
- Blog: Length-Induced Embedding Collapse in Transformer-based Models
- Blog: ModernVBERT: Towards Smaller Visual Document Retrievers
- Blog: Leveraging Decoder Architectures for Learned Sparse Retrieval
- Blog: Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation
- Blog: Thinking Like Transformers
- Blog: Decoder-Only LLMs can be Masked Auto-Encoders
- Blog: Mamba Retriever: Utilizing Mamba for Effective and Efficient Dense Retrieval
- Blog: Generative Representational Instruction Tuning
- Blog: Simple Context Compression: Mean-Pooling and Multi-Ratio Training
- Blog: Directed-Tokens: A Robust Multi-Modality Alignment Approach to Large Language-Vision Models
- Blog: TimelyGPT: Recurrent Convolutional Transformer for Long Time-series Representation
- Blog: Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context
- Blog: Improving Input-label Mapping with Demonstration Replay for In-context Learning
- Blog: Characterizing the Expressivity of Fixed-Precision Transformer Language Models
- Blog: Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More
- Blog: Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation
- Blog: Large Language Models Are Reasoning Teachers
- Blog: DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers
- Blog: Re-Reading Improves Reasoning in Large Language Models
- Blog: Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings
- Blog: MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators
- Blog: EEL: Efficiently Encoding Lattices for Reranking
- Blog: What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization?
- Blog: How Do Hyenas Deal with Human Speech? Speech Recognition and Translation with ConfHyena
- Blog: MAGNET: Augmenting Generative Decoders with Representation Learning and Infilling Capabilities
- Blog: Denoising Autoregressive Representation Learning
- Blog: Scaling Sparse and Dense Retrieval in Decoder-Only LLMs
- Blog: Thinking Like Transformers
- Blog: NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
- Blog: Two-Way Neural Machine Translation: A Proof of Concept for Bidirectional Translation Modeling Using a Two-Dimensional Grid
- Blog: Universal Conditional Masked Language Pre-training for Neural Machine Translation
- Blog: Future-Guided Incremental Transformer for Simultaneous Translation
- Blog: The Counting Power of Transformers
- Blog: LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendation
- Blog: Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context
- Blog: Tighter Bounds on the Expressivity of Transformer Encoders
- Blog: Contrastive Learning with Bidirectional Transformers for Sequential Recommendation
- Blog: Just read twice: closing the recall gap for recurrent language models
- Blog: Closing the Data-Efficiency Gap Between Autoregressive and Masked Diffusion LLMs
- Blog: OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses
- Blog: Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation
- Blog: ENTP: Encoder-only Next Token Prediction
- Blog: Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings
- Blog: Nemotron ColEmbed V2: Top-Performing Late Interaction embedding models for Visual Document Retrieval
- Blog: When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality
- Blog: NoPE: The Counting Power of Transformers with No Positional Encodings
- Blog: Masked AutoDecoder is Effective Multi-Task Vision Generalist
- Blog: Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling
- Blog: LLM-supertagger: Categorial Grammar Supertagging via Large Language Models
- Blog: CSE-SFP: Enabling Unsupervised Sentence Representation Learning via a Single Forward Pass
- Blog: Insertion Transformer: Flexible Sequence Generation via Insertion Operations
Showing 60 of 84 papers.
History & citation
Current · Permanent link
- This direction split from Overcoming the Causal Bottleneck in Generative Transformers (85% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_causal_attention_bottleneck_e01151ab,
title = {causal attention bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/causal-attention-bottleneck/rdr_e01151ab0c294c56a575845867732b06},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 1, frozen text):
@misc{lacuna_causal_attention_bottleneck_e01151ab_v1,
title = {causal attention bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/causal-attention-bottleneck/rdr_e01151ab0c294c56a575845867732b06/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}