No article text was recorded for this version.
causal attention workload imbalance
Version 1 of 116 papers33 conceptsfrozen 2026-10-04
Papers in this version
- Blog: KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
- Blog: Compute or Load KV Cache? Why Not Both?
- Blog: TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models
- Blog: DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
- Blog: FoldMoE: Efficient Long Sequence MoE Training via Attention-MoE Pipelining
- Blog: HPipe: Large Language Model Pipeline Parallelism for Long Context on Heterogeneous Cost-effective Devices
- Blog: Hierachical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
- Blog: Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
- Blog: WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training
- Blog: Efficient Distributed MLLM Training with Cornstarch
- Blog: FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
- Blog: Revisiting Parameter Server in LLM Post-Training
- Blog: Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment
- Blog: Bi-semantic Chemical Embedder for Joint Representation Learning of SMILES and Natural Language
- Blog: Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognition
- Blog: StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
History & citation
Current · Permanent link
- This direction split from Load Balancing for Long Context Transformers (81% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_causal_attention_workload_imbalance_5fbe9d7c,
title = {causal attention workload imbalance},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/causal-attention-workload-imbalance/rdr_5fbe9d7cce2847c1ac1ceef76c5cd8e3},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_causal_attention_workload_imbalance_5fbe9d7c_v1,
title = {causal attention workload imbalance},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/causal-attention-workload-imbalance/rdr_5fbe9d7cce2847c1ac1ceef76c5cd8e3/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}