No article text was recorded for this version.
communication-computation overlap (computation-communication overlap)
Version 1 of 134 papers46 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Deep Learning at Scale and at Ease
- Blog: ACCO: Accumulate while you Communicate, Hiding Communications in Distributed LLM Training
- Blog: Scaling Neural Machine Translation
- Blog: Distributed Training Large-Scale Deep Architectures
- Blog: Scalable Distributed DL Training: Batching Communication and Computation
- Blog: Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD
- Blog: Streaming DiLoCo with overlapping communication
- Blog: MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms
- Blog: Stochastic Gradient Push for Distributed Deep Learning
- Blog: PyTorch Distributed: Experiences on Accelerating Data Parallel Training
- Blog: Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
- Blog: Distributed Deep Learning In Open Collaborations
- Blog: Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs
- Blog: ZeCO: Zero-Communication Overhead Sequence Parallelism for Linear Attention
- Blog: Convolutional Neural Network Training with Distributed K-FAC
- Blog: A Decentralized Parallel Algorithm for Training Generative Adversarial Nets
- Blog: MassiveGNN: Efficient Training via Prefetching for Massively Connected Distributed Graphs
- Blog: PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
- Blog: CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters
- Blog: MGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication-Computation Pipelining on Multi-GPU Platforms
- Blog: LoCo: Local Contrastive Representation Learning
- Blog: Priority-based Parameter Propagation for Distributed DNN Training
- Blog: Pretraining Billion-Scale Geospatial Foundational Models on Frontier
- Blog: On the Utility of Gradient Compression in Distributed Training Systems
- Blog: Memory-Efficient Split Federated Learning for LLM Fine-Tuning on Heterogeneous Mobile Devices
- Blog: Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion
- Blog: RingAttention with Blockwise Transformers for Near-Infinite Context
- Blog: ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
- Blog: Characterizing Compute-Communication Overlap in GPU-Accelerated Distributed Deep Learning: Performance and Power Implications
- Blog: MegaScale: Scaling Large Language Model Training to More Than 10, 000 GPUs
- Blog: Characterizing Deep Learning Training Workloads on Alibaba-PAI
- Blog: Faster Distributed Deep Net Training: Computation and Communication Decoupled Stochastic Gradient Descent
- Blog: SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training
- Blog: Evaluation of pre-training large language models on leadership-class supercomputers
History & citation
Current · Permanent link
- This direction split from Overlapping Communication and Computation in Distributed Deep Learning (76% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_communication_computation_overlap_comput_9ecd6c67,
title = {communication-computation overlap (computation-communication overlap)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/communication-computation-overlap-computation-communication-overlap/rdr_9ecd6c6772d441f4b86e3d401559908c},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_communication_computation_overlap_comput_9ecd6c67_v1,
title = {communication-computation overlap (computation-communication overlap)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/communication-computation-overlap-computation-communication-overlap/rdr_9ecd6c6772d441f4b86e3d401559908c/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}