No article text was recorded for this version.
topology-aware communication
Version 1 of 145 papers59 conceptsfrozen 2026-10-04
Papers in this version
- Blog: MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
- Blog: Faster and Cheaper: Parallelizing Large-Scale Matrix Factorization on GPUs
- Blog: Two Tiered Distributed Training Algorithm for Acoustic Modeling
- Blog: FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models
- Blog: Rail-only: A Low-Cost High-Performance Network for Training LLMs with Trillion Parameters
- Blog: High-Performance Distributed RMA Locks
- Blog: Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models
- Blog: GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
- Blog: Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models
- Blog: Accelerating Geo-distributed Machine Learning with Network-Aware Adaptive Tree and Auxiliary Route
- Blog: BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
- Blog: NetMoE: Accelerating MoE Training through Dynamic Sample Placement
- Blog: swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight
- Blog: Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale
- Blog: PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models
- Blog: Communication Optimization for Distributed Training: Architecture, Advances, and Opportunities
- Blog: Optimizing Communication in 2D Grid-Based MPI Applications at Exascale
- Blog: Bandwidth optimal all-reduce algorithms for clusters of workstations
- Blog: Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
- Blog: Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect
- Blog: Parallel deep neural network training for LVCSR tasks using blue gene/Q
- Blog: ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive
- Blog: Node aware sparse matrix-vector multiplication
- Blog: SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
- Blog: MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
- Blog: PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
- Blog: Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms
- Blog: Improving MPI Collective I/O for High Volume Non-Contiguous Requests With Intra-Node Aggregation
- Blog: Configurable Low-Latency Interconnect for Multi-core Clusters
- Blog: Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
- Blog: Optimizing Distributed Training on Frontier for Large Language Models
- Blog: Analyzing the impact of the MPI allreduce in distributed training of convolutional neural networks
- Blog: HyPar: Towards Hybrid Parallelism for Deep Learning Accelerator Array
- Blog: Efficient large-scale language model training on GPU clusters using megatron-LM
- Blog: The Design of a Marker Passing Architecture for Knowledge Processing
- Blog: Snap ML: A Hierarchical Framework for Machine Learning
- Blog: Exascale deep learning for climate analytics
- Blog: Efficient Direct-Connect Topologies for Collective Communications
- Blog: SCoMoE: Efficient Mixtures of Experts with Structured Communication
- Blog: An Evaluation of DRete on CUPlD for OPSS Matching
- Blog: Don't Use Large Mini-batches, Use Local SGD
- Blog: DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs
- Blog: StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
- Blog: DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
- Blog: Communication-Optimal Parallel Algorithm for Strassen's Matrix Multiplication
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_topology_aware_communication_ee9133f2,
title = {topology-aware communication},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/topology-aware-communication/rdr_ee9133f21cca4ec4a790dd074fd90316},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_topology_aware_communication_ee9133f2_v1,
title = {topology-aware communication},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/topology-aware-communication/rdr_ee9133f21cca4ec4a790dd074fd90316/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}