MPMoE: Memory Efficient MoE for Pre-Trained Models With Adaptive Pipeline Parallelism

Dacheng Tao, Zheng Zhang, Yaqi Xia, Hulin Wang, Donglin Yang, Chuang Hu, Xiaobo Zhou, Dazhao ChengIEEE TPDS 2024 · DOI

BibTeX
@article{tao2024mpmoe,
  author = {Dacheng Tao and Zheng Zhang and Yaqi Xia and Hulin Wang and Donglin Yang and Chuang Hu and Xiaobo Zhou and Dazhao Cheng},
  title = {{MPMoE: Memory Efficient MoE for Pre-Trained Models With Adaptive Pipeline Parallelism}},
  journal = {IEEE TPDS},
  year = {2024},
  doi = {10.1109/tpds.2024.3385639},
}

Loading related content...

Figure