Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving

Yinwei Dai, Rui Pan, Anand Iyer, Kai Li, Ravi Netravali

BibTeX
@inproceedings{dai2024apparate,
  author = {Yinwei Dai and Rui Pan and Anand Iyer and Kai Li and Ravi Netravali},
  title = {{Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving}},
  booktitle = {SOSP},
  year = {2024},
  doi = {10.1145/3694715.3695963},
}
Figure