Lightweight Expert Pruning for Edge-Deployable MoE LLMs

Lujun LiOpenReview 2026 · OpenReview

Abstract

Deploying MoE LLMs on edge devices is challenging due to their large memory footprint and high computational cost. Expert pruning is a key compression technique, but existing methods often require complex retraining or large computational resources, making them unsuitable for edge deployment. To address this, we propose Lightweight Expert Pruning (LEP), a fast, training-free pruning framework tailored for edge-deployable MoE LLMs. LEP leverages two lightweight components: 1) a Fast Expert Importance Estimator (FEIE) that computes expert importance using only a small calibration dataset (1% of the training data) and avoids gradient computations, reducing pruning time by 70% compared to gradient-based methods; 2) a Greedy Pruning Algorithm (GPA) that iteratively prunes the least important experts, with a stopping criterion based on memory and latency constraints (e.g., target memory usage for edge devices). LEP also includes a Lightweight Router Adjustment (LRA) module that fine-tunes the router with minimal computational cost (≤5% of full retraining). Experiments on MoE-LLMs show that LEP prunes 50% of experts in under 1 hour (on a single GPU), maintaining 98.5% of the full-model performance on edge-relevant tasks (e.g., text classification, small-scale QA). LEP reduces the model’s memory footprint by 48% and inference latency by 39%, making it suitable for deployment on edge devices such as smartphones and IoT devices.
BibTeX
@misc{li2026lightweight,
  author = {Lujun Li},
  title = {{Lightweight Expert Pruning for Edge-Deployable MoE LLMs}},
  year = {2026},
  url = {https://openreview.net/forum?id=swVgH46pif},
  howpublished = {OpenReview},
}

Loading related content...

Figure