Lightweight Expert Pruning for Edge-Deployable MoE LLMs
Lists
Only you see this
Abstract
Deploying MoE LLMs on edge devices is challenging due to their large memory footprint and high computational cost. Expert pruning is a key compression technique, but existing methods often require complex retraining or large computational resources, making them unsuitable for edge deployment. To address this, we propose Lightweight Expert Pruning (LEP), a fast, training-free pruning framework tailored for edge-deployable MoE LLMs. LEP leverages two lightweight components: 1) a Fast Expert Importance Estimator (FEIE) that computes expert importance using only a small calibration dataset (1% of the training data) and avoids gradient computations, reducing pruning time by 70% compared to gradient-based methods; 2) a Greedy Pruning Algorithm (GPA) that iteratively prunes the least important experts, with a stopping criterion based on memory and latency constraints (e.g., target memory usage for edge devices). LEP also includes a Lightweight Router Adjustment (LRA) module that fine-tunes the router with minimal computational cost (≤5% of full retraining). Experiments on MoE-LLMs show that LEP prunes 50% of experts in under 1 hour (on a single GPU), maintaining 98.5% of the full-model performance on edge-relevant tasks (e.g., text classification, small-scale QA). LEP reduces the model’s memory footprint by 48% and inference latency by 39%, making it suitable for deployment on edge devices such as smartphones and IoT devices.
BibTeX
@misc{li2026lightweight,
author = {Lujun Li},
title = {{Lightweight Expert Pruning for Edge-Deployable MoE LLMs}},
year = {2026},
url = {https://openreview.net/forum?id=swVgH46pif},
howpublished = {OpenReview},
}Loading related content...