Towards Edge General Intelligence: Knowledge Distillation for Mobile Agentic AI

Zhaohui Yang, Yuxuan Wu, Linghan Ma, Ruichen Zhang, Yinqiu Liu, Dusit Niyato, Shunpu Tang, Zehui Xiong, Zhu Han, Zhaohui Yang 0001, Kaibin Huang, Zhaoyang Zhang, Kai-Kit WongarXiv 2025 · DOI · arXiv

Introduction

The landscape of Artificial Intelligence is shifting from passive pattern recognition to "agentic" systems—autonomous entities capable of perceiving their environment, planning complex sequences of actions, and learning from experience. However, a significant "deployment chasm" exists. The Large Language Models (LLMs) that serve as the cognitive engines for these agents are computationally expensive, while the mobile edge—comprising smartphones, drones, and IoT sensors—operates under strict power and memory constraints.

To bridge this gap, this paper explores Edge General Intelligence (EGI), positioning Knowledge Distillation (KD) as the critical mechanism for transferring the sophisticated reasoning capabilities of massive cloud-based models into compact, efficient "student" models deployable at the wireless edge.

The EGI Framework

The transition to EGI requires a systematic approach to model efficiency that preserves general intelligence. Rather than simple compression, the goal is capability transfer: ensuring that a tiny model on a device can mimic the sophisticated reasoning of a teacher model in the cloud.

Conceptual framework of Edge General Intelligence (EGI) powered by Knowledge Distillation for Mobile Agentic AI.

As illustrated above, the framework involves a hierarchy where large "Teacher" models optimize and compress their knowledge into "Student" models. This process empowers IoT edge systems—such as smart homes, autonomous vehicles, and UAVs—to evolve from simple nodes into mobile agentic AI capable of real-time adaptation and autonomous task execution.

Knowledge Distillation Paradigms

The paper categorizes the transfer of knowledge into three primary paradigms:

  1. Response-Based Distillation: The student mimics the teacher's final output distributions (soft targets), capturing nuanced class relationships.
  2. Feature-Based Distillation: The student aligns its internal representations (hidden layers) with those of the teacher, effectively learning the teacher's "thought process."
  3. Relation-Based Distillation: The student learns the structural relationships between different data samples as perceived by the teacher, enhancing generalization.

Beyond these standards, the authors highlight Agentic-specific KD, such as Policy Distillation for control logic and Chain-of-Thought (CoT) Distillation to teach smaller models procedural reasoning. Furthermore, Cross-Architecture Distillation is proposed to transfer knowledge from heavy Transformers to linear-time architectures like Mamba or RWKV.

The Agentic Workflow

A core tenet of EGI is the "Agentic Loop," which defines how an autonomous system interacts with the world. KD is essential at every stage of this cycle to ensure low-latency, local execution.

The workflow of an AI agent, illustrating the cycle of Perception, Planning, Action, and Memory.

The workflow consists of four pillars:

  • Perception: Processing physical modalities (sensors, cameras) and digital channels (APIs) to extract key features.
  • Memory: Utilizing short-term context and long-term retrieval-augmented generation (RAG) to maintain continuity and learn from past experiences.
  • Planning: Moving from static sequences to dynamic plans that update based on environmental feedback.
  • Action: Executing tasks via GUI interactions (clicking, typing) or API calls to modify device settings or retrieve data.

Key Results and Performance

Synthesis of current research shows that KD significantly mitigates the resource bottleneck at the edge:

  • Model Efficiency: TinyBERT, a distilled language model, achieved a 9.4x speedup while retaining 96.8% of the performance of its larger counterpart.
  • Wireless Edge: In Channel State Information (CSI) tasks, KD-trained models improved accuracy by 7.6 dB while operating at only 25-43% of the teacher's complexity.
  • UAV Autonomy: For UAVs, local execution of distilled planning models reduced task completion delays by up to 95% compared to cloud-offloading strategies.

Challenges and Future Directions

Despite its promise, EGI faces several hurdles. The "Explainability Crisis" remains a concern, as distilled models often lack transparency. Furthermore, security risks like "logit poisoning" in federated environments and the potential for students to amplify a teacher's "hallucinations" must be addressed. Finally, as agents gain autonomy at the edge, navigating the ethical labyrinth of algorithmic integrity and human value alignment becomes paramount. The future of EGI lies in moving beyond Transformers toward linear-time architectures, using KD to imbue these efficient structures with the pre-trained wisdom of the world's most powerful AI models.

Summary updated

Abstract

Edge General Intelligence (EGI) represents a paradigm shift in mobile edge computing, where intelligent agents operate autonomously in dynamic, resource-constrained environments. However, the deployment of advanced agentic AI models on mobile and edge devices faces significant challenges due to limited computation, energy, and storage resources. To address these constraints, this survey investigates the integration of Knowledge Distillation (KD) into EGI, positioning KD as a key enabler for efficient, communication-aware, and scalable intelligence at the wireless edge. In particular, we emphasize KD techniques specifically designed for wireless communication and mobile networking, such as channel-aware self-distillation, cross-model Channel State Information (CSI) feedback distillation, and robust modulation/classification distillation. Furthermore, we review novel architectures natively suited for KD and edge deployment, such as Mamba, RWKV (Receptance, Weight, Key, Value) and Cross-Architecture distillation, which enhance generalization capabilities. Subsequently, we examine diverse applications in which KD-driven architectures enable EGI across vision, speech, and multimodal tasks. Finally, we highlight the key challenges and future directions for KD in EGI. This survey aims to provide a comprehensive reference for researchers exploring KD-driven frameworks for mobile agentic AI in the era of EGI.
BibTeX
@misc{yang2025towards,
  author = {Zhaohui Yang and Yuxuan Wu and Linghan Ma and Ruichen Zhang and Yinqiu Liu and Dusit Niyato and Shunpu Tang and Zehui Xiong and Zhu Han and Zhaohui Yang and Kaibin Huang and Zhaoyang Zhang and Kai-Kit Wong},
  title = {{Towards Edge General Intelligence: Knowledge Distillation for Mobile Agentic AI}},
  year = {2025},
  eprint = {2511.19947},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.19947},
}
Figure