GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant

Zhuokang Shen, Yifan Wang, Hanyu Chen, Yunhang Shen, Wenxuan Huang, Gaoqi He, Jiao Xie, Rongrong Ji, Shaohui LinarXiv 2026 · arXiv

TL;DR

GroupGPT is a novel multi-agent framework designed to bring intelligent, proactive AI assistants into multi-user group chats efficiently and securely. By decoupling the decision of when to speak from what to say using an edge-cloud collaboration architecture, it slashes token costs by up to 3×, protects user privacy via local data sanitization, and introduces the first real-world group chat intervention benchmark, MUIR.

What Problem Does This Paper Address?

Most modern AI chatbots are designed for single-user, turn-based interactions. When dropped into a multi-user group chat (like Discord, Slack, or WeChat), these systems struggle. A successful group chat assistant must be proactive—knowing exactly when to chime in with emotional support, fact corrections, or suggestions without disrupting the natural flow of human conversation.

Existing approaches typically route every single message through a massive cloud-based Large Language Model (LLM) to evaluate whether an intervention is needed. This creates three major bottlenecks:

  1. High Token Consumption: Constantly querying cloud LLMs for every message in a fast-moving chat is prohibitively expensive and scales poorly.
  2. Privacy Risks: Sending raw, un-sanitized group chat logs containing personally identifiable information (PII) to external cloud servers raises severe data security concerns.
  3. Lack of Multimodal Support: Real-world group chats are messy and filled with memes, images, videos, and voice notes, which standard text-only frameworks ignore.

Methods

GroupGPT addresses these challenges using an edge-cloud model collaboration architecture that splits the workload between lightweight models running locally (on the edge) and powerful models running in the cloud.

Figure 2: GroupGPT can identify the right moment to chime in, rewrite sensitive personally identifiable information(gray part), understand multimodal content such as memes, and actively participate in group discussions.

The framework consists of five specialized sub-agents:

  • Intervention Judge: A lightweight Small Language Model (SLM), specifically a fine-tuned Qwen-3-4B, that runs locally. It continuously monitors the chat stream and decides if the bot should intervene or "Stay Silent" based on six conversational roles (e.g., Fact Correction, Emotional Support).
  • Privacy Transcriber: A local SLM (Llama-3.2-3B) that scans incoming messages for PII, rewriting sensitive details (e.g., changing a specific name to "my partner") before any data leaves the local device.
  • Multimodal Message Processor: Converts non-textual inputs like memes, videos, and audio into structured text tags so the text-based models can understand them.
  • Chat Frequency Logger: Tracks conversation velocity and user participation metrics to help the bot understand the chat's dynamics.
  • Final Respondent: A powerful cloud LLM (GPT-4o) that is only activated when the local Intervention Judge decides it is time to speak. It takes the sanitized history and context to generate the final response.

Datasets

Because public, high-quality group chat datasets are rare due to privacy restrictions, the authors introduced MUIR (Multi-user chat assistant Intervention Reasoning).

MUIR is the first benchmark dataset for group chat intervention featuring human-annotated rationales. It contains 2,500 group chat segments spanning 50 real-world scenarios (e.g., gaming, cooking, technical support, emotional wellness). The dataset is split into 2,000 training instances and 500 test instances, annotated using a hybrid pipeline of GPT-4o labeling and rigorous human verification.

Key Insights and Ideas

  • Smaller Models are Better Timers: Surprisingly, massive proprietary models like GPT-4o perform poorly at deciding when to chime in, often exhibiting overly conservative behavior (achieving less than 67% accuracy on timing). Local, fine-tuned SLMs significantly outperform them at managing conversational timing.
  • Decoupling Saves Massive Resources: By filtering out the noise locally and only invoking the cloud LLM when a response is genuinely required, the framework avoids massive API overhead.
  • Privacy as a Pre-processing Step: Local PII sanitization proves that privacy preservation doesn't have to compromise the intelligence of the final response, effectively decoupling data security from cloud capabilities.

Key Results

  • 3× Token Reduction: GroupGPT achieves an approximate 3× reduction in token usage compared to standard LLM-only baselines, dramatically lowering operational API costs.
Figure 3: Token consumption comparison.
  • State-of-the-Art Timing: The fine-tuned Qwen-3-4B Intervention Judge achieved a Chime-in Timing accuracy of 83.40% and an F1 score of 88.67%, vastly outperforming vanilla GPT-4o's 58.02% accuracy on the MUIR benchmark.
  • High Response Quality: In LLM-as-a-judge evaluations, GroupGPT scored an average of 4.72/5.0 across relevance, coherence, fluency, and helpfulness.
  • Low Hardware Requirements: The entire local pipeline runs efficiently on consumer-grade hardware, drawing a total GPU memory footprint of just 18.41 GB.

Limitations and Future Work

While highly effective, the authors note a few limitations and areas for future exploration:

  • Information Loss: The Privacy Transcriber can occasionally introduce minor information loss during the rewriting process, which marginally impacts the context of the final response.
  • Group-Level Personalization: Current models struggle with "collective memory"—remembering shared concepts, past images, or specific group dynamics over long periods.
  • Advanced Multimodal Nuance: Future iterations need a deeper understanding of casual content like memes versus semantically critical images to improve contextual reasoning.

Summary updated

Abstract

Recent advances in large language models (LLMs) have enabled increasingly capable chatbots. However, most existing systems focus on single-user settings and do not generalize well to multi-user group chat interactions, where agents require more proactive and accurate intervention under complex, evolving contexts. Existing approaches typically rely on LLMs for both intervention reasoning and response generation, leading to high token consumption, limited scalability, and potential privacy risks. To address these challenges, we propose GroupGPT, a token-efficient and privacy-preserving agentic framework for multi-user chat assistant. GroupGPT adopts an edge-cloud model collaboration architecture to decouple intervention timing from response generation, enabling efficient and accurate decision-making while preserving user privacy through on-device processing of sensitive information. The framework also supports multimodal inputs, including memes, images, videos, and voice messages.To support evaluation of timing accuracy and response quality, we further introduce MUIR, a benchmark dataset for multi-user chat assistant intervention reasoning. MUIR contains 2,500 annotated group chat segments with intervention labels and rationales. We evaluate a range of models on MUIR, spanning from open-source to proprietary variants, including both LLMs and their smaller counterparts. Extensive experiments demonstrate that GroupGPT generates accurate and well-timed responses, achieving an average score of 4.72/5.0 in LLM-based evaluation, and is well-received by users across diverse group chat scenarios. Moreover, GroupGPT reduces the token usage by up to 3 times compared to baselines, while providing privacy sanitization of user messages before cloud transmission. Code is available at: https://github.com/Eliot-Shen/GroupGPT .
BibTeX
@misc{shen2026groupgpt,
  author = {Zhuokang Shen and Yifan Wang and Hanyu Chen and Yunhang Shen and Wenxuan Huang and Gaoqi He and Jiao Xie and Rongrong Ji and Shaohui Lin},
  title = {{GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant}},
  year = {2026},
  eprint = {2603.01059},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.01059},
}

Loading related content...

Figure