DASR: Distributed Adaptive Scene Recognition - A Multi-Agent Cloud-Edge Framework for Language-Guided Scene Detection

Can Cui, Yongkang Liu, Seyhan Uçar, Juntong Peng, Ahmadreza Moradipari, Maryam Khabazi, Ziran Wang2025 · DOI

TL;DR

This paper introduces DASR, a novel cloud-edge framework for autonomous vehicles that uses a multi-agent system to intelligently identify and store critical driving scenarios. By leveraging a cloud-based LLM for prompt interpretation and an edge-based VLM for real-time decision-making, DASR significantly reduces data storage needs while improving the accuracy of scene recognition for ADAS development.

What Problem Does This Paper Address?

Developing and validating advanced driver-assistance systems (ADAS) for autonomous vehicles requires analyzing specific, often rare, driving scenarios. Traditional methods involve either collecting massive amounts of driving data and manually sifting through it, or using rigid, threshold-based recording systems that can miss crucial "edge cases." These approaches are inefficient and can lead to missed opportunities for improving system safety and performance. Furthermore, the computational demands of state-of-the-art vision-language models (VLMs) make real-time, on-vehicle deployment challenging, creating a bottleneck in the development cycle.

Methods

DASR proposes a multi-agent, cloud-edge collaboration framework. The core idea is to distribute the workload:

  • Cloud-based LLM Coordinator: A powerful LLM (GPT-40) acts as the central coordinator. It receives natural language prompts from developers (e.g., "show me emergency braking scenarios") and analyzes them to determine which specialized tools (e.g., object detector, distance estimator) and sensor data streams (e.g., IMU, CAN bus) are necessary for accurate scene identification. This LLM also categorizes the task difficulty and selects the most relevant inputs to avoid overwhelming the system.
  • Edge-deployed VLM: A lightweight VLM (Qwen2-VL-2B, fine-tuned and quantized to 4-bit for efficiency) runs on the vehicle. It receives the processed information from the cloud LLM and the selected edge tools. This VLM then makes real-time decisions about whether the current scene matches the developer's prompt.
  • Edge Toolbox: A set of specialized tools (Object Detector, Weather Detector, Vehicle Dynamics Monitor, Distance Estimator, Time-of-Day Detector) are available on the edge device. These tools provide specific, quantitative data to enhance the VLM's scene understanding.

This architecture allows for intelligent data collection, preserving only the most relevant video frames and sensor data, thereby optimizing limited on-vehicle buffer storage.

Datasets

  • Nuplan-QA-Eval: Used for fine-tuning the VLM. The authors restructured this dataset from a multiple-choice format to a binary question-answering format.
  • DRAMA (Tokyo, Japan): Used as an out-of-distribution (OOD) test set.
  • HAD (San Francisco, USA): Used as another OOD test set.

The DRAMA and HAD datasets were converted into a balanced yes/no question format for evaluation.

Key Insights and Ideas

  • Intelligent Data Filtering: The most significant insight is the effectiveness of a hierarchical, multi-agent approach for filtering driving data. By using a cloud LLM to interpret complex prompts and guide edge tools, the system can precisely identify relevant scenarios, moving beyond rigid, predefined thresholds.
  • Cloud-Edge Synergy: The paper demonstrates the power of combining the broad reasoning capabilities of cloud LLMs with the real-time processing power of efficient edge VLMs. This synergy allows for sophisticated scene understanding without requiring massive on-vehicle computational resources.
  • Specialist Augmentation: Integrating domain-specific "specialists" (tools) significantly enhances the VLM's ability to understand nuanced scenarios, proving that a modular approach can outperform monolithic models for complex tasks.
  • Storage Optimization: The framework's ability to drastically reduce storage requirements (by 28.73%) is a critical practical benefit, directly addressing the limitations of on-vehicle hardware.

Key Results

  • Superior Scene Recognition: DASR achieved the highest total precision (91.35%) and mean average precision across various baselines, outperforming even larger models.
  • Strong Generalization: The framework demonstrated exceptional performance on out-of-distribution datasets, achieving 91.93% precision on DRAMA and 77.68% on HAD, indicating robust transferability across different driving environments.
  • Significant Storage Reduction: DASR reduced storage requirements by an average of 28.73% compared to baseline methods, making more efficient use of limited on-vehicle buffer capacity.
  • Efficiency: The use of a quantized, fine-tuned Qwen2-VL-2B model ensures efficient deployment on edge devices.

Limitations and Future Work

  • Cyber Security Risk: The cloud-edge architecture introduces potential vulnerabilities due to communication channels between the vehicle and the cloud, which could be targets for attacks.
  • Network Dependency: Consistent connectivity between the vehicle and the cloud infrastructure is crucial. Limited network coverage or interruptions can temporarily impair the system's ability to identify valuable data.
  • Limited Multi-Modal Integration: The current framework primarily focuses on visual data. Future work could explore integrating other sensor modalities like LiDAR and radar for more comprehensive scene understanding.

The authors suggest that these improvements will lead to faster validation cycles for new ADAS features and accelerate time-to-market.

Summary updated

BibTeX
@misc{cui2025dasr,
  author = {Can Cui and Yongkang Liu and Seyhan Uçar and Juntong Peng and Ahmadreza Moradipari and Maryam Khabazi and Ziran Wang},
  title = {{DASR: Distributed Adaptive Scene Recognition - A Multi-Agent Cloud-Edge Framework for Language-Guided Scene Detection}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-industry.57},
}
Figure