YOLO-based Semantic Communication with Generative AI-aided Resource Allocation for Digital Twins Construction

Baoxia Du, Hongyang Du, Haifeng Liu, Dusit Niyato, Peng Xin, Jun Yu, Mingyang Qi, You Tang

Introduction

Digital Twins (DTs) are essential for real-time monitoring and management in precision agriculture. However, maintaining high-fidelity digital replicas of environments like apple orchards requires constant data synchronization. Traditional methods involving high-resolution image transmission from Unmanned Aerial Vehicles (UAVs) to edge servers face significant challenges due to bandwidth constraints and power consumption in 6G networks. Much of the captured data—such as leaves and sky—is redundant for the specific task of apple monitoring. This paper proposes a semantic communication framework that prioritizes the transmission of task-relevant information, significantly reducing network load while maintaining DT accuracy.

System Framework

The proposed framework shifts the paradigm from transmitting raw data to transmitting "meaning." By extracting semantic features at the edge (on the UAV), the system filters out noise and focuses on the critical elements: the apples.

Overview of the importance-based semantic communication framework for apple orchard digital twins.

The pipeline consists of three main stages: enhanced object detection for semantic extraction, importance-based data prioritization, and generative AI-aided resource allocation to ensure reliable transmission over noisy wireless channels.

Methodology

YOLOv7-HS for Semantic Extraction

To efficiently identify apples in complex orchard environments characterized by high occlusion, the researchers developed YOLOv7-HS. This model improves upon the standard YOLOv7-X architecture through two primary innovations:

  1. ELAN-H (Efficient Layer Aggregation Network-HorNet): This module replaces standard layers to reduce computational complexity while maintaining high feature extraction capabilities.
  2. SimAM Attention Module: A parameter-free 3D attention module that enhances the model's ability to detect small, densely packed objects without increasing the model's weight.
The ELAN-H model structure and the internal components of the HorNet block used in YOLOv7-HS.

The UAV uses YOLOv7-HS to extract "semantic features," which include cropped images of detected apples, their spatial coordinates, and the detector's confidence scores. This process effectively compresses the scene by discarding irrelevant background data.

Diffusion-Aided Resource Allocation

A standout contribution of this work is the use of a Diffusion Model for intelligent resource allocation. Unlike traditional Reinforcement Learning, the Diffusion Model is used to "generate" an optimal power distribution scheme. It treats the wireless channel state as input and allocates more transmission power to high-importance semantic data (determined by confidence scores). This ensures that the most critical information for the Digital Twin—clear, high-confidence detections—is transmitted with the highest reliability, even as signal quality degrades over distance.

Key Results

The framework was evaluated using the MinneApple dataset, which presents challenging real-world conditions like heavy leaf occlusion.

  • Communication Efficiency: The semantic approach achieved a 91% reduction in data transmission, cutting the volume from 595.2 MB to just 55.4 MB.
  • Detection Performance: YOLOv7-HS achieved an [email protected] of 89.8%, a 2% improvement over the baseline YOLOv7-X. Crucially, the model is 24% lighter and operates at 34 FPS, making it suitable for real-time deployment on UAV hardware.
  • Transmission Robustness: The diffusion-based allocation scheme maintained superior performance (measured by the Metric for Image Semantic Transmission, or MIST) compared to standard allocation methods, particularly at distances up to 30 meters where signal attenuation is significant.

Conclusion

This research demonstrates that semantic communication, powered by task-specific object detection and generative AI for resource management, offers a viable path for scalable Digital Twin construction. By transmitting only the "essence" of the scene and intelligently protecting that data during transmission, the system overcomes the bandwidth bottlenecks of traditional 6G agricultural applications. Future work will focus on optimizing these models for even more constrained hardware and extending the framework to multi-UAV collaborative environments.

Summary updated

BibTeX
@article{du2024yolo,
  author = {Baoxia Du and Hongyang Du and Haifeng Liu and Dusit Niyato and Peng Xin and Jun Yu and Mingyang Qi and You Tang},
  title = {{YOLO-based Semantic Communication with Generative AI-aided Resource Allocation for Digital Twins Construction}},
  journal = {IEEE Internet of Things Journal},
  year = {2024},
  doi = {10.1109/jiot.2023.3317629},
}
Figure