Knowledge Graph Driven UAV Cognitive Semantic Communication Systems for Efficient Object Detection

Xi Song, Lu Yuan, Zhibo Qu, Fuhui Zhou, Qihui Wu, Tony Q. S. Quek, Rose Qingyang Hu

TL;DR

This paper introduces a Knowledge Graph (KG)-driven semantic communication system designed to help Unmanned Aerial Vehicles (UAVs) perform object detection under tight resource constraints. By transmitting compressed "semantic features" instead of raw pixels and using a Knowledge Graph to provide contextual "sanity checks" at the receiver, the system achieves higher accuracy and better noise resistance than traditional methods while using significantly less bandwidth and power.

What Problem Does This Paper Address?

UAVs are essential for tasks like disaster monitoring and urban planning, but they face a "triple threat" of constraints: limited battery life, modest on-board computing power, and unreliable wireless links.

Current systems usually take one of two paths:

  1. On-board processing: The UAV does the heavy lifting, which drains the battery rapidly.
  2. Cloud/Edge offloading: The UAV streams raw video to a ground server. This requires massive bandwidth and is highly susceptible to "noise" (interference) in the wireless channel, which can garble the image and ruin detection accuracy.

The authors tackle this by moving away from traditional data transmission toward Semantic Communication (SemCom). Instead of trying to reconstruct every pixel perfectly, the goal is to transmit only the "meaning" required for the specific task of object detection.

Methods

The proposed framework, called MSED+KG (Multi-scale Semantic Encoder-Decoder + Knowledge Graph), consists of three main stages:

  1. Multi-Scale Semantic Extraction: Since aerial images contain objects of vastly different sizes (e.g., a tiny car vs. a large harbor), the system uses a Feature Pyramid Network (FPN) combined with a ResNet-34 backbone. This extracts features at five different scales simultaneously.
  2. Task-Oriented Compression: A parallel encoder-decoder architecture compresses these features based on the available bandwidth. Instead of standard JPEG compression, it uses a learned "semantic" compression that prioritizes information relevant to identifying objects.
  3. Knowledge Graph Enhancement: This is the core innovation. At the ground server, the system doesn't just look at the noisy visual features. It consults a Knowledge Graph (using ConceptNet and metapath2vec embeddings).
  4. If the visual data is blurry due to channel noise, the system uses a Relational Graph Attention Network (R-GAT) to weigh the visual evidence against logical relationships. For example, if the system is unsure if it sees a "ship" or a "baseball," but it definitely sees a "harbor," the KG helps it correctly identify the "ship" based on contextual probability.

Datasets

The authors evaluated their system using the DOTA (Dataset for Object deTection in Aerial images).

  • Scale: It contains large-scale images (approx. 4000 × 4000 pixels).
  • Diversity: It features 15 categories of objects (planes, ships, bridges, etc.) in various orientations and scales, making it a rigorous benchmark for UAV-based vision.

Key Insights and Ideas

  • Context as a Buffer for Noise: The most striking idea is using the Knowledge Graph as a "denoiser." In traditional systems, if the signal-to-noise ratio (SNR) drops, the image becomes "garbage" and the detector fails. Here, the KG provides a "global understanding" that allows the system to make accurate detections even when the visual signal is heavily distorted.
  • Semantic vs. Data Redundancy: Traditional compression (like JPEG) removes data redundancy. Semantic compression removes meaning redundancy. By only sending what matters for the "task," the system achieves a much better trade-off between accuracy and transmission volume.
  • Multi-Scale is Mandatory: For UAVs, a single-scale approach is insufficient. The parallel structure of the multi-scale encoder ensures that small objects aren't "lost in translation" during the compression process.

Key Results

The researchers compared their system against two baselines: A+F (an attention-based deep joint source-channel coding system) and JPEG+LDPC (a traditional digital communication pipeline).

  • Robustness to Noise: In a Rayleigh fading channel (a realistic model for wireless interference), the proposed system (MSED+KG) saw only a 0.6% drop in mean Average Precision (mAP), whereas the traditional JPEG+LDPC system plummeted by 7.4%.
  • Bandwidth Efficiency: The system maintained high accuracy even at extremely low bandwidth compression ratios (e.g., $R=1/12$), significantly outperforming the benchmarks.
  • Computational Savings: As shown in Table I, the MSED+KG model requires roughly 5x fewer parameters and 4x fewer additions/multiplications than the A+F baseline, making it much more suitable for deployment on resource-constrained UAV hardware.

Limitations and Future Work

While the paper demonstrates impressive results, a few areas remain for exploration:

  • KG Generalization: The system relies on a pre-defined Knowledge Graph. If the UAV encounters a completely novel environment or object not represented in the KG, the "contextual correction" might lead to false positives.
  • Real-time Latency: While the paper discusses computational complexity, it doesn't provide end-to-end latency measurements (in milliseconds) for the R-GAT processing on the server side.
  • Dynamic Environments: Future work could involve "online" knowledge graph updates, where the system learns new relationships on the fly as it explores new territories.

Summary updated

Abstract

Unmanned aerial vehicles (UAVs) are widely used for object detection. However, the existing UAV-based object detection systems are subject to the serious challenge, namely, the finite computation, energy and communication resources, which limits the achievable detection performance. In order to overcome this challenge, a UAV cognitive semantic communication system is proposed by exploiting knowledge graph. Moreover, a multi-scale compression network is designed for semantic compression to reduce data transmission volume while guaranteeing the detection performance. Furthermore, an object detection scheme is proposed by using the knowledge graph to overcome channel noise interference and compression distortion. Simulation results conducted on the practical aerial image dataset demonstrate that compared to the benchmark systems, our proposed system has superior detection accuracy, communication robustness and computation efficiency even under high compression rates and low signal-to-noise ratio (SNR) conditions.
BibTeX
@inproceedings{song2024knowledge,
  author = {Xi Song and Lu Yuan and Zhibo Qu and Fuhui Zhou and Qihui Wu and Tony Q. S. Quek and Rose Qingyang Hu},
  title = {{Knowledge Graph Driven UAV Cognitive Semantic Communication Systems for Efficient Object Detection}},
  booktitle = {ICC},
  year = {2024},
  doi = {10.1109/icc51166.2024.10622874},
}

Loading related content...

Figure