Distributed Task-Oriented Communication Networks with Multimodal Semantic Relay and Edge Intelligence

Jie Guo, Hao Chen, Bin Song, Yuhao Chi, Chau Yuen, Fei Richard Yu, Geoffrey Ye Li, Dusit Niyato

Abstract

In this article, we present a novel framework, named distributed task-oriented communication networks (DTCN), based on recent advances in multimodal semantic transmission and edge intelligence. In DTCN, the multimodal knowledge of semantic relays and the adaptive adjustment capability of edge intelligence can be integrated to improve task performance. Specifically, we propose the key techniques in the framework, such as semantic alignment and complement, a semantic relay scheme for deep joint source-channel relay coding, and collaborative device-server optimization and inference. Furthermore, a multimodal classification task is used as an example to demonstrate the benefits of the proposed DTCN over existing methods. Numerical results validate that DTCN can significantly improve the accuracy of classification tasks, even in harsh communication scenarios (e.g., low signal-to-noise regime), thanks to multimodal semantic relay and edge intelligence.

Introduction

Traditional wireless communication systems are primarily "data-oriented," focusing on the perfect recovery of every transmitted bit. However, modern AI-driven applications like autonomous driving and remote image recognition often do not require full data reconstruction. Instead, they rely on the underlying meaning or semantics relevant to a specific task. While "task-oriented" communication has emerged to address this by transmitting only relevant features, it faces significant challenges. Semantic features can be easily corrupted by "semantic noise" in harsh environments with low signal-to-noise ratios (SNR). Furthermore, most current systems treat different data modalities (e.g., images and text) in isolation, failing to exploit cross-modal correlations that could compensate for missing information.

The Distributed Task-Oriented Communication Network (DTCN) framework addresses these gaps by reimagining edge servers as active "semantic relays" that fuse multimodal knowledge to repair and strengthen information during transmission.

The DTCN Framework

The DTCN architecture is built upon three core technological pillars designed to enhance semantic reliability and system efficiency.

The DTCN framework architecture illustrating hierarchical multimodal knowledge and Deep-JSCRC with relay channels.

Hierarchical Multimodal Knowledge

To process complex data, DTCN utilizes Vision Transformers (ViT) for image feature extraction and BERT for text. A Heterogeneous Multimodal Graph, powered by a Multi-Graph Attention Network (Multi-GAT), aligns these disparate data types. This alignment enables the system to establish correlations between visual features and textual descriptions, providing a robust semantic foundation.

Deep Joint Source-Channel Relay Coding (Deep-JSCRC)

A pivotal innovation in DTCN is the semantic relay. Unlike traditional relays that merely amplify signals, the DTCN relay decodes incoming features and fuses them with local "background knowledge." It then re-encodes a "boosted" semantic representation for the receiver, effectively acting as a knowledge-based error correction mechanism.

Edge Intelligence and Federated Learning

To ensure privacy and efficiency, DTCN employs Federated Learning (FL) to train models across distributed edge servers without sharing raw data. Additionally, an LSTM-based workload predictor dynamically manages computational resources, shifting tasks from overloaded devices to idle ones to maintain low latency.

Experimental Evaluation

The framework was evaluated using the UPMC Food-101 dataset, which contains over 90,000 samples of food images paired with text recipes across 101 categories. This dataset is particularly challenging because visual similarities between dishes often require textual "semantic anchors" for accurate classification.

Classification accuracy of DTCN versus baseline methods across varying SNR levels.

Key Results

Numerical results demonstrate the superiority of DTCN over traditional Joint Source-Channel Coding (JSCC) and Information Bottleneck (DIB-VFE) methods:

  • Robustness in High Noise: At an SNR of -10 dB, DTCN achieved a classification accuracy of 62.86%, significantly outperforming image-only (13.27%) and text-only (22.7%) baselines.
  • Resilience to Data Loss: DTCN maintained high performance even when 50% of the transmitted image was masked, leveraging correlated text features from the relay to fill the information gap.
  • Distributed Efficiency: The FL-based approach achieved performance within 2% of centralized training, validating the feasibility of privacy-preserving distributed deployment.

Conclusion and Future Work

DTCN demonstrates that treating relays as active "knowledge boosters" can dramatically improve the reliability of task-oriented communications. By transmitting abstract semantic features rather than raw data, the framework also provides inherent privacy benefits.

Future research will focus on reducing the computational overhead of Transformers and Multi-GAT at the edge, extending the framework to generative tasks like video streaming, and developing methods for relays to autonomously update their background knowledge bases in real-time.

Summary updated

BibTeX
@article{guo2024distributed,
  author = {Jie Guo and Hao Chen and Bin Song and Yuhao Chi and Chau Yuen and Fei Richard Yu and Geoffrey Ye Li and Dusit Niyato},
  title = {{Distributed Task-Oriented Communication Networks with Multimodal Semantic Relay and Edge Intelligence}},
  journal = {IEEE Communications Magazine},
  year = {2024},
  doi = {10.1109/mcom.001.2300155},
}
Figure