Toward Real-Time Edge AI: Model-Agnostic Task-Oriented Communication With Visual Feature Alignment

Songjie Xie, Hengtao He, Shenghui Song, Jun Zhang, Khaled B. Letaief

TL;DR

This paper introduces a framework for "cross-model" task-oriented communication in Edge AI, allowing edge devices and servers with different neural network architectures to collaborate. By using shared "anchor data" to align incoherent feature spaces—either through linear transformation estimation at the server or relative representations on the device—the authors enable seamless, real-time inference without the need for exhaustive retraining or raw data transmission.

What Problem Does This Paper Address?

In the world of Edge AI, we often split a Deep Neural Network (DNN) between a mobile device and a powerful edge server. Instead of sending heavy raw images, the device sends compressed "features" that are just enough for the server to complete a specific task (like object detection). This is called task-oriented communication.

The catch? It only works if the device and the server are running the exact same model. If your phone uses a VGG-based encoder but the nearest available edge server uses a ResNet-based decoder, the communication breaks. Even two identical architectures trained with different random seeds will produce "incoherent" feature spaces—essentially speaking different dialects of the same language. This lack of interoperability is a major hurdle for 6G and ubiquitous AI services where devices must roam between different service providers.

Methods

The authors propose a Model-Agnostic framework that relies on a small set of shared anchor data (a "knowledge base" known to both the device and server). They tackle the alignment problem from two angles:

  1. Server-Based Alignment: Based on the observation that visual features from different models are often related by a simple linear transformation, the server estimates a transformation matrix ($M$) to map the incoming features into its own space. They propose three estimators:

  2. OS-MLP: A single-layer neural network.

  3. OS-LS: A classic Least Squares approach.
  4. OS-MMSE: A Minimum Mean-Square Error estimator that accounts for channel noise.
  5. On-Device Alignment (Relative Representations): This approach exploits the "angle-preserving" nature of neural networks. Instead of sending the raw feature vector $z$, the device calculates the cosine similarity between $z$ and the features of the anchor data. This "relative representation" is naturally more consistent across different models, allowing the server to recognize the input based on its relationship to the anchors rather than its absolute position in a feature space.

Datasets

The researchers evaluated their framework using two standard computer vision benchmarks:

  • SVHN (Street View House Numbers): Over 600,000 real-world images of house numbers.
  • CIFAR-10: 60,000 tiny color images across 10 classes.

They tested these across three "backbone" configurations: VGG-based, ResNet-based, and a Mixed architecture to simulate heterogeneous edge environments.

Key Insights and Ideas

  • Linear Invariance is Key: The paper reinforces the idea that while different models look different on the surface, their high-level semantic features are often just rotated or scaled versions of one another.
  • The Power of Anchors: By agreeing on a tiny set of "reference points" (anchors), two completely independent systems can calibrate their communication on the fly.
  • Zero-Latency Inference: The on-device alignment method is particularly clever because it integrates the alignment into the encoding process itself. Once the model is trained with this "relative" mindset, there is zero additional computational overhead during the actual inference phase.

Key Results

  • Accuracy Recovery: Without alignment, cross-model accuracy was abysmal (often <20%). The proposed alignment methods restored performance to near-native levels (e.g., achieving ~94% accuracy on SVHN, comparable to the original single-model performance).
  • Speed: The server-based LS and MMSE estimators are incredibly fast, requiring less than 0.6 ms to compute the alignment.
  • Efficiency: Compared to standard "fine-tuning" (a common baseline), the proposed methods reduced computational overhead (FLOPs) by orders of magnitude while providing more stable performance across different signal-to-noise ratios (SNR).
  • Communication Overhead: The authors found that using just 32 to 100 anchor data points was sufficient for high-quality alignment, keeping the initial "setup" phase of the communication very lean.

Limitations and Future Work

  • Channel Mismatch: While the methods handle noise well, a significant mismatch between the noise levels during training and testing can still cause a performance gap. The authors suggest exploring Attention modules or Hypernetworks to bridge this.
  • Task Complexity: The current study focuses on image classification. Extending this to more complex, multi-task scenarios (like simultaneous segmentation and tracking) remains an open challenge.
  • Privacy: While sending features is generally more private than sending raw images, the use of shared anchor data might introduce new privacy considerations that need robust safeguards.
  • Scaling Anchors: As the number of anchor data points increases, so does the transmission latency for the "relative representation" method. Finding the optimal balance between the number of anchors and accuracy is a critical next step.

Summary updated

Abstract

Task-oriented communication presents a promising approach to improve the communication efficiency of edge inference systems by optimizing learning-based modules to extract and transmit relevant task information. However, real-time applications face practical challenges, such as incomplete coverage and potential malfunctions of edge servers. This situation necessitates cross-model communication between different inference systems, enabling edge devices from one service provider to collaborate effectively with edge servers from another. Independent optimization of diverse edge systems often leads to incoherent feature spaces, which hinders the cross-model inference for existing task-oriented communication. To facilitate and achieve effective cross-model task-oriented communication, this study introduces a novel framework that utilizes shared anchor data across diverse systems. This approach addresses the challenge of feature alignment in both server-based and on-device scenarios. In particular, by leveraging the linear invariance of visual features, we propose efficient server-based feature alignment techniques to estimate linear transformations using encoded anchor data features. For on-device alignment, we exploit the angle-preserving nature of visual features and propose to encode relative representations with anchor data to streamline cross-model communication without additional alignment procedures during the inference. The experimental results on computer vision benchmarks demonstrate the superior performance of the proposed feature alignment approaches in cross-model task-oriented communications. The runtime and computation overhead analysis further confirm the effectiveness of the proposed feature alignment approaches in real-time applications.
BibTeX
@article{xie2025toward,
  author = {Songjie Xie and Hengtao He and Shenghui Song and Jun Zhang and Khaled B. Letaief},
  title = {{Toward Real-Time Edge AI: Model-Agnostic Task-Oriented Communication With Visual Feature Alignment}},
  journal = {IEEE Journal on Selected Areas in Communications},
  year = {2025},
  doi = {10.1109/jsac.2025.3623175},
}
Figure