CALTeC: Content-Adaptive Linear Tensor Completion for Collaborative Intelligence

Ashiv Dhondea, Robert A. Cohen, Ivan V. Bajic

Abstract

Collaborative Intelligence (CI) distributes deep neural network (DNN) inference between edge devices and cloud servers. This paradigm is vulnerable to packet loss during the transmission of intermediate feature tensors, which significantly degrades model accuracy. We present CALTeC (Content-Adaptive Linear Tensor Completion), a non-iterative, training-free method for real-time feature tensor recovery. By exploiting inter-channel redundancy through local affine transformations, CALTeC reconstructs missing data packets with high fidelity and minimal latency. Experimental results on ResNet-18 using ImageNet data demonstrate that CALTeC maintains classification accuracy under bursty packet loss conditions, outperforming traditional iterative methods in speed-constrained scenarios.

Introduction

In Collaborative Intelligence (CI), a DNN is partitioned into a front-end on the edge and a back-end in the cloud. The edge device transmits intermediate feature tensors across unreliable networks. Packet loss results in "holes" within these tensors, leading to severe performance drops in the cloud-based back-end. As shown in Figure 1, these tensors are structured as multi-channel data blocks where each packet typically corresponds to a set of rows.

Structure of a feature tensor partitioned into packets for transmission.

Existing recovery methods face a trade-off: iterative mathematical approaches like SiLRTC are accurate but computationally prohibitive for real-time applications, while pre-trained methods like ALTeC are fast but require extensive, layer-specific training.

Methodology

CALTeC leverages the observation that feature channels within the same layer often exhibit high spatial correlation. When a packet is lost in a "damaged" channel, CALTeC performs a three-step recovery process:

  1. Donor Selection: The algorithm identifies the most correlated "donor" channel by calculating the Pearson correlation coefficient between the available spatial neighbors of the missing packet and the corresponding regions in other channels.
  2. Affine Relationship Estimation: It assumes a local affine relationship ($y = ax + b$) between the donor and damaged channels. The optimal parameters $a$ and $b$ are computed via fast least-squares minimization using neighboring data.
  3. Packet Reconstruction: The missing packet is filled by applying the estimated affine transformation to the co-located packet in the donor channel.
The CALTeC recovery process: identifying a candidate channel, estimating an affine transform from neighbors, and filling the damaged packet.

This closed-form solution enables near-instantaneous recovery without prior training, making it highly adaptable to different models and layers.

Experimental Evaluation

The performance of CALTeC was evaluated using a ResNet-18 model on a subset of the ImageNet dataset. Realistic bursty packet loss was simulated using the Gilbert-Elliott model.

Results

  • Efficiency: CALTeC achieved a recovery time of approximately 77.5 ms per image for the add_1 layer of ResNet-18. While slightly slower than the pre-trained ALTeC (30.5 ms), it eliminates the need for hours of offline training.
  • Accuracy: In time-constrained tests, CALTeC significantly outperformed iterative methods. While traditional solvers failed to converge to useful results within the inference window, CALTeC maintained high classification accuracy.
  • Visual Fidelity: Qualitative analysis demonstrates that CALTeC successfully restores structural details, such as object outlines and textures, which are lost in standard zero-filling or slow-converging iterative methods.

Conclusion

CALTeC provides a robust, "plug-and-play" solution for data recovery in CI systems. It bridges the gap between slow, general-purpose mathematical methods and fast, specialized pre-trained models. Future work includes addressing edge cases like total channel loss and exploring non-linear transformation models for more complex feature maps. Integration into emerging standards like Video Coding for Machines (VCM) is also a promising direction.

Summary updated

Abstract

In collaborative intelligence, an artificial intelligence (AI) model is typically split between an edge device and the cloud. Feature tensors produced by the edge sub-model are sent to the cloud via an imperfect communication channel. At the cloud side, parts of the feature tensor may be missing due to packet loss. In this paper we propose a method called Content-Adaptive Linear Tensor Completion (CALTeC) to recover the missing feature data. The proposed method is fast, data-adaptive, does not require pre-training, and produces better results than existing methods for tensor data recovery in collaborative intelligence.
BibTeX
@inproceedings{dhondea2021caltec,
  author = {Ashiv Dhondea and Robert A. Cohen and Ivan V. Bajic},
  title = {{CALTeC: Content-Adaptive Linear Tensor Completion for Collaborative Intelligence}},
  booktitle = {ICIP},
  year = {2021},
  doi = {10.1109/icip42928.2021.9506372},
}
Figure