How to Cache Important Contents for Multi-modal Service in Dynamic Networks: A DRL-based Caching Scheme

Zhe Zhang, Marc St-Hilaire, Xin Wei, Haiwei Dong, Abdulmotaleb El-Saddik

Abstract

With the continuous evolution of networking technologies, multi-modal services involving video, audio, and haptic content are expected to become dominant multimedia services. Edge caching is a key technology to reduce network load and transmission latency, which is critical for multi-modal delivery. However, existing caching approaches rely on limited factors like popularity, which is inefficient for multi-modal content in dynamic environments. We propose a content importance-based caching scheme consisting of a content importance evaluation model and a caching model. Leveraging a Dueling Double Deep Q Network (D3QN), the evaluation model adaptively assesses content importance. Simulation results show the proposed scheme outperforms existing methods, achieving at least a 15% higher cache hit ratio and a 47% reduction in unsatisfied requests.

Introduction: The Multi-modal Challenge

As we transition toward the "Tactile Internet" and the Metaverse, multimedia services are evolving beyond sight and sound to include haptic (touch) feedback. Haptic data introduces unique challenges: it requires ultra-low latency and extremely high reliability compared to traditional video or audio streams.

Traditional edge caching—storing data closer to users—usually relies on simple metrics like popularity or recency. These metrics fail in multi-modal environments for several reasons:

  1. Diverse Requirements: A small haptic file may be more critical for Quality of Experience (QoE) than a large, popular video file, yet traditional systems might evict the haptic data to make room for the video.
  2. Dynamic Networks: Network conditions like bandwidth and request patterns change constantly, making static caching weights obsolete.
  3. The "New Content" Problem: Many Deep Reinforcement Learning (DRL) models struggle with new content because their action space is tied to specific content IDs.
Table 1: QoS requirements for different content modalities.

As shown in the table above, haptic content has significantly stricter latency and data loss requirements than video or audio, necessitating a more nuanced approach to caching priority.

Proposed DRL-based Caching Scheme

To address these challenges, we propose a two-part system that decouples content importance evaluation from the caching decision logic.

Figure 1: Architecture of the proposed DRL-based caching scheme.

Content Importance Evaluation Model

We utilize a Dueling Double Deep Q Network (D3QN) to evaluate content importance. The "Double" component prevents the overestimation of values common in standard Q-learning, while the "Dueling" architecture separates the estimation of the state value from the advantage of specific actions.

Instead of the DRL agent deciding exactly which packet to cache (which can be computationally expensive), the agent evaluates a content's Importance Value based on four dynamic factors: available bandwidth, local popularity, content modality (e.g., video vs. haptic), and content size. To handle newly released content, we remove the Content ID from the DRL state space and instead feed the model content features.

Caching and Replacement Policy

Once the importance value is calculated, a lightweight algorithm handles the execution. If the cache is full, the system sorts items by their importance value and evicts the least important content. This separation of evaluation and execution avoids the high computational latency typically associated with AI-based networking tools.

Key Results and Insights

The proposed scheme was evaluated in a simulated environment with 500 unique multi-modal contents and over 24,000 user requests mimicking dynamic peak and idle periods.

  • Hit Ratio: The scheme achieved at least a 15% higher hit ratio than baseline schemes like Leave Copy Everywhere (LCE).
  • Network Efficiency: Reduced the average number of network hops by up to 27% and overall network traffic load by up to 22%.
  • QoE Improvement: Most significantly, it reduced the ratio of "unsatisfied requests" (where QoS targets were not met) by more than 47% compared to standard DRL approaches.

By including "modal type" in the state space, the agent learns that haptic data—despite its small size—is vital to cache at the edge to meet strict latency requirements.

Conclusion and Future Work

This research demonstrates that decoupling importance evaluation from caching execution allows for a more flexible and efficient multi-modal network. Future work will explore the correlations between modalities (e.g., a VR video request implying a future haptic request) to enable proactive caching. Additionally, we aim to investigate how generative AI might be used to synthesize one modality from another, further evolving how we value content for edge storage.

Summary updated

Abstract

With the continuous evolution of networking technologies, multi-modal services that involve video, audio, and haptic contents are expected to become the dominant multimedia service in the near future. Edge caching is a key technology that can significantly reduce network load and content transmission latency, which is critical for the delivery of multi-modal contents. However, existing caching approaches only rely on a limited number of factors, e.g., popularity, to evaluate their importance for caching, which is inefficient for caching multi-modal contents, especially in dynamic network environments. To overcome this issue, we propose a content importance-based caching scheme which consists of a content importance evaluation model and a caching model. By leveraging dueling double deep Q networks (D3QN) model, the content importance evaluation model can adaptively evaluate contents' importance in dynamic networks. Based on the evaluated contents' importance, the caching model can easily cache and evict proper contents to improve caching efficiency. The simulation results show that the proposed content importance-based caching scheme outperforms existing caching schemes in terms of caching hit ratio (at least 15% higher), reduced network load (up to 22% reduction), average number of hops (up to 27% lower), and unsatisfied requests ratio (more than 47% reduction).
BibTeX
@article{zhang2024how,
  author = {Zhe Zhang and Marc St-Hilaire and Xin Wei and Haiwei Dong and Abdulmotaleb El-Saddik},
  title = {{How to Cache Important Contents for Multi-modal Service in Dynamic Networks: A DRL-based Caching Scheme}},
  journal = {IEEE Trans. Multimedia},
  year = {2024},
  url = {https://openreview.net/forum?id=ZMbwfL3C6m},
}
Figure