How to Cache Important Contents for Multi-Modal Service in Dynamic Networks: A DRL-Based Caching Scheme

Zhe Zhang, Marc St-Hilaire, Xin Wei, Haiwei Dong, Abdulmotaleb El-Saddik

TL;DR

This paper introduces a Deep Reinforcement Learning (DRL) based caching scheme designed specifically for multi-modal services (video, audio, and haptics) in dynamic edge networks. By decoupling content importance evaluation from the caching decision logic using a Dueling Double Deep Q Network (D3QN), the system achieves significantly higher hit ratios and lower latency compared to traditional popularity-based methods.

What Problem Does This Paper Address?

As we move toward the "Tactile Internet" and the Metaverse, multimedia is no longer just about sight and sound; it includes haptic (touch) feedback. Haptic data is uniquely demanding: it requires ultra-low latency (often <1ms) and extremely high reliability.

Traditional edge caching—storing popular data closer to the user to save bandwidth—usually relies on simple metrics like "popularity" or "recency." However, these fail in multi-modal environments because:

  1. Diverse Requirements: A small haptic file might be more "important" for user experience (QoE) than a large, popular video file, yet traditional systems might evict the haptic data to make room for the video.
  2. Dynamic Networks: Network conditions (bandwidth, user request patterns) change constantly. Static caching weights can't keep up.
  3. The "New Content" Problem: Most DRL-based caching models struggle when new, unseen content is introduced because their "action space" is tied to specific content IDs.

Methods

The authors propose a two-part system: a Content Importance Evaluation Model and a Caching Model.

  • D3QN for Importance: They leverage a Dueling Double Deep Q Network (D3QN). The "Double" part prevents the overestimation of values common in standard Q-learning, while the "Dueling" architecture separates the estimation of the "state value" from the "advantage" of specific actions.
  • Decoupled Logic: Instead of the DRL agent deciding exactly which packet to cache (which is slow), the agent evaluates a content's Importance Value based on four factors: available bandwidth, local popularity, content modality (video vs. haptic), and content size.
  • The Mapping Function: To handle newly released content, they remove the Content ID from the DRL state space. Instead, they feed the model the features of the content. A mapping function then links the resulting importance score back to the specific ID.
  • Caching & Replacement Policy: Once importance is calculated, a lightweight algorithm handles the actual storage. If the cache is full, it simply sorts by the importance value and evicts the least important items.

Datasets

The study uses a simulated environment modeled after real-world network topologies.

  • Content Library: 500 unique multi-modal contents.
  • Request Data: A dataset of 24,412 user requests generated to mimic dynamic "peak" and "idle" periods over a 70-minute simulation.
  • Modality Specs: Content is categorized into 11 types (various video resolutions, audio qualities, and haptic fidelities) with specific bandwidth and latency requirements based on IEEE standards.

Key Insights and Ideas

  • Modality is a Priority: The "aha moment" here is that size and popularity aren't everything. By including "modal type" in the state space, the agent learns that haptic data—despite being tiny—is vital to cache at the edge to meet its strict 1ms latency requirements.
  • Efficiency through Separation: By using DRL only for evaluation and a simple algorithm for execution, the system avoids the high computational latency that usually plagues AI-based networking tools.
  • Generalization: By training on content features rather than IDs, the model stays relevant even as the content library changes, making it viable for real-world dynamic "live" environments.

Key Results

The proposed scheme was compared against several baselines, including Leave Copy Everywhere (LCE) and other DRL variants (CP-DQN).

  • Hit Ratio: Achieved at least a 15% higher hit ratio than baseline schemes.
  • Latency (Hops): Reduced the average number of network hops by up to 27%, directly translating to faster response times.
  • Network Load: Reduced overall network traffic load by up to 22%.
  • QoE (Unsatisfied Requests): Most importantly, it reduced the ratio of "unsatisfied requests" (where QoS targets weren't met) by more than 47% compared to standard DRL approaches.

Limitations and Future Work

The authors acknowledge that while they treat video, audio, and haptics as separate streams, these modalities are often correlated (e.g., a user requesting a VR video will almost certainly need the corresponding haptic stream).

Future Work: The authors suggest exploring these correlations. If an AI can predict that a request for "Modal A" implies a future request for "Modal B," the caching scheme could become even more proactive. Additionally, exploring how generative AI might create one modality from another (e.g., generating haptic effects from video) could further change how we value content for caching.

Summary updated

Abstract

With the continuous evolution of networking technologies, multi-modal services that involve video, audio, and haptic contents are expected to become the dominant multimedia service in the near future. Edge caching is a key technology that can significantly reduce network load and content transmission latency, which is critical for the delivery of multi-modal contents. However, existing caching approaches only rely on a limited number of factors, e.g., popularity, to evaluate their importance for caching, which is inefficient for caching multi-modal contents, especially in dynamic network environments. To overcome this issue, we propose a content importance-based caching scheme which consists of a content importance evaluation model and a caching model. By leveraging dueling double deep Q networks (D3QN) model, the content importance evaluation model can adaptively evaluate contents' importance in dynamic networks. Based on the evaluated contents' importance, the caching model can easily cache and evict proper contents to improve caching efficiency. The simulation results show that the proposed content importance-based caching scheme outperforms existing caching schemes in terms of caching hit ratio (at least 15% higher), reduced network load (up to 22% reduction), average number of hops (up to 27% lower), and unsatisfied requests ratio (more than 47% reduction).
BibTeX
@article{zhang2024how,
  author = {Zhe Zhang and Marc St-Hilaire and Xin Wei and Haiwei Dong and Abdulmotaleb El-Saddik},
  title = {{How to Cache Important Contents for Multi-Modal Service in Dynamic Networks: A DRL-Based Caching Scheme}},
  journal = {IEEE Trans. Multimedia},
  year = {2024},
  url = {https://openreview.net/forum?id=ZMbwfL3C6m},
}

Loading related content...

Figure