Abstract
Audiences can gain an immersive experience watching videos from multiple angles (a.k.a. viewpoints). Free Viewpoint Video (FVV) is developed to enable users to choose their preferred viewpoints during the play of a video. However, users may experience a delay if video frames of the chosen viewpoint cannot be timely loaded, or synthesized from multiple video streams of neighboring viewpoints. To address this problem, we present Edge-FVV, an edge-assisted FVV system that employs edge caches to reduce the delay in streaming the requested FVV from the server to client users. We first analyze the capacity and delay at edge caches when answering FVV requests. Next, we propose two types of machine learning algorithms that allocate the users’ requests to appropriate edge caches. Our evaluation shows that two types of proposed algorithms outperform benchmarks by 4.2-7.4% and 4.6-6.8%, respectively, in reducing the delay for FVV requests.
Introduction
Free Viewpoint Video (FVV) allows viewers to "fly" around a scene, choosing any angle they desire—an interactive version of the "bullet time" effect. However, delivering this experience presents significant networking challenges. Synthesis bottlenecks occur when a user selects an angle not captured by a physical camera, requiring the system to synthesize it from neighboring views. Performing this on a central server lacks scalability, while mobile devices lack the processing power. Furthermore, a "Cache Dilemma" exists: too many edge caches lead to low hit rates and high download delays, while too few caches result in overloaded nodes and high synthesis delays.
System Architecture and Methods
To address these hurdles, Edge-FVV utilizes a three-tier architecture consisting of a Main Server, Edge Caches, and Users.
The system employs several key technical components:
- VFI-based Synthesis: Instead of computationally expensive 3D reconstruction, the system uses Video Frame Interpolation (VFI). By treating viewpoint changes as motion, it synthesizes new views using only two adjacent reference streams.
- Mathematical Modeling: The authors derived a formal expectation of processing time ($E(T)$) that accounts for both cache hit probabilities and queuing delays for synthesis.
- Intelligent Allocation: Two approaches were developed to assign users to caches: a distributed approach using Multi-armed Bandits (e.g., $\epsilon$-greedy, UCB) and a centralized approach using a Deep Q-Network (DQN) with UserNet and VideoNet sub-networks to optimize long-term rewards.
Key Insights and Results
A critical finding is the "Local Minimum" of latency. Total delay is non-linear; there is an optimal number of edge caches where load balancing and data locality are perfectly balanced. Standard heuristics often fail because they prioritize either load or locality exclusively, whereas Edge-FVV's learning algorithms balance both.
The evaluation, using both simulated and real-world commercial data, yielded the following results:
- Delay Reduction: The proposed machine learning algorithms outperformed standard benchmarks by 4.2–7.4% in distributed settings and 4.6–6.8% in centralized settings.
- Model Accuracy: As shown in the figure above, mathematical predictions for processing time closely matched measured results, validating the theoretical framework.
- Strategy Performance: In multi-armed bandit tests, $\epsilon$-first and $\epsilon$-decay strategies performed best, suggesting users benefit from settling on a reliable cache rather than constant exploration.
Conclusion
Edge-FVV demonstrates that an edge-assisted system combined with reinforcement learning can significantly reduce the latency of FVV streaming. By optimizing the allocation of user requests to edge caches, the system overcomes traditional synthesis and caching bottlenecks, providing a more seamless immersive experience. Future work will focus on deriving closed-form optimizations for cache numbers and adapting to more dynamic network environments.
Summary updated