Abstract
Gaussian splatting (GS) struggles with degraded rendering quality on low-cost devices. To address this issue, we present edge collaborative GS (ECO-GS), where each user can switch between a local small GS model to guarantee timeliness and a remote large GS model to guarantee fidelity. However, deciding how to engage the large GS model is nontrivial, due to the interdependency between rendering requirements and resource conditions. To this end, we propose integrated rendering and communication (IRAC), which jointly optimizes collaboration status (i.e., deciding whether to engage large GS) and edge power allocation (i.e., enabling remote rendering) under communication constraints across different users by minimizing a newly-derived GS switching function. Despite the nonconvexity of the problem, we propose an efficient penalty majorization minimization (PMM) algorithm to obtain the critical point solution. Furthermore, we develop an imitation learning optimization (ILO) algorithm, which reduces the computational time by over 100x compared to PMM. Experiments demonstrate the superiority of PMM and the real-time execution capability of ILO.
Introduction
3D Gaussian Splatting (GS) has revolutionized real-time 3D reconstruction, yet its high resource demands pose significant challenges for low-power mobile devices such as AR glasses and smartphones. To operate on these devices, GS models are often pruned or quantized, which unfortunately results in blurred or inaccurate visual outputs. While offloading rendering to a powerful edge server is a potential solution, it introduces complex trade-offs. Specifically, developers must determine when offloading is necessary to avoid wasting battery and bandwidth, and how to manage communication bottlenecks that can lead to high latency and stuttering.
Existing solutions typically treat rendering and communication as independent problems. This paper introduces Integrated Rendering and Communication (IRAC) to address these challenges jointly within a unified framework.
The ECO-GS Framework
The proposed Edge Collaborative Gaussian Splatting (ECO-GS) framework employs an intelligent switching mechanism to decide whether each frame should be rendered locally or offloaded to an edge server. This decision is governed by a novel GS Switching Function, which estimates the visual gain of using the edge server by comparing the discrepancy between a compressed local model and a high-fidelity remote model without requiring ground-truth images in real-time.
To solve the resulting non-convex optimization problem—which involves both binary offloading decisions and continuous power allocation variables—the authors developed the Penalty Majorization Minimization (PMM) algorithm. To meet strict real-time requirements, they further introduced Imitation Learning Optimization (ILO). ILO uses a Deep Neural Network (DNN) to mimic the PMM solver, enabling optimal decision-making in under 1 millisecond.
Experimental Evaluation
The system was evaluated using the standard Truck dataset. The local model was compressed to 9.3 MB, while the edge model remained at 509.8 MB. The simulation included up to 20 users in a Massive MIMO context.
Key results include:
- Quality Improvement: ECO-GS achieved a Peak Signal-to-Noise Ratio (PSNR) improvement of over 1.3 dB compared to local-only rendering.
- Efficiency: Rendering loss was reduced by at least 15% relative to local baselines.
- Computational Speed: The ILO algorithm demonstrated a 100x speedup over the PMM solver, maintaining execution times under 1ms even as the number of users increased.
- Latency Guarantees: ECO-GS successfully maintained a latency budget of under 60ms, avoiding the 80ms spikes typical of remote-only approaches and ensuring smooth user experiences.
Conclusion
ECO-GS demonstrates that joint optimization of rendering and communication can significantly enhance the performance of 3D Gaussian Splatting on resource-constrained devices. By identifying "pain points" where local models fail and leveraging imitation learning for rapid decision-making, the framework achieves high fidelity with minimal latency. Future work will explore model synchronization in dynamic environments and scalability in scenarios with fewer server antennas.
Summary updated