VVRec: Reconstruction Attacks on DL-based Volumetric Video Upstreaming via Latent Diffusion Model with Gamma Distribution
As 3D volumetric video streaming gains traction in the Metaverse, VR gaming, and telepresence, the industry is shifting toward deep learning (DL) for efficient data compression. In a typical "upstreaming" scenario, a user's device uses a DL encoder to compress high-fidelity 3D point clouds into low-dimensional "shape latents" for transmission to a cloud server.
The Privacy Threat in Volumetric Upstreaming
While network transmission is often encrypted, the underlying DL architecture introduces a significant privacy vulnerability: reconstruction attacks. If an adversary intercepts these shape latents, they can potentially reconstruct the original 3D video, exposing sensitive personal information such as facial features, gender, and private surroundings.
The primary hurdle for such an attacker is the distribution shift. The attacker typically lacks access to the cloud's private training data and proprietary decoder. Training a surrogate decoder on different data usually results in severe distortion. VVRec asks: Can an attacker overcome this distribution shift and accurately reconstruct high-fidelity 3D point clouds from intercepted DL latents?
The VVRec Attack Framework
To address these challenges, this paper introduces VVRec, a novel reconstruction attack specifically tailored for DL-based volumetric video pipelines. VVRec leverages Latent Diffusion Models (LDMs) to bridge the gap between intercepted latents and high-fidelity 3D reconstructions.
Methodology
The VVRec pipeline consists of three primary stages as shown in the architecture above:
- Latent Points Generation: Intercepted shape latents are processed through a Shape Latent Diffusion Model (SLDM) and a Latent Point Generator (LPG). This stage converts the abstract latent representation into a point cloud format (latent points).
- Point Cloud Generation: This stage features the Latent Point Gamma Diffusion Model (LPGDM). Unlike standard diffusion models that use Gaussian noise, VVRec employs a Gamma distribution. This choice is critical because Gaussian noise often fails to capture the complex geometric nuances of high-dimensional 3D point clouds (the "prior hole" problem). A Reconstruction Decoder (RD) then maps these outputs to spatial 3D coordinates.
- Point Cloud Refinement: A post-processing step converts the raw reconstructed points into a watertight mesh and resamples them to eliminate artifacts and noise generated during the diffusion process.
Key Insights and Results
The researchers evaluated VVRec against several benchmark datasets, including 8iv2, Owlii, and MVUB. The results demonstrate a significant security risk:
- Superior Performance: VVRec achieved a point-to-point PSNR of up to 64.70 dB, outperforming traditional supervised and GAN-based baselines by up to 37.20% and reducing overall distortion by 46.39%.
- Robustness to Distribution Shift: Even when trained on datasets different from the victim's (e.g., training on upper bodies to attack full-body models), VVRec maintained high reconstruction quality.
- Gamma Distribution Efficacy: Ablation studies confirmed that swapping the Gamma distribution for standard Gaussian noise resulted in a substantial 6.61 dB performance drop, highlighting the importance of the Gamma-based diffusion process for 3D data.
- Vulnerability of Color: Beyond geometry, VVRec can reconstruct color information (skin tone, clothing) if the victim's encoder preserves such features, significantly elevating the privacy risk.
Conclusion
VVRec exposes a critical privacy flaw in emerging 3D streaming architectures. By demonstrating that high-fidelity reconstruction is possible even without access to private cloud decoders or training data, this work underscores the need for new defensive strategies. Future research must focus on protecting latent spaces from diffusion-based attacks to ensure the privacy of users in the growing volumetric video ecosystem.
Summary updated