Privacy-preserving Pseudonym Schemes for Personalized 3D Avatars in Mobile Social Metaverses

Zhaohui Yang, Cheng Su, Xiaofeng Luo, Zhenmou Liu, Jiawen Kang, Min Hao, Zehui Xiong, Zhaohui Yang 0001, Chongwen HuangarXiv 2024 · DOI · arXiv

Abstract

As the mobile social metaverse expands, the demand for high-fidelity 3D avatars grows alongside significant privacy concerns regarding facial data. This paper proposes a novel framework that decouples avatar generation from real-world scans using a two-layer construction process involving Stable Diffusion and 3D Morphable Models (3DMM). To ensure anonymity, we introduce an attribute-based pseudonym framework and a new metric, Privacy of Personalized Avatars (PoPA). The distribution of these pseudonyms is optimized through a Stackelberg game and Deep Reinforcement Learning (DRL), specifically Proximal Policy Optimization (PPO). Results demonstrate that the proposed system maintains high immersion while significantly improving privacy and resource allocation efficiency compared to existing baselines.

Introduction

The mobile social metaverse represents the next frontier of digital interaction, where 3D avatars serve as the primary medium for user presence. However, the current reliance on high-fidelity facial scans—exemplified by systems like Apple’s Persona—introduces significant privacy risks. When a digital avatar is a precise 3D replica of a user's physical face, their real-world identity becomes inextricably linked to their virtual activities, facilitating persistent tracking and potential data breaches. The challenge lies in a classic trade-off: users desire high-quality, personalized immersion but require robust privacy protections. Furthermore, managing the temporary digital identities (pseudonyms) needed for these avatars creates network congestion and resource allocation issues for edge servers.

System Architecture

To address these challenges, we propose a multi-layered solution that balances immersion with anonymity. The architecture is divided into two primary frameworks: Personalized 3D Avatar Construction and an Attribute-based Pseudonym Framework.

System architecture overview showing the interaction between physical and virtual spaces, avatar construction, and pseudonym management.

Two-Layer Avatar Construction

Our approach decouples avatar generation from physical scans by utilizing AI-driven synthesis. The first layer uses Stable Diffusion to generate 2D profile images based on text or image prompts. This allows users to choose styles (e.g., "realistic" vs. "anime") that do not necessarily mirror their actual physical appearance. The second layer performs 3D reconstruction using a 3D Morphable Model (3DMM) and StyleGAN2 to transform the 2D image into a full 3D head with high-quality textures.

Attribute-based Pseudonym Framework

Instead of assigning random ID numbers, the system generates pseudonyms based on avatar attributes (such as lip color or position) combined with random numbers. This makes the digital identity harder to track across different virtual sessions. We quantify this protection using the Privacy of Personalized Avatars (PoPA) metric, which employs entropy-like calculations to measure how unique—and thus potentially identifiable—an avatar's attributes are.

Privacy and Resource Optimization

Managing the cost and distribution of pseudonyms is modeled as a Stackelberg game between the Local Authority (the leader/seller) and the users (the followers/buyers). Because the server lacks complete information regarding specific user demands, we utilize Proximal Policy Optimization (PPO), a Deep Reinforcement Learning algorithm, to find the optimal pricing and distribution strategy. This ensures that privacy protection is treated as a limited resource allocated efficiently to avoid network congestion at the edge.

Experimental Results

The framework was evaluated using state-of-the-art pre-trained models including Deep3D and HiFi3D++ for shape regression. Qualitative results show that the framework produces professional-grade 3D avatars for both realistic and stylized aesthetics.

Rendered 3D avatar head demonstrating high-fidelity texture and shape reconstruction.

Quantitatively, the proposed method achieved a higher PoPA score compared to the existing DreamFace baseline, indicating superior privacy-preserving performance. In economic simulations, the DRL-based pseudonym distribution scheme outperformed baseline random and greedy algorithms by 27.2% and 55.7% respectively in terms of utility for the network provider, successfully reaching a Nash equilibrium even under conditions of incomplete information.

Conclusion

This research provides a scalable solution for privacy-preserving identity management in the metaverse. By breaking the link between physical humans and digital entities at the source, users can enjoy personalized experiences without compromising their real-world anonymity. Future work will focus on incorporating social interaction attributes into satisfaction metrics and refining security metrics to quantify the long-term un-linkability of avatars.

Summary updated

BibTeX
@misc{yang2024privacy,
  author = {Zhaohui Yang and Cheng Su and Xiaofeng Luo and Zhenmou Liu and Jiawen Kang and Min Hao and Zehui Xiong and Zhaohui Yang and Chongwen Huang},
  title = {{Privacy-preserving Pseudonym Schemes for Personalized 3D Avatars in Mobile Social Metaverses}},
  year = {2024},
  eprint = {2406.11208},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.11208},
}
Figure