Artificial intelligence for the metaverse: A survey

Thien Huynh-The, Quoc-Viet Pham, Xuan-Qui Pham, Thanh Thi Nguyen, Zhu Han, Dong-Seong Kim

The term "Metaverse" often feels like a buzzword caught between science fiction and corporate rebranding. However, beneath the hype lies a complex convergence of technologies. In their survey paper, "Artificial Intelligence for the Metaverse: A Survey," Thien Huynh-The and colleagues argue that Artificial Intelligence (AI) isn't just an add-on—it is the foundational engine that will make a persistent, immersive, and scalable virtual world possible.

TL;DR

This paper provides a comprehensive roadmap of how AI serves as the backbone for the Metaverse, spanning six technical pillars: NLP, machine vision, blockchain, networking, digital twins, and neural interfaces. It moves beyond the "hype" to categorize specific machine learning architectures that enable everything from realistic avatars to ultra-low-latency 6G networking.

What Problem Does This Paper Address?

While virtual environments have existed for decades (think Second Life or World of Warcraft), they have largely been "incoherent" silos. The modern vision of the Metaverse requires a seamless, integrated platform that is persistent, scalable, and capable of supporting millions of users simultaneously.

The challenge is that the sheer volume of data and the complexity of real-time interaction in a 3D space exceed the capabilities of traditional, hard-coded software. The authors address the "how" of the Metaverse: How do we manage massive network traffic? How do we create "hyper-real" content at scale? How do we ensure security in a decentralized economy? Their answer is a deep integration of AI across every layer of the stack.

Methods

As a survey paper, the authors don't propose a single new algorithm. Instead, they provide a taxonomy of AI's role across a seven-layer Metaverse platform (Infrastructure, Human Interface, Decentralization, Spatial Computing, Creator Economy, Discovery, and Experience).

They specifically analyze the application of:

  • Deep Learning (DL): Using CNNs for machine vision and RNNs/LSTMs for natural language processing.
  • Reinforcement Learning (RL): Applied to optimize network resource allocation and "intelligent" NPC (non-player character) behavior.
  • Federated Learning (FL): Highlighted as a key method for training AI models on user data (like gestures or voice) without compromising privacy, which is critical for blockchain-based decentralization.
  • Generative Models: Such as GANs for procedural content generation, allowing the Metaverse to expand without manual 3D modeling for every object.

Datasets

As this is a survey, the authors do not introduce a new benchmark dataset. Instead, they reference a wide array of datasets used in the field, including:

  • NLP: Wikipedia corpus for language modeling.
  • Machine Vision: Standard image datasets for object detection and semantic segmentation.
  • Neural Interfaces: EEG signal datasets for training Brain-Computer Interfaces (BCIs).
  • Networking: Real-world cellular traffic data for forecasting and network slicing.

Key Insights and Ideas

  • The Six Pillars: The paper identifies six technical aspects where AI is non-negotiable:
  • NLP: For multilingual, real-time translation and intelligent virtual assistants.
  • Machine Vision: For "inside-out" tracking in VR headsets and realistic avatar expressions.
  • Blockchain: Using AI to detect fraudulent transactions and optimize smart contracts.
  • Networking: AI-driven "network slicing" in 5G/6G to ensure the Metaverse doesn't lag.
  • Digital Twins: Creating real-time, AI-synced digital replicas of physical objects or cities.
  • Neural Interfaces: Moving beyond controllers to direct brain-to-VR interaction.
  • AI as a Content Creator: One of the most important "aha" moments is the role of AI in the Creator Economy. Manual 3D asset creation is too slow; AI-aided procedural generation is the only way to build a "universe-sized" world.
  • The Privacy Paradox: The Metaverse requires intimate user data (biometrics, eye-tracking) to function. The authors emphasize that Federated Learning is the primary solution to enable AI personalization while keeping data on the user's device.

Key Results

The paper synthesizes quantitative trends from industry and academia:

  • Market Growth: Citing Bloomberg Intelligence, the authors note the global Metaverse revenue opportunity is expected to grow from $500 billion in 2020 to $800 billion by 2024.
  • Technical Performance: The survey highlights that DL-based methods (like NVIDIA’s DLSS) are already providing massive speedups in rendering high-resolution 3D environments, making "hyper-realism" computationally affordable.
  • Network Efficiency: AI-based traffic forecasting has shown significantly higher accuracy than traditional models, which is vital for maintaining the <10ms latency required for a comfortable VR experience.

Limitations and Future Work

The authors acknowledge several hurdles that AI has yet to clear:

  • Explainable AI (XAI): Many Metaverse AI models are "black boxes." If an AI agent makes a decision in a virtual healthcare setting or a DeFi transaction, we need to know why.
  • Ethical Concerns: The potential for "deepfakes" and AI-driven social manipulation in an immersive 3D space is much higher than on current 2D social media.
  • Computational Cost: Training the massive models required for the Metaverse has a significant carbon footprint and requires specialized hardware (GPUs/TPUs) that isn't yet universal.

In conclusion, the paper suggests that the "Metaverse" isn't a single product we will buy, but a complex ecosystem that will only succeed if AI can successfully manage the chaos of a billion-user virtual world.

Summary updated

Abstract

Along with the massive growth of the Internet from the 1990s until now, various innovative technologies have been created to bring users breathtaking experiences with more virtual interactions in cyberspace. Many virtual environments with thousands of services and applications, from social networks to virtual gaming worlds, have been developed with immersive experience and digital transformation, but most are incoherent instead of being integrated into a platform. In this context, metaverse, a term formed by combining meta and universe, has been introduced as a shared virtual world that is fueled by many emerging technologies, such as fifth-generation networks and beyond, virtual reality, and artificial intelligence (AI). Among such technologies, AI has shown the great importance of processing big data to enhance immersive experience and enable human-like intelligence of virtual agents. In this survey, we make a beneficial effort to explore the role of AI in the foundation and development of the metaverse. We first deliver a preliminary of AI, including machine learning algorithms and deep learning architectures, and its role in the metaverse. We then convey a comprehensive investigation of AI-based methods concerning six technical aspects that have potentials for the metaverse: natural language processing, machine vision, blockchain, networking, digital twin, and neural interface, and being potential for the metaverse. Subsequently, several AI-aided applications, such as healthcare, manufacturing, smart cities, and gaming, are studied to be deployed in the virtual worlds. Finally, we conclude the key contribution of this survey and open some future research directions in AI for the metaverse.
BibTeX
@article{huynhthe2023artificial,
  author = {Thien Huynh-The and Quoc-Viet Pham and Xuan-Qui Pham and Thanh Thi Nguyen and Zhu Han and Dong-Seong Kim},
  title = {{Artificial intelligence for the metaverse: A survey}},
  journal = {Engineering Applications of AI},
  year = {2023},
  doi = {10.1016/j.engappai.2022.105581},
}

Loading related content...

Figure