Abstract
Creating immersive and playable 3D worlds from texts or images remains a fundamental challenge in computer vision and graphics. Existing world generation approaches typically fall into two categories: video-based methods that offer rich diversity but lack 3D consistency and rendering efficiency, and 3D-based methods that provide geometric consistency but struggle with limited training data and memory-inefficient representations. To address these limitations, we present HunyuanWorld 1.0, a novel framework that combines the best of both worlds for generating immersive, explorable, and interactive 3D scenes from text and image conditions. Our approach features three key advantages: 1) 360° immersive experiences via panoramic world proxies; 2) mesh export capabilities for seamless compatibility with existing computer graphics pipelines; 3) disentangled object representations for augmented interactivity. The core of our framework is a semantically layered 3D mesh representation that leverages panoramic images as 360° world proxies for semantic-aware world decomposition and reconstruction, enabling the generation of diverse 3D worlds. Extensive experiments demonstrate that our method achieves state-of-the-art performance in generating coherent, explorable, and interactive 3D worlds while enabling versatile applications in virtual reality, physical simulation, game development, and interactive content creation.
Introduction
Generating a cohesive digital world is significantly more complex than generating isolated 3D objects. Traditional methods often struggle with a trade-off between visual diversity and structural integrity. Video-based generative models produce high-fidelity sequences but suffer from "drift" and lack true 3D consistency, making them unsuitable for real-time interaction. Conversely, 3D-native methods often produce monolithic, static blocks that are difficult to edit or interact with. HunyuanWorld 1.0 bridges this gap by utilizing 360° panoramic images as a structural proxy, enabling the creation of high-quality, interactive 3D environments that are fully exportable as standard meshes.
Methodology
The HunyuanWorld 1.0 pipeline follows a staged approach to transform 2D pixels into interactive 3D environments:
1. Panorama-DiT
The process begins with Panorama-DiT, a Diffusion Transformer designed to generate 360° equirectangular panoramas. To eliminate the visible seams typically found in wrapped images, the system employs circular denoising and elevation-aware augmentation. This ensures a seamless, immersive view from the starting viewpoint.
2. Agentic World Layering
Rather than treating the panorama as a single flat image, HunyuanWorld uses a Vision-Language Model (VLM) as an intelligent agent. This agent identifies and segments the scene into distinct layers: foreground objects (e.g., furniture, trees), the background terrain, and the sky. The system then "inpaints" the occluded areas behind foreground objects, creating a layered representation that allows for independent manipulation of scene elements.
3. Layer-Wise Reconstruction and Optimization
Depth is estimated for each layer independently and then aligned to form a coherent 3D structure. These layers are converted into 3D meshes using a "sheet warping" technique. To ensure the resulting assets are practical for real-time applications, the system applies mesh decimation and Draco compression, resulting in lightweight models compatible with web and VR platforms.
4. Voyager: Long-Range Exploration
To support exploration beyond the initial 360° view, the Voyager module uses a world-caching mechanism. It leverages a video diffusion model to consistently "hallucinate" new terrain and objects as the user moves through the environment, maintaining visual continuity over long distances.
Key Insights: Semantic Decomposition
A primary innovation of HunyuanWorld 1.0 is its Semantic Decomposition. By separating the world into Sky, Background, and Foreground layers, the system enables true object interaction. Unlike "baked" 3D scenes, users can pick up, move, or rescale individual generated objects within the environment. This agentic reasoning allows the system to handle complex indoor and outdoor layouts with a level of intelligence that simple depth-map approaches cannot match.
Experimental Results and Applications
HunyuanWorld 1.0 consistently outperforms state-of-the-art baselines, such as Diffusion360 and MVDiffusion, across multiple benchmarks. It achieves superior scores in visual fidelity (BRISQUE/NIQE) and prompt alignment (CLIP-I/T).
The framework's versatility is demonstrated through several high-impact applications:
- Game Development: Rapidly generating explorable levels from text descriptions.
- Physical Simulation: Creating realistic environments where 3D assets can interact with generated terrain.
- Virtual Reality: Seamless deployment on hardware like Meta Quest and Apple Vision Pro.
Conclusion and Future Work
HunyuanWorld 1.0 represents a significant step toward automated, high-quality 3D world creation. While the system handles complex scenes effectively, future research will focus on refining inpainting for extreme occlusions and optimizing memory management for infinite, long-range exploration. By combining generative flexibility with structured 3D representations, HunyuanWorld 1.0 opens new possibilities for interactive content creation.