Abstract
3D Snapshot introduces a novel framework for embedding complex 3D neural representations, such as Neural Radiance Fields (NeRFs), into a single, seemingly ordinary 2D "host" image. By utilizing invertible neural networks (INNs), the method hides 3D scene data as imperceptible high-frequency patterns. This allows a standard photo to serve as a container for a full 3D scene, enabling high-quality arbitrary view synthesis and real-time decoding on mobile devices without the storage overhead of traditional 3D formats.
Introduction
Neural Radiance Fields (NeRFs) have transformed 3D scene reconstruction but often suffer from large storage requirements and specialized file formats. While video-based invertible methods exist for hiding visual data, they typically face a "capacity wall" where restoration quality degrades significantly as more views are added. 3D Snapshot addresses this challenge by embedding the continuous neural representation itself into a single 2D image, bypassing the limitations of discrete frame storage and allowing for arbitrary view synthesis from a single file.
Methodology
The core of 3D Snapshot is an Invertible Neural Network (INN) architecture comprising 16 invertible blocks. These blocks utilize coupling layers—a standard technique in flow-based generative models—to ensure the transformation between the 3D representation and the 2D host image is mathematically reversible.
Within each block, 5-layer densely connected convolutional networks provide the non-linearity required to handle complex visual data. The 3D information is encoded as high-frequency, imperceptible patterns within the host image. Optimization is performed using the Adam optimizer with a dynamic "target constant noise tensor" to stabilize the early stages of training.
Experiments and Results
The framework was evaluated on the NeRF-Synthetic and Tanks and Temples datasets. A key finding is the robustness of the method regarding the "host" image; the choice of the initial 2D view does not significantly impact the final 3D reconstruction quality. For instance, in the "Hotdog" scene, the Peak Signal-to-Noise Ratio (PSNR) remained remarkably stable (between 29.97 and 30.14) regardless of the host perspective.
As shown in the qualitative results, 3D Snapshot achieves rendering quality comparable to state-of-the-art models like TensoRF and Plenoxels while maintaining a significantly smaller and more portable model size. It outperforms previous video-based embedding methods, particularly in maintaining visual consistency when synthesizing views that deviate significantly from the original host perspective.
Discussion and Limitations
While 3D Snapshot enables mobile deployment—demonstrated via an app that decodes printed snapshots in real-time—computational constraints on mobile hardware currently necessitate reduced rendering resolutions (e.g., 380 x 380) to maintain fluency. Additionally, while the method handles complex geometry, extremely high-entropy or massive-scale environments may eventually push the limits of what can be hidden imperceptibly in a single 2D image.
Conclusion
3D Snapshot provides a bridge between physical media and immersive digital content. By turning 2D images into "portals" for 3D scenes, this technology offers a portable, efficient, and visually seamless way to share and store 3D neural representations, potentially serving as a foundational technology for future metaverse applications.
Summary updated