Author Correction: Towards real-time photorealistic 3D holography with deep neural networks

Liang Shi, Beichen Li, Chang-Il Kim, Petr Kellnhofer, Wojciech MatusikNature 2021 · DOI

Introduction

In 2021, the publication of "Towards real-time photorealistic 3D holography with deep neural networks" marked a significant milestone in display technology. The research introduced "Tensor Holography," a deep-learning-based approach designed to bypass the computational intensity of traditional computer-generated holography (CGH). By utilizing a convolutional neural network (CNN) to approximate complex physics simulations, the authors demonstrated a path toward high-quality, real-time 3D displays. This author correction provides critical refinements to the physical model and the computational requirements of the neural network, revealing that the system is even more efficient than initially reported.

Refinements to the Physical Model

The correction addresses two primary discrepancies regarding the physical hardware setup and its mathematical representation.

First, the authors clarify the orientation of the optical components. In the original Fig. 3a, a beamsplitter was illustrated at a 90° offset from its actual orientation on the optical bench. This correction ensures that the primary schematic is consistent with the Extended Data and the actual physical alignment used during the experiment, which is essential for reproducibility.

Second, a mathematical oversight regarding the variable $\Delta p$ has been corrected. While originally labeled as the "SLM pixel pitch," $\Delta p$ is actually the grating pitch, which is twice the size of the SLM pixel pitch. This distinction is fundamental to how the network models the diffraction of light across the spatial light modulator.

Neural Network Optimization and Efficiency

The most impactful insight arising from this correction concerns the architecture of the neural network. The "receptive field"—the area of the input image that the network considers at any given time—is mathematically tied to the physical dimensions of the hardware (the grating pitch).

Because the grating pitch is twice as large as previously calculated, the spatial extent the network needs to "see" to accurately model the physics of light diffraction is smaller. Consequently, the authors found that the "CNN small" architecture is more powerful than initially estimated. The correction reveals that the minimal number of convolution layers required to model the 3D scene effectively is 15, rather than the 30 layers originally reported.

Implications for Real-Time Applications

In the context of deep learning, halving the depth of a network significantly reduces the computational "cost." This reduction in complexity has several benefits for the future of holographic displays:

  • Reduced Memory and Processing: Fewer layers mean lower power consumption and faster processing times.
  • Edge Computing: The 15-layer requirement makes the "Tensor Holography" method even more suitable for mobile devices and consumer-grade hardware.
  • Real-Time Performance: These efficiency gains further solidify the feasibility of generating photorealistic 3D holograms at interactive frame rates on devices with limited computational resources, such as VR and AR headsets.

Conclusion

This correction clarifies the physical and mathematical framework of Tensor Holography, ensuring the theoretical model perfectly matches the experimental reality. By proving that a shallower 15-layer network is sufficient for high-quality 3D synthesis, the authors have demonstrated that their method is even more optimized for real-time, portable applications than previously thought. This update provides a more accurate roadmap for researchers and engineers working to bring holographic interfaces into the mainstream.

Summary updated

BibTeX
@inproceedings{shi2021author,
  author = {Liang Shi and Beichen Li and Chang-Il Kim and Petr Kellnhofer and Wojciech Matusik},
  title = {{Author Correction: Towards real-time photorealistic 3D holography with deep neural networks}},
  booktitle = {Nature},
  year = {2021},
  doi = {10.1038/s41586-021-03476-5},
}
Figure