Light Field Neural Network

Yuchi Huo, Sung‐Eui Yoon

TL;DR

This paper introduces the Light Field Neural Network (LFNN), a novel optical neural network system built from off-the-shelf components. LFNNs promise light-speed inference with significantly lower power consumption, enabling new possibilities for optical computation, advanced displays, and augmented/mixed reality applications.

What Problem Does This Paper Address?

Traditional electronic processors for deep neural networks (DNNs) face limitations in terms of size, power consumption, computation cost, and latency. These issues become particularly problematic for embedded, mobile, and real-time systems. While optical neural networks (ONNs) offer a promising alternative due to their inherent parallelism and power efficiency, existing ONN approaches often rely on complex nanophotonic circuits or diffraction layers, which can introduce coherence issues, phase encoding noise, and a lack of programmability. This paper addresses the need for a more practical, programmable, and efficient ONN system that can be built using readily available components.

Methods

The core innovation is the Light Field Neural Network (LFNN) structure, which leverages a light field modulation technique using common display panels (like OLEDs) and optical components (like lenses). Instead of complex nanophotonic circuits, LFNNs utilize a layered approach:

  • Scattering Layers: Made from diffuse or non-linear transmission materials, these layers gather and propagate energy, acting as virtual neurons. The number and size of these neurons can be adapted.
  • Bias and Addition: Simulated by transparent OLED (tOLED) layers or holographic optical elements (HOEs).
  • Matrix Multiplication (Weights): Achieved using multiple liquid crystal layers with polarizers, which can be configured to form an attenuation matrix.
  • Non-linearity (Activation Function 'f'): Implemented using non-diffuse transmission materials.

The system is called a "light field" neural network because weights and biases are modulated by light field synthesis techniques, and signals are transported via light. The paper also proposes advanced network structures, including incorporating phase encoding patterns for negative operations and using static diffraction patterns (like holograms) for improved power efficiency by acting as general feature extractors before dynamic LFNN layers.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Off-the-Shelf Components for ONNs: The most significant insight is the feasibility of building programmable ONNs using readily available, non-specialized optical and display components, moving beyond highly specialized nanophotonic fabrication.
  • Light Field Modulation: The core mechanism of using light field modulation to represent weights and propagate signals offers a new paradigm for optical computation.
  • Integrated Processor and Display: LFNNs uniquely combine neural network processing and display capabilities, paving the way for systems that can process and display light signals simultaneously.
  • Addressing Vergence-Accommodation Conflict: The LFNN's light field display capabilities are highlighted as a potential solution for the vergence-accommodation conflict in head-mounted displays for VR/AR/MR.
  • Passive Image Processing: The ability to process incoherent light opens up applications for passive image processing, such as enhancing low-light images or denoising without GPUs.

Key Results

The paper primarily focuses on the proposed system architecture and its potential applications rather than presenting specific quantitative results from a fully implemented prototype. The authors emphasize the potential benefits:

  • Speed of Light Inference: Processing occurs at the speed of light.
  • Low Power Consumption: Significantly lower power usage compared to electronic systems.
  • Near-Zero Latency and Computation Cost: Due to direct optical processing.
  • High Resolution/Frame Rate: Enabled by display technologies.
  • Programmability: The system is designed to be programmable.
  • See-Through Capability: Facilitates AR/MR applications.

The paper discusses several application prototypes, including optical computation, light field display, augmented/mixed reality (AR/MR) display, and passive light/image processing.

Limitations and Future Work

  • Prototype Development Timeline: The authors acknowledge that the physical prototype implementation may take longer than initially expected, suggesting that detailed physical data is still pending.
  • Power Efficiency of Dynamic Layers: While dynamic LFNN layers (OLED, liquid-crystal) offer advantages, they might affect power efficiency. The proposed hybrid module with static diffraction layers aims to mitigate this.
  • Future Work: The paper suggests future work in exploring various applications of LFNNs, including optical computation, advanced displays (3D, light field), and AR/MR systems. The integration of light field synthesis with LFNNs for displays is a key area for further development.

Summary updated

Abstract

We introduce an optical neural network system made by off-the-shelf components. In order to test the evaluate the physical property of the proposed system, we are making a prototype. After further discussions with our cooperators, we are agreed that the prototype implementation may take longer time than we expected earlier. Therefore we reach a consensus on withdrawing the paper until the physical data is available.
BibTeX
@misc{huo2018light,
  author = {Yuchi Huo and Sung‐Eui Yoon},
  title = {{Light Field Neural Network}},
  year = {2018},
  eprint = {1809.07009},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1809.07009},
}
Figure