Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing

Manchao Bao, Shengjiang Fang, Tao Yue, Xuemei Hu

Introduction

Depth sensing is a fundamental technology for autonomous navigation, robotics, and remote sensing. Currently, the field is divided between two primary methodologies: Direct Time-of-Flight (dToF) and Indirect Time-of-Flight (iToF). While dToF offers high accuracy at long ranges, it requires expensive, specialized hardware such as picosecond-resolution timers and ultra-short pulse lasers. Conversely, iToF is more cost-effective and utilizes standard CMOS sensors, but it is plagued by "phase wrapping"—a phenomenon where periodic wave signals cannot distinguish between objects at different multiples of the ambiguity range. Furthermore, iToF suffers from a significant drop in signal-to-noise ratio (SNR) as distance increases due to $1/d^2$ light attenuation.

To address these challenges, researchers from Nanjing University have introduced Burst-Encodable Time-of-Flight (BE-ToF). This new imaging paradigm bridges the gap between dToF and iToF by combining a novel "burst-mode" light emission strategy with an end-to-end learnable framework, achieving millimeter-level accuracy at distances up to 93 meters.

The BE-ToF Paradigm

The core innovation of BE-ToF lies in its modulation scheme. Traditional iToF typically uses continuous sine or square waves. BE-ToF instead employs Burst-Mode Modulation, where light is emitted in discrete bursts. Within a single long burst period ($T_{burst}$), a short pulse ($T_m$) is transmitted.

This approach provides two distinct advantages:

  1. Unambiguous Range: The burst period defines a massive unambiguous range, effectively eliminating the phase wrapping problem.
  2. High Precision: The narrow pulse width within the burst maintains the high precision typically associated with short-range sensors.

By ensuring the phase shift of the reflected signal spans the full $[0, 2\pi]$ range within a single burst, the system fundamentally resolves the periodic ambiguity that limits traditional iToF systems.

End-to-End Learnable Framework

BE-ToF does not rely on fixed, hand-crafted algorithms for depth calculation. Instead, it utilizes a joint optimization framework that treats both hardware coding (the camera's exposure patterns) and neural reconstruction as a single, differentiable pipeline.

Hardware-Aware Optimization

To ensure that the "learned" exposure codes are physically implementable, the authors incorporate specific hardware-aware constraints into the loss function:

  • Double Well Function Loss: This forces the learned coding patterns to be binary (0 or 1), matching the physical capabilities of sensor gates.
  • First-order Difference Loss: This prevents the generation of pulses that are too narrow for the hardware to accurately reproduce.

RSCF-Net Architecture

The reconstruction is handled by the Restormer-based Spatial-Channel Fusion Network (RSCF-Net). This network is designed to handle the low SNR conditions of long-distance sensing through two specialized components:

  • Channel Feature Extraction Block (CFEB): This block identifies and leverages relationships between different measurement channels to improve depth estimation.
  • Multi-scale Feature Fusion Block (MFFB): This component preserves critical spatial details, ensuring that object boundaries remain sharp even at extreme distances.

Fisher Guidance

A notable theoretical contribution is the inclusion of a Fisher Guidance Loss. Based on information theory, this loss guides the network toward coding functions that are mathematically optimal for depth estimation, providing a principled foundation for the learned hardware parameters.

Experimental Results and Validation

The system was rigorously tested using the NYU-V2 dataset, simulated for long-range scenarios, and validated with a physical hardware prototype featuring a 905nm fiber-based pulsed laser and an encodable image intensifier camera.

Quantitative Performance

In comparative tests across various ranges, BE-ToF demonstrated significant superiority over state-of-the-art methods, particularly in challenging low-SNR environments:

  • At 90-93 meters: BE-ToF achieved a Mean Absolute Error (MAE) of 29.58 mm.
  • Comparisons: Under the same conditions, traditional Sine-wave iToF resulted in an error of 244.29 mm, while advanced "DeepToF" methods recorded an error of 56.97 mm.

Real-World Prototype

The physical prototype successfully mapped complex outdoor environments at a distance of 23 meters. The system was able to clearly distinguish between intricate features such as tree branches, steps, and flower beds, while effectively suppressing background clutter that often confuses standard iToF sensors.

Conclusion and Future Work

BE-ToF represents a significant step forward in making long-range, high-fidelity depth sensing more accessible. By jointly optimizing hardware modulation and neural reconstruction, the system overcomes the traditional SNR and ambiguity barriers of indirect Time-of-Flight imaging.

While the current system requires an image intensifier, future research will focus on adapting these learnable burst codes for more common CMOS-based iToF sensors. Additionally, the authors highlight the need for responsible deployment, as the high-fidelity nature of long-distance imaging necessitates careful consideration of privacy in public spaces.

Summary updated

BibTeX
@inproceedings{bao2025learnable,
  author = {Manchao Bao and Shengjiang Fang and Tao Yue and Xuemei Hu},
  title = {{Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=zL4ifL17bU},
}
Figure