Abstract
The depth resolution of a continuous-wave time-of-flight (CW-ToF) imaging system is determined by its coding functions. Recently, there has been growing interest in the design of new high-performance CW-ToF coding functions. However, these functions are typically designed in a hardware agnostic manner, i.e., without considering the practical device limitations, such as bandwidth, source power, digital (binary) function generation. Therefore, despite theoretical improvements, practical implementation of these functions remains a challenge. We present a constrained optimization approach for designing practical coding functions that adhere to hardware constraints. The optimization problem is non-convex with a large search space and no known globally optimal solutions. To make the problem tractable, we design an iterative, alternating least-squares algorithm, along with convex relaxation of the constraints. Using this approach, we design high-performance coding functions that can be implemented on existing hardware with minimal modifications. We demonstrate the performance benefits of the resulting functions via extensive simulations and a hardware prototype.
Introduction
Continuous-Wave Time-of-Flight (CW-ToF) cameras, such as the original Microsoft Kinect, have become ubiquitous in consumer electronics for 3D sensing. These systems operate by illuminating a scene with modulated light and measuring the phase shift of the reflected signal at the sensor. The precision of these depth measurements is fundamentally tied to the "coding functions" used to modulate the light source and demodulate the sensor signal.
While theoretical advancements have introduced "Hamiltonian" codes capable of achieving near-optimal depth resolution, a significant gap exists between theory and practice. Most theoretical models assume idealized hardware capable of infinite peak power, infinite bandwidth, and arbitrary analog signal generation. In reality, off-the-shelf LEDs and CMOS sensors are constrained by finite power limits, low-pass filtering effects, and a preference for binary (ON/OFF) signaling. This paper bridges this gap by proposing a framework for designing hardware-aware coding functions.
Hardware-Aware Optimization Framework
The core of this research is a constrained optimization approach that treats the design of modulation ($M$) and demodulation ($D$) functions as a two-stage process.
1. Correlation Design
The first stage involves determining the ideal correlation function—the mathematical relationship between the light source and sensor—that minimizes the expected depth error for a given signal-to-noise ratio (SNR).
2. Practical Factorization
The second, more critical stage is factorizing this ideal correlation into practical $M$ and $D$ functions. Because this factorization is non-convex, the authors employ an iterative, alternating least-squares (ALS) algorithm. By fixing one function and solving for the other, the solver converges toward a practical pair of codes. Crucially, hardware constraints are integrated directly into this optimization:
- Peak Power Constraints: The light source modulation is limited to a maximum amplitude, preventing the solver from relying on physically impossible "delta-function" pulses.
- Bandwidth Constraints: The system models the frequency response of the hardware (e.g., using a Hanning window). This ensures that the designed codes do not contain high-frequency components that would be lost to electronic blurring.
- Binary Signaling: To accommodate digital hardware, the framework uses Total Variation (TV) regularization to encourage "square-like" functions, followed by a binary re-projection step to ensure compatibility with standard 2-bucket ToF sensors.
Experimental Results
The proposed framework was evaluated through extensive simulations and a custom hardware prototype. The results demonstrate that "hardware-aware" codes significantly outperform traditional sinusoidal modulation.
As shown in the performance plots, the redesigned Hamiltonian codes achieve a Mean Depth Error (MDE) that is 2–5x lower than standard sinusoids across a wide range of source and ambient power levels. One key insight is the effectiveness of "wide" pulses; when peak power is limited, the optimization favors pulses with a higher duty cycle that are carefully shaped to maximize energy without exceeding hardware limits.
In hardware experiments using a point-scanning CW-ToF system, the proposed codes maintained high accuracy even in low-SNR conditions. While traditional multi-frequency sinusoids often suffer from "phase unwrapping" errors—leading to massive depth outliers—the optimized codes remained robust. Qualitative tests on complex objects, such as a volleyball or textured scenes, showed smoother surfaces and significantly fewer artifacts at object boundaries.
Conclusion and Future Work
This work provides a practical path for implementing high-performance coding functions on existing ToF hardware. By accounting for physical limitations during the design phase, the authors have turned theoretical "Hamiltonian" codes into a viable technology for consumer-grade sensors.
However, challenges remain. The current optimization is a two-step heuristic; a unified "one-step" approach that optimizes $M$ and $D$ directly for depth error could potentially yield even better results. Furthermore, the current model does not account for Multi-path Interference (MPI), where light reflects off multiple surfaces before reaching the sensor. Future research incorporating MPI robustness into the hardware-aware optimization framework could further enhance the reliability of ToF imaging in complex indoor environments.
Summary updated