Visual Navigation with a 2-pixel Camera - Possibilities and Limitations

John Baillieul, Feiyang KangarXiv 2021 · OpenReview

Abstract

Borrowing terminology from fluid mechanics, the concepts of Eulerian and Lagrangian optical flow sensing are introduced. Eulerian optical flow sensing assumes that each photoreceptor in the camera or eye can instantaneously detect feature image points and their velocities on the retina. If this assumption is satisfied, even a two pixel imaging system can provide a moving agent with information about its movement along a corridor that is sufficiently precise as to be used as a robustly reliable steering signal. Implementing Eulerian optical flow sensing poses significant challenges, however. Lagrangian optical flow, on the other hand, tracks feature image points as they move on the retina. This form of visual sensing is the basis for many standard computer vision implementations, including Lukas-Kanade and Horn-Schunck. Lagrangian optical flow has its own challenges, not least of which is that it is badly confounded by rotational components of motion. Combined steering and sensing strategies for mitigating the effects of rotational motions are considered.

Introduction: The Challenge of Rotational Confounding

Navigating using optical flow—the pattern of apparent motion of objects in a visual scene—is a staple of bio-inspired robotics. However, extracting a clean steering signal from a moving camera is notoriously difficult. The primary challenge is rotational confounding: when a robot turns, the resulting image motion is a mix of its forward progress (translation) and its rotation. For a robot trying to stay centered in a corridor, these rotations "pollute" the visual cues needed to estimate distance to the walls. This research investigates the minimal sensing requirements for stable navigation and how to mitigate the errors introduced when a robot inevitably has to turn.

Sensing Paradigms: Eulerian vs. Lagrangian Flow

The authors distinguish between two types of optical flow sensing:

  1. Eulerian Sensing: Inspired by fluid mechanics, this assumes sensors (photoreceptors) at fixed locations on the "eye" can instantaneously detect the velocity of features passing through them. Under this assumption, a 2-pixel camera is theoretically sufficient for robust corridor following.
  2. Lagrangian Sensing: This is the standard computer vision approach (e.g., Lucas-Kanade), where specific features are tracked as they move across the image plane over time. While more common, it is highly sensitive to rotational noise.

Steering via Time-to-Transit ($\tau$)

The core of the proposed control strategy is the time-to-transit ($\tau$). If a vehicle is moving toward a plane, $\tau$ is the time remaining before it crosses that plane. The authors propose a steering law where the turning rate is proportional to the difference in $\tau$ values perceived on the left and right sides of the camera's field of view: $u(t) = k(\tau_\ell - \tau_r)$.

Geometry of a vehicle navigating a corridor by balancing time-to-transit values from two symmetric sensors.

Mathematical proofs (Theorems 1 and 2) demonstrate that this law can asymptotically guide a robot to the center of a corridor. Furthermore, the law is robust to asymmetry; if the photoreceptors are not perfectly aligned, the robot simply converges to a line parallel to the walls rather than the exact center.

Perceptual Distortion and the Sense-Perceive-Act Cycle

A critical insight in the paper is the visualization of how turning affects perceived $\tau$. Under Lagrangian sensing, when a vehicle turns toward a feature, the perceived time-to-transit increases much more drastically than the actual geometric reality. This creates a dangerous feedback loop for control systems.

Comparison between geometric (ground truth) tau and perceived tau. Turning (green/red lines) causes the perceived value to deviate significantly from the blue "straight" path.

To overcome this, the authors suggest a "Sense-Perceive-Act" cycle. This sample-and-hold approach involves sensing and calculating $\tau$ while moving in short, straight segments (where rotation is zero), then using that "clean" data to execute a curved steering maneuver.

Conclusion and Limitations

This work provides a theoretical foundation for ultra-low-bandwidth visual navigation, proving that a 2-pixel camera can provide a stable steering signal. However, the models are currently stylized, assuming constant forward speed and perfectly detectable features in a 2D world. Future research will focus on testing these "Sense-Perceive-Act" cycles on physical laboratory robots in more complex, non-linear environments.

Summary updated

Abstract

Borrowing terminology from fluid mechanics, the concepts of {\em Eulerian} and {\em Lagrangian optical flow sensing} are introduced. Eulerian optical flow sensing assumes that each photoreceptor in the camera or eye can instantaneously detect feature image points and their velocities on the retina. If this assumption is satisfied, even a two pixel imaging system can provide a moving agent with information about its movement along a corridor that is sufficiently precise as to be used as a robustly reliable steering signal. Implementing Eulerian optical flow sensing poses significant challenges, however. Lagrangian optical flow, on the other hand, tracks feature image points as they move on the retina. This form of visual sensing is the basis for many standard computer vision implementations, including Lukas-Kanade and Horn-Schunck. Lagrangian optical flow has its own challenges, not least of which is that it is badly confounded by rotational components of motion. Combined steering and sensing strategies for mitigating the effects of rotational motions are considered.
BibTeX
@misc{baillieul2021visual,
  author = {John Baillieul and Feiyang Kang},
  title = {{Visual Navigation with a 2-pixel Camera - Possibilities and Limitations}},
  year = {2021},
  eprint = {2103.00285},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.00285},
}
Figure