PixRO: Pixel-Distributed Rotational Odometry with Gaussian Belief Propagation

Ignacio Alzugaray, Riku Murai, Andrew J. DavisonarXiv 2024 · DOI · arXiv

Abstract

PixRO introduces a decentralized algorithm for 3D camera rotation estimation designed to run directly at the pixel level. By leveraging Gaussian Belief Propagation (GBP) on a hierarchical "sharded" graph topology, the system enables individual pixel-processors to achieve a global consensus on motion. This approach eliminates the need to transmit raw image data to a central processor, offering significant advantages in power efficiency, latency, and privacy. Experimental results on the Pano3D GibsonV2 dataset demonstrate that the sharded topology achieves convergence and accuracy comparable to centralized baselines while maintaining a decentralized architecture.

Introduction

In traditional computer vision, sensors act as passive collectors of light, transmitting massive amounts of redundant data to a central CPU or GPU for processing. This centralized model is inefficient, consuming high power and introducing latency and privacy risks. While "smart sensors" are emerging, most algorithms still treat image data as a globally accessible block. PixRO addresses this by deconstructing 3D motion estimation into local "conversations" between neighboring pixels, enabling on-sensor processing where raw data never leaves the pixel array.

Comparison of centralized off-sensor estimation versus pixel-distributed on-sensor estimation.

Methods

The researchers model rotation estimation as a Factor Graph problem where each pixel is a variable node trying to estimate the camera's global rotation ($SO(3)$). Three types of constraints (factors) are utilized: Photometric Data Factors (comparing pixel intensity across frames), Prior Factors (for stability), and Regularization Factors (the "social" component for local consensus).

To solve this without a central controller, Gaussian Belief Propagation (GBP) is employed, where nodes asynchronously pass messages (means and variances) to their neighbors. Two communication topologies were evaluated:

  • Flat Topology: Pixels communicate only with their immediate four neighbors.
  • Sharded Topology: A hierarchical pyramid structure where groups of pixels report to intermediate variables, leading to a single "apex" variable.

The algorithm was tested using 128x128 synthesized images from the Pano3D GibsonV2 dataset, simulating small camera rotations (up to 1°) to test convergence in realistic settings.

Results and Key Insights

The study revealed critical differences between the topologies. The Flat topology, while simpler, creates a graph full of loops that can lead to "overconfidence" and false certainty due to feedback cycles. In contrast, the Sharded topology functions as a tree, allowing information to propagate across the sensor in $O(\log N)$ steps rather than $O(N)$.

Normalized average rotational error over iterations for Centralized, Flat, and Sharded topologies.

As shown in the performance metrics, the Sharded topology reaches global consensus significantly faster than the Flat topology and achieves an error rate nearly identical to the centralized gold standard. Furthermore, the Sharded model proved robust under high sensor noise, whereas the Flat model often struggled to converge to a meaningful estimate. This hierarchical approach also ensures privacy, as raw photometric data never leaves the bottom level of the sensor.

Conclusion

PixRO demonstrates the feasibility of high-level motion estimation directly on-sensor. While current hardware like the SCAMP-7 lacks the floating-point capabilities for full GBP, this work provides a theoretical foundation for future programmable sensors. Future research will focus on handling larger motions through multi-scale processing and addressing dynamic scenes with independently moving objects.

Summary updated

BibTeX
@misc{alzugaray2024pixro,
  author = {Ignacio Alzugaray and Riku Murai and Andrew J. Davison},
  title = {{PixRO: Pixel-Distributed Rotational Odometry with Gaussian Belief Propagation}},
  year = {2024},
  eprint = {2406.09726},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.09726},
}
Figure