N-QR: Natural Quick Response Codes for Multi-Robot Instance Correspondence

Nathaniel Moore Glaser, Rajashree Ravi, Zsolt Kira

Abstract

In large-scale robotic systems such as vertical farms, tracking individual objects across thousands of robot stations is a significant challenge due to visual ambiguity, non-rigid growth, and bandwidth constraints. This paper introduces N-QR (Natural Quick Response codes), a decentralized system that treats the unique, natural patterns of plants and soil as biometric identifiers. By employing a multi-stage pipeline of alignment, patch ensembling, and iterative bandwidth-efficient transmission, N-QR achieves high-accuracy instance correspondence. Experimental results from a production-scale vertical farm demonstrate a top-1 retrieval accuracy of 88.2%, a 12.5x reduction in bandwidth usage, and a 20.5x speedup in computation compared to centralized baselines.

Introduction: The Challenge of Tracking in Robotic Farming

Modern industrial automation, particularly in vertical farming, requires tracking millions of individual biological entities from seed to harvest to optimize yield. In systems like those operated by Bowery Farming, plants are moved between various robotic modules, such as seeding stations and growing racks. Establishing correspondence between these stations is difficult for several reasons:

  1. Scale: Managing millions of plants across thousands of robots makes centralized processing a bottleneck.
  2. Visual Ambiguity: Standard planting trays and soil cells appear nearly identical. The most prominent features (grid lines) are non-unique, while the identifying features (soil texture, sprout shape) are subtle and evolving.
  3. Non-Rigid Growth: Unlike static industrial parts, plants undergo continuous changes in shape, size, and color, rendering traditional corner-based feature matching ineffective.
The challenge of establishing correspondence between seeding and growing robots in a large-scale vertical farm.

Methodology: The N-QR Pipeline

The N-QR system addresses these challenges by breaking the correspondence problem into two primary stages: spatial alignment and discrete matching.

Alignment and Normalization

To ensure robots are comparing the same physical regions, N-QR utilizes specialized neural networks (BBox NN and Vertex NN) to detect the corners of planting rafts and individual cells. This allows the system to warp images into a normalized coordinate frame, mitigating perspective shifts between different camera stations.

Patch Ensembling and Metric Learning

Rather than processing high-resolution global images, N-QR samples multiple small patches from corresponding locations. A ResNet-based encoder extracts features from these patches, which are then ensembled. This approach forces the model to ignore misleading global similarities (like the plastic tray) and focus on the "weak" but unique natural features of the soil and plant. The model is trained using a triplet loss function with hard-negative sampling to maximize the distance between similar-looking neighbors.

Decentralized and Bandwidth-Efficient Transmission

To maintain scalability, N-QR moves computation to the "edge" (the individual robots). Instead of transmitting full images, robots iteratively exchange low-dimensional embeddings. If a match is determined with high confidence based on a distance threshold $\delta$, transmission stops. If the match is ambiguous, more detailed embeddings are sent sequentially.

The N-QR architecture featuring patch extraction, feature encoding, and iterative transmission logic.

Experimental Evaluation

The system was evaluated using a unique, real-world dataset from a production-scale vertical farm consisting of 1 seeding robot and 100 growing robots. The dataset included 17 unique seeding rafts matched against a pool of 473 growing rafts, covering approximately 700,000 individual plants across 10 species over several months.

Key Results

  • Accuracy: N-QR achieved a top-1 retrieval accuracy of 88.2% in a 1-vs-many matching scenario, significantly outperforming standard Siamese Networks or DeepMIL. In pairwise matching across a 100-robot farm, accuracy reached 99.8%.
  • Efficiency: The decentralized approach and iterative transmission policy resulted in a 20.5x speedup (reducing processing from ~21 minutes to 64 seconds) and a 12.5x reduction in network bandwidth.
  • Biological Insights: By successfully tracking 700k plants, researchers discovered that seeds planted in the center of a cell have a 52.9% growth rate, compared to 29.1% for those near the edges. This insight—impossible without individual tracking—allows for immediate ROI by refining seeding robot precision.

Conclusion and Future Work

N-QR demonstrates that "natural QR codes"—the inherent biometric patterns of biological objects—can serve as robust identifiers in massive multi-robot swarms. While resolution disparities between cameras remain a challenge, the system's ability to handle non-rigid growth and extreme visual similarity marks a significant step forward for robotic ecology. Future research will explore applying N-QR to broader localization and mapping tasks in environments where traditional landmarks are unreliable.

Summary updated

BibTeX
@inproceedings{glaser2024n,
  author = {Nathaniel Moore Glaser and Rajashree Ravi and Zsolt Kira},
  title = {{N-QR: Natural Quick Response Codes for Multi-Robot Instance Correspondence}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  doi = {10.1109/icra57147.2024.10611236},
}
Figure