GLANCE: Gaze-Led Attention Network for Compressed Edge-inference

Neeraj Solanki, Hong Ding, Sepehr Tabrizchi, Ali Shafiee Sarvestani, Shaahin Angizi, David Z. Pan, Arman Roohi

Abstract

Real-time object detection in AR/VR systems faces critical computational constraints, requiring sub-10ms latency within tight power budgets. Inspired by biological foveal vision, we propose a two-stage pipeline that combines differentiable weightless neural networks for ultra-efficient gaze estimation with attention-guided region-of-interest object detection. Our approach eliminates arithmetic-intensive operations by performing gaze tracking through memory lookups rather than multiply-accumulate computations, achieving an angular error of $8.32^{\circ}$ with only 393 MACs and 2.2 KiB of memory per frame. Gaze predictions guide selective object detection on attended regions, reducing computational burden by 40-50% and energy consumption by 65%. Deployed on the Arduino Nano 33 BLE, our system achieves 48.1% mAP on COCO (51.8% on attended objects) while maintaining sub-10ms latency, meeting stringent AR/VR requirements by improving the communication time by $\times 177$. This work shows that memory-centric architectures with explicit attention modeling offer better efficiency and accuracy for resource-constrained wearable platforms than uniform processing.

Introduction

Modern AR/VR headsets like the Meta Quest Pro or Apple Vision Pro face a grueling "power-latency-accuracy" trilemma. To prevent motion sickness and maintain immersion, these devices need to detect objects with less than 10ms of latency. However, state-of-the-art detectors are computationally expensive, often requiring over $10^{11}$ FLOPs per frame—far exceeding the tight power budgets of battery-operated wearables.

The core inefficiency in standard detectors is that they treat every pixel equally, processing the entire field of view at high resolution. In contrast, human vision is highly selective; our fovea (the center of our gaze) captures high detail in only about 2° of our visual field. GLANCE aims to bring this "foveated" efficiency to edge AI by decoupling where to look from how to detect.

System Architecture

The GLANCE pipeline consists of two primary stages designed for resource-constrained wearable platforms.

The GLANCE system architecture showing the two-stage pipeline: (1) Always-on gaze sensing and (2) Host-side ROI-based object detection.
  1. Stage 1: Gaze Estimation: A low-power microcontroller processes eye images using Differentiable Weightless Neural Networks (DWNs) to predict 3D gaze direction.
  2. Stage 2: ROI Detection: Gaze predictions define a Region of Interest (ROI). The system creates a "union-of-ROIs mosaic"—a compact collage of the most important parts of the frame—which is transmitted to a host for detection.

Weightless Gaze Estimation

To achieve ultra-low-power gaze tracking, GLANCE utilizes DWNs. Unlike standard CNNs that rely on power-hungry multiply-accumulate (MAC) operations, DWNs perform inference primarily through memory lookup tables (LUTs).

Detailed architecture of the Differentiable Weightless Neural Network (DWN) for gaze estimation, utilizing Look-Up Tables (LUTs) to minimize MAC operations.

The DWN takes eye images (56x56) and maps features to LUT addresses via thermometer encoding. This "weightless" approach is incredibly lean, requiring only 393 MACs and 2.2 KiB of memory. This allows for "always-on" gaze tracking on the tiniest microcontrollers with negligible power draw.

Attention-Guided Detection and Stabilization

The gaze prediction guides the selection of ROIs from the primary camera. To ensure the system doesn't "lose" objects during head movements, GLANCE integrates several stabilization mechanisms:

  • Temporal Union: Maintains a temporal memory of attended regions over several frames.
  • Motion Stabilization: Uses IMU (Inertial Measurement Unit) data to stabilize the ROIs during head movements.
  • Mosaic Construction: By only transmitting small ROI crops (48–80 px) instead of full 640x640 frames, GLANCE reduces communication time by a staggering 177x.

Key Results

Evaluated on the MPIIGaze and COCO datasets, GLANCE demonstrates significant efficiency and accuracy gains:

  • Efficiency: Reduces computational burden by 40-50% and energy consumption by 65% compared to uniform processing.
  • Gaze Accuracy: The DWN achieved an angular error of 8.32°, sufficient for guiding attention despite being orders of magnitude lighter than previous models.
  • Detection Performance: On the COCO dataset, the system achieved 48.1% mAP. Notably, for small, medium, and large objects, the ROI-based method yielded 51.3%, 72.1%, and 88.1% accuracy, respectively—outperforming the global YOLOv12n baseline (39.2%, 63.4%, and 83.1%). This suggests that focusing representational capacity on a smaller crop avoids the "signal dilution" common in full-frame processing.
  • Hardware Deployment: Deployed on an Arduino Nano 33 BLE, the system maintained sub-10ms latency, meeting stringent AR/VR requirements.

Conclusion

GLANCE demonstrates that memory-centric architectures with explicit attention modeling offer superior efficiency for resource-constrained wearables. By mimicking the selective nature of human vision, the system overcomes the computational bottlenecks of real-time object detection in AR/VR. Future work will explore integrating multi-modal attention cues and further refining gaze estimation accuracy to allow for even tighter ROI crops.

Summary updated

Abstract

Real-time object detection in AR/VR systems faces critical computational constraints, requiring sub-10\,ms latency within tight power budgets. Inspired by biological foveal vision, we propose a two-stage pipeline that combines differentiable weightless neural networks for ultra-efficient gaze estimation with attention-guided region-of-interest object detection. Our approach eliminates arithmetic-intensive operations by performing gaze tracking through memory lookups rather than multiply-accumulate computations, achieving an angular error of $8.32^{\circ}$ with only 393 MACs and 2.2 KiB of memory per frame. Gaze predictions guide selective object detection on attended regions, reducing computational burden by 40-50\% and energy consumption by 65\%. Deployed on the Arduino Nano 33 BLE, our system achieves 48.1\% mAP on COCO (51.8\% on attended objects) while maintaining sub-10\,ms latency, meeting stringent AR/VR requirements by improving the communication time by $\times 177$. Compared to the global YOLOv12n baseline, which achieves 39.2\%, 63.4\%, and 83.1\% accuracy for small, MEDium, and LARGE objects, respectively, the ROI-based method yields 51.3\%, 72.1\%, and 88.1\% under the same settings. This work shows that memory-centric architectures with explicit attention modeling offer better efficiency and accuracy for resource-constrained wearable platforms than uniform processing.
BibTeX
@misc{solanki2026glance,
  author = {Neeraj Solanki and Hong Ding and Sepehr Tabrizchi and Ali Shafiee Sarvestani and Shaahin Angizi and David Z. Pan and Arman Roohi},
  title = {{GLANCE: Gaze-Led Attention Network for Compressed Edge-inference}},
  year = {2026},
  eprint = {2603.15717},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.15717},
}

Loading related content...

Figure