Motion Focus Recognition in Fast-Moving Egocentric Video

Si-En Hong, James Tribble, Alexander Lake, Hao Wang, Chaoyi Zhou, Ashish Bastola, Siyu Huang, Eisa Chaudhary, Brian Canada, Ismahan Arslan-Ari, Abolfazl Razi

TL;DR

This paper introduces a real-time motion focus recognition method that estimates a person's locomotion intent directly from egocentric (first-person) video without requiring any additional training. By optimizing a heavy foundation model with a sliding-window inference strategy and physical motion priors, the system achieves over 30 FPS on consumer hardware, making motion-centric analysis practical for edge deployment in sports and robotics.

What Problem Does This Paper Address?

Egocentric vision—capturing video from a first-person perspective via wearable cameras or smart glasses—is inherently messy. Unlike stable cameras mounted on autonomous vehicles, egocentric video suffers from constant head bobbing, body sway, and sudden gaze shifts, leading to highly unstable viewpoints and noisy motion feedback.

Traditional attention or visual saliency models rely heavily on static or appearance-based cues (i.e., what object stands out visually). However, in fast-moving scenarios like running, biking, or scootering, these visually striking objects are often transient and irrelevant to actual navigation. Static models fail to capture the temporally persistent motion cues that indicate where a person is actually heading. Furthermore, state-of-the-art foundation models capable of understanding these environments are computationally heavy, quickly running out of memory on long video sequences and preventing real-time deployment on edge devices.

Methods

The authors build a lightweight Motion Focus Recognition framework that operates in real time through two main innovations:

  1. System-Level Optimization (Sliding-Window Inference): Instead of feeding an entire video into the foundation model (Depth Anything 3), which crashes standard GPUs on sequences longer than 1000 frames, they process the video in overlapping batches (sliding windows). To maintain temporal consistency across batches, they implement an incremental anchoring process that locks gravity-aligned rotation (pitch and roll) and only propagates horizontal yaw and translation, preventing global drift.
Figure 2. Framework for motion focus prediction and projection.
  1. Acceleration-Based Motion Focus: Instead of using velocity, the method calculates a discrete acceleration vector across three consecutive frames in the world coordinate space. This vector is transformed into the local camera frame and projected onto the image plane using a standard pinhole camera model. By aggregating these points over a short window (e.g., $N=15$ frames) and applying Gaussian kernels, they generate a "Motion Focus Map" that naturally anticipates turns and decelerations.
Sliding window strategy for real-time depth alignment.

Datasets

Because existing egocentric datasets focus heavily on action recognition rather than fast locomotion, the authors collected a new, lightweight egocentric action video dataset:

  • Data Collection: Recorded via a monocular handheld smartphone mounted on a specialized rig carried by a walking, scootering, or biking subject.
  • Environments: Challenging winter conditions (outdoor sidewalks, pedestrian walkways, uneven surfaces, and icy roads).
  • Scale: Contains 4 distinct scenes (Suburban, City, Town, Campus) across 206 clips, totaling over 370,000 frames and more than 2.5 hours of footage at varying resolutions (up to 4K) and frame rates (30/60 FPS).
Figure 1. (a) Selected frame from the dataset collected during the winter season. (i) is the raw city street image, (ii) is the motion- focus map, and (iii) is the motion-guided depth map. (b) is the example camera system for data capturing.

Key Insights and Ideas

  • Acceleration > Velocity for Intent: Predicting where someone will go is better captured by acceleration than velocity. Acceleration naturally captures the intent to turn or slow down before the velocity vector fully shifts.
  • Physical Priors Eliminate Training: By leveraging the geometric and physical constraints of camera pose dynamics, the system requires zero additional training or fine-tuning, making it highly adaptable to any camera setup out of the box.
  • Filtering Visual Clutter: The motion focus map successfully suppresses visually salient but navigationally irrelevant objects (like parked cars on the side) while highlighting immediate path obstacles (like trees, poles, or trash bins).

Key Results

  • Real-Time Performance: Using the DA3 Base model at 384px resolution, the system achieves 34.01 FPS using 5.5 GB of GPU memory.
  • Edge-Friendly Scaling: By switching to the DA3 Small variant, performance reaches 38.44 FPS while dropping memory consumption to just 3.4 GB, making it highly viable for consumer-grade hardware.
  • Qualitative Success: In testing across diverse scenarios (scootering, walking, biking), the motion focus accurately tracked the user's torso-driven locomotion path even when the user turned their head away from the direction of travel.

Limitations and Future Work

The authors note a clear trade-off in batch sizing: while smaller batch sizes reduce peak memory consumption, they also lead to decreased throughput due to increased I/O and scheduling overhead.

For future work, the authors suggest that grounding egocentric perception in physically significant motion is an essential stepping stone for downstream tasks like human-robot interaction, assistive navigation for the visually impaired, and real-time sports analytics.

Summary updated

Abstract

From Vision-Language-Action (VLA) systems to robotics, existing egocentric datasets primarily focus on action recognition tasks, while largely overlooking the inherent role of motion analysis in sports and other fast-movement scenarios. To bridge this gap, we propose a real-time motion focus recognition method that estimates the subject's locomotion intention from any egocentric video. We leverage the foundation model for camera pose estimation and introduce system-level optimizations to enable efficient and scalable inference. Evaluated on a collected egocentric action dataset, our method achieves real-time performance with manageable memory consumption through a sliding batch inference strategy. This work makes motion-centric analysis practical for edge deployment and offers a complementary perspective to existing egocentric studies on sports and fast-movement activities.
BibTeX
@inproceedings{hong2026motion,
  author = {Si-En Hong and James Tribble and Alexander Lake and Hao Wang and Chaoyi Zhou and Ashish Bastola and Siyu Huang and Eisa Chaudhary and Brian Canada and Ismahan Arslan-Ari and Abolfazl Razi},
  title = {{Motion Focus Recognition in Fast-Moving Egocentric Video}},
  booktitle = {BSN},
  year = {2026},
  url = {https://openreview.net/forum?id=gcSuBwk2r0},
}
Figure