TL;DR
This paper introduces a real-time motion focus recognition method that estimates a person's locomotion intent directly from egocentric (first-person) video without requiring any additional training. By optimizing a heavy foundation model with a sliding-window inference strategy and physical motion priors, the system achieves over 30 FPS on consumer hardware, making motion-centric analysis practical for edge deployment in sports and robotics.
What Problem Does This Paper Address?
Egocentric vision—capturing video from a first-person perspective via wearable cameras or smart glasses—is inherently messy. Unlike stable cameras mounted on autonomous vehicles, egocentric video suffers from constant head bobbing, body sway, and sudden gaze shifts, leading to highly unstable viewpoints and noisy motion feedback.
Traditional attention or visual saliency models rely heavily on static or appearance-based cues (i.e., what object stands out visually). However, in fast-moving scenarios like running, biking, or scootering, these visually striking objects are often transient and irrelevant to actual navigation. Static models fail to capture the temporally persistent motion cues that indicate where a person is actually heading. Furthermore, state-of-the-art foundation models capable of understanding these environments are computationally heavy, quickly running out of memory on long video sequences and preventing real-time deployment on edge devices.
Methods
The authors build a lightweight Motion Focus Recognition framework that operates in real time through two main innovations:
- System-Level Optimization (Sliding-Window Inference): Instead of feeding an entire video into the foundation model (Depth Anything 3), which crashes standard GPUs on sequences longer than 1000 frames, they process the video in overlapping batches (sliding windows). To maintain temporal consistency across batches, they implement an incremental anchoring process that locks gravity-aligned rotation (pitch and roll) and only propagates horizontal yaw and translation, preventing global drift.
- Acceleration-Based Motion Focus: Instead of using velocity, the method calculates a discrete acceleration vector across three consecutive frames in the world coordinate space. This vector is transformed into the local camera frame and projected onto the image plane using a standard pinhole camera model. By aggregating these points over a short window (e.g., $N=15$ frames) and applying Gaussian kernels, they generate a "Motion Focus Map" that naturally anticipates turns and decelerations.
Datasets
Because existing egocentric datasets focus heavily on action recognition rather than fast locomotion, the authors collected a new, lightweight egocentric action video dataset:
- Data Collection: Recorded via a monocular handheld smartphone mounted on a specialized rig carried by a walking, scootering, or biking subject.
- Environments: Challenging winter conditions (outdoor sidewalks, pedestrian walkways, uneven surfaces, and icy roads).
- Scale: Contains 4 distinct scenes (Suburban, City, Town, Campus) across 206 clips, totaling over 370,000 frames and more than 2.5 hours of footage at varying resolutions (up to 4K) and frame rates (30/60 FPS).
Key Insights and Ideas
- Acceleration > Velocity for Intent: Predicting where someone will go is better captured by acceleration than velocity. Acceleration naturally captures the intent to turn or slow down before the velocity vector fully shifts.
- Physical Priors Eliminate Training: By leveraging the geometric and physical constraints of camera pose dynamics, the system requires zero additional training or fine-tuning, making it highly adaptable to any camera setup out of the box.
- Filtering Visual Clutter: The motion focus map successfully suppresses visually salient but navigationally irrelevant objects (like parked cars on the side) while highlighting immediate path obstacles (like trees, poles, or trash bins).
Key Results
- Real-Time Performance: Using the
DA3 Basemodel at 384px resolution, the system achieves 34.01 FPS using 5.5 GB of GPU memory. - Edge-Friendly Scaling: By switching to the
DA3 Smallvariant, performance reaches 38.44 FPS while dropping memory consumption to just 3.4 GB, making it highly viable for consumer-grade hardware. - Qualitative Success: In testing across diverse scenarios (scootering, walking, biking), the motion focus accurately tracked the user's torso-driven locomotion path even when the user turned their head away from the direction of travel.
Limitations and Future Work
The authors note a clear trade-off in batch sizing: while smaller batch sizes reduce peak memory consumption, they also lead to decreased throughput due to increased I/O and scheduling overhead.
For future work, the authors suggest that grounding egocentric perception in physically significant motion is an essential stepping stone for downstream tasks like human-robot interaction, assistive navigation for the visually impaired, and real-time sports analytics.
Summary updated