Abstract
Motion estimation is a kernel algorithm in many video applications. Full search integer motion estimation (FSIME) can find the best result but it usually takes plenty of time. Previous work on accelerating FSIME exploited intra-frame data reuse within reference frames. We propose a novel inter-frame data reuse scheme that significantly reduces off-chip memory traffic by scheduling frame processing to load each frame into on-chip memory only once.
Introduction
In modern video processing, Motion Estimation (ME) is essential for both compression and Frame Rate Up-Conversion (FRUC). The latter is responsible for the smooth motion seen in high-frame-rate broadcasts. While Full Search Integer Motion Estimation (FSIME) offers the highest accuracy, its implementation is frequently bottlenecked by the "memory wall"—the latency and power consumption associated with fetching data from off-chip DRAM.
In traditional FRUC-ME workflows, a single video frame is often loaded from off-chip memory twice: first as the "current" frame and subsequently as the "reference" frame for the following set of motion vectors. This redundancy leads to wasted bandwidth and increased energy consumption.
Proposed Inter-Frame Data Reuse Schemes
To address these inefficiencies, this research proposes a shift from intra-frame to inter-frame data reuse. By interleaving the processing of adjacent frames, a single frame residing in the on-chip buffer can simultaneously serve as the reference for one calculation and the current frame for another.
The paper categorizes these schemes into three levels based on hardware constraints:
- Inter-E: Maintains two entire frames on-chip, maximizing bandwidth savings but requiring significant memory.
- Inter-D: Stores "Search Range Strips" (SRS) on-chip. This level is identified as the optimal "sweet spot" for modern hardware architectures, balancing memory footprint with performance.
- Inter-C: Stores individual search ranges, requiring minimal buffer space while still enabling substantial reuse.
Interleaved Scheduling Logic
The core innovation lies in the scheduling of frame processing. Instead of a strictly linear approach, the system alternates between "current" frames. This allows the hardware to satisfy multiple temporal requirements with a single memory fetch, effectively achieving a "load once" philosophy for the video stream.
To quantify these improvements, the authors redefined the Redundancy Access Factor ($Ra$) to include the memory cost of loading the current frame, a metric often overlooked in previous studies.
Performance Evaluation
The proposed schemes were evaluated across standard resolutions including 1080p, 720p, and 4K. The quantitative results demonstrate significant reductions in off-chip memory traffic compared to traditional intra-frame methods:
- Inter-C: 18.8% reduction.
- Inter-C+: 30% reduction.
- Inter-D: 37.5% reduction.
For a 4K 60fps video stream, the Inter-D method reduced the required off-chip bandwidth from 995 MByte/sec to 622 MByte/sec. These gains are consistent across resolutions, proving the scalability of the logic.
Conclusion
The proposed inter-frame data reuse method offers a hardware-friendly solution to the memory bottleneck in motion estimation. By optimizing scheduling and buffer utilization, it achieves substantial bandwidth savings without sacrificing the regularity required for ASIC or GPU acceleration. Future research will explore the integration of fractional motion estimation and detailed physical power analysis to further refine efficiency on mobile platforms.
Summary updated