VID-WIN: Fast Video Event Matching With Query-Aware Windowing at the Edge for the Internet of Multimedia Things

Piyush Yadav, Dhaval Salwala, Edward CurryIEEE Internet of Things Journal 2021 · DOI

TL;DR

This paper introduces VID-WIN, a novel two-stage windowing approach for efficient video event matching on resource-constrained edge devices. By intelligently processing video streams at both the edge and cloud, VID-WIN significantly boosts throughput and reduces latency and bandwidth usage while maintaining high accuracy for complex event queries.

What Problem Does This Paper Address?

The Internet of Multimedia Things (IoMT) generates massive amounts of video data from sources like CCTV cameras and smartphones. Processing this data for event detection (e.g., traffic analytics, object identification) is crucial but challenging, especially at the edge. Edge devices have limited computational power and memory, making it difficult to run computationally intensive Deep Neural Networks (DNNs) required for video analysis. While traditional Complex Event Processing (CEP) systems use windowing techniques, they often assume structured data and don't effectively handle the unstructured nature of video or the constraints of edge computing. This paper addresses the need for an efficient, adaptive, and resource-aware video event matching system that can operate effectively in an edge-cloud continuum.

Methods

VID-WIN employs a novel two-stage allied windowing approach:

  • Edge Window (winedge): This stage runs on the resource-constrained edge device. It acts as a pre-processing unit, performing:
  • Adaptive Micro-Batching: Groups similar frames into micro-batches to improve DNN processing efficiency.
  • Query-Aware Frame Resizing: Dynamically adjusts the resolution of micro-batches based on query accuracy requirements, preserving aspect ratios to minimize accuracy loss.
  • Query-Aware Adaptive Filtering: Implements a multi-stage filtering policy (eager, lazy with cache, and utility-based) to drop irrelevant frames or entire micro-batches based on video content, query relevance, and available edge resources (CPU/memory).
  • Cloud Window (wincloud): This stage runs on a high-end cloud node. It acts as a post-processing unit, receiving optimized frames from the edge. It can further refine processing, for example, by identifying Regions of Interest (ROIs) using a Spatial Mapper.

The system leverages input-based tunable knobs of DNN models (batch size, resolution, frame rate, ROI) and adapts them based on query characteristics and resource availability.

Datasets

The authors evaluated VID-WIN on five real-world video datasets:

  • Jackson Hole
  • Southampton
  • Auburn Toomer's Corner
  • Sandy Lane
  • Times Square

Key Insights and Ideas

  • Edge-Cloud Synergy for Video: The paper demonstrates that a two-stage approach, with intelligent pre-processing at the edge and post-processing in the cloud, is highly effective for video event analytics.
  • Query-Awareness is Key: Adapting processing (like resizing and filtering) based on the specific query (e.g., required accuracy, objects of interest) is crucial for balancing performance and accuracy.
  • Content-Driven Filtering Saves Resources: By analyzing video content and query relevance, VID-WIN can discard a significant amount of data, leading to substantial bandwidth and resource savings.
  • Micro-Batching for DNN Efficiency: Grouping frames into micro-batches significantly improves DNN throughput and reduces latency compared to frame-by-frame processing.
  • Aspect Ratio Preservation Matters for Accuracy: Resizing frames while maintaining their aspect ratio leads to much better accuracy than simply resizing to a square.

Key Results

  • Throughput: VID-WIN achieves approximately 2.3X higher throughput compared to other baselines.
  • Latency: Minimal latency is maintained, with significant reductions observed (e.g., 4.3X less batch latency than a 100-frame batch).
  • Bandwidth Reduction: Achieves ~99% bandwidth reduction compared to other baselines.
  • Resource Usage: Significantly reduces CPU and memory usage on edge devices.
  • Accuracy: Maintains high query-level accuracy, with only a minor drop (e.g., ~1.4% for event accuracy) compared to unfiltered approaches.

Limitations and Future Work

  • Fixed Background Assumption: The current VID-WIN controller design assumes videos with a fixed background, which is common for CCTV cameras but might not apply to all scenarios.
  • Edge-to-Edge MEC: The current design is limited to an edge-cloud scenario and cannot be deployed in edge-to-edge MEC settings due to the computational demands of video processing and state management.
  • Spatiotemporal Queries: The current work focuses on temporal queries. Future work aims to incorporate the spatiotemporal nature of videos into the filtering strategy for more complex event matching.

Summary updated

BibTeX
@article{yadav2021vid,
  author = {Piyush Yadav and Dhaval Salwala and Edward Curry},
  title = {{VID-WIN: Fast Video Event Matching With Query-Aware Windowing at the Edge for the Internet of Multimedia Things}},
  journal = {IEEE Internet of Things Journal},
  year = {2021},
  doi = {10.1109/jiot.2021.3075336},
}
Figure