Bandwidth limited object recognition in high resolution imagery.

Laura Lopez-Fuentes, Andrew D. Bagdanov, Joost van de Weijer, Harald Skinnemoen

TL;DR

This paper introduces an "active information seeking" framework that slashes bandwidth requirements for high-resolution object recognition by up to 90%. By intelligently requesting only small, high-resolution sub-regions of an image—such as a license plate on a detected car—the system maintains high recognition accuracy while transmitting significantly fewer pixels than traditional compression methods.

What Problem Does This Paper Address?

Modern smartphones and drones capture images at 10-20 megapixels, providing rich detail necessary for tasks like license plate transcription or face recognition. However, in critical scenarios—such as disaster relief, fire detection via drones, or satellite communications—bandwidth is extremely limited and expensive (sometimes costing $1 per megabyte).

Standard solutions like JPEG compression or downscaling often fail because they discard the high-frequency details needed for fine-grained recognition. As shown in the figure below, a downscaled image might look fine to a human, but the "zoomed-in" details required for OCR (Optical Character Recognition) are completely lost.

Figure 1: The top-left image corresponds to the original 16 megapixel image, and the top-right image to a downscaled version at 640×480 pixels. The information loss may not be visible at first glance, however when zooming in it is clear that valuable information such as faces or license plate numbers have been lost.

The authors aim to bridge this gap: how can we perform high-resolution recognition at the server-side without sending the entire high-resolution image?

Methods

The authors propose two models based on the concept of active information seeking, which mimics how a human operator might look at a low-res thumbnail and then request a high-res "zoom" of a specific area.

  1. Recursive Fast-RCNN: This is a multi-step approach. The system first inspects a low-resolution version of the whole image to detect "parent" objects (e.g., cars). It then requests higher-resolution versions of those specific regions to detect "child" objects (e.g., license plates). Finally, it requests the license plate at full resolution for OCR.
  2. Multi-stage Network: This is a more aggressive optimization. Instead of a separate detection step for the child object, the network detects the car and simultaneously predicts the coordinates of the license plate from the low-resolution input. This allows the system to skip an entire transmission stage, requesting the high-res license plate pixels directly.
Figure 2: A graphical overview of our approach. A low resolution version of the original image is first inspected to identify regions of interest (e.g. cars if we are searching for license plates). These regions are then inspected at higher resolution to search for objects of higher semantic interest (e.g. license plates themselves).

To handle the uncertainty of predicting a tiny license plate from a low-res car image, the Multi-stage model uses a clever margin mechanism. If the network is very confident in its prediction, it requests a tight crop; if confidence is low, it requests a larger margin around the predicted area to ensure the plate is captured.

Datasets

The researchers introduced a new Car Recognition Dataset specifically designed for this task. Existing datasets usually focus on either just cars (at low res) or just license plates (tightly cropped). This new dataset includes:

  • 500 images taken in various cities, times, and lighting conditions.
  • Manual annotations for both car and license plate bounding boxes.
  • Difficulty labels: "Easy", "Medium", and "Hard" based on occlusion and size.
  • Transcriptions: The actual text of the license plates for OCR evaluation.
Figure 4: Examples from the new dataset showing Easy (green), Medium (yellow), and Hard (red) annotations.

Key Insights and Ideas

  • Semantic Hierarchy Matters: The core "aha" moment is leveraging structural relationships. You don't need to search the whole sky for a license plate; you only need to find a car first. This hierarchy acts as a powerful prior that reduces false positives.
  • CNNs are Resolution-Limited: Most state-of-the-art CNNs (like AlexNet) resize inputs to tiny squares (e.g., 224x224). The authors argue that instead of trying to make CNNs handle massive images, we should use them to "point" to where the high-res data is needed.
  • Bandwidth vs. Accuracy Trade-off: The Multi-stage model is faster and uses less bandwidth than the Recursive model, but it is slightly less accurate because predicting a sub-object's location from low-res data is inherently difficult.

Key Results

The proposed methods were compared against a JPEG baseline where the entire image is compressed at various quality levels.

  • Bandwidth Savings: The Recursive Fast-RCNN reduced bandwidth usage by one order of magnitude (approx. 90%) compared to sending the full image.
  • Accuracy: The system achieved 64% character accuracy, which is only 1% lower than the baseline that uses the original high-resolution image, but at a fraction of the data cost.
  • Efficiency: The Multi-stage Network was even more efficient, inspecting only 17.6% of the total pixels on average, compared to 18.6% for the Recursive model.
Figure 5: Recall and accuracy results relative to image edge size.

Limitations and Future Work

While the results are promising, the authors note a few areas for improvement:

  • Dataset Size: The current dataset of 500 images is relatively small for deep learning standards; training on a larger corpus could improve the sub-object estimation.
  • OCR Integration: The OCR (Tesseract) was used as an off-the-shelf component. The authors suggest that integrating the OCR directly into the neural network pipeline could further improve character recognition accuracy.
  • Hybrid Compression: Future work could involve combining this "active seeking" approach with standard JPEG compression on the requested sub-regions to save even more bandwidth.

Summary updated

Abstract

This paper proposes a novel method to optimize bandwidth usage for object detection in critical communication scenarios. We develop two operating models of active information seeking. The first model identifies promising regions in low resolution imagery and progressively requests higher resolution regions on which to perform recognition of higher semantic quality. The second model identifies promising regions in low resolution imagery while simultaneously predicting the approximate location of the object of higher semantic quality. From this general framework, we develop a car recognition system via identification of its license plate and evaluate the performance of both models on a car dataset that we introduce. Results are compared with traditional JPEG compression and demonstrate that our system saves up to one order of magnitude of bandwidth while sacrificing little in terms of recognition performance.
BibTeX
@misc{lopezfuentes2017bandwidth,
  author = {Laura Lopez-Fuentes and Andrew D. Bagdanov and Joost van de Weijer and Harald Skinnemoen},
  title = {{Bandwidth limited object recognition in high resolution imagery.}},
  year = {2017},
  eprint = {1701.04210},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1701.04210},
}
Figure