View planning methods of object detection: a survey

Jianyu Wang, Feng Zhu, Yingming Hao, Qun Wang, Pengfei Zhao, Haibo SunJournal of Image and Graphics 2024 · DOI

TL;DR

This paper provides a comprehensive survey of Active Object Detection (AOD), a paradigm shift where computer vision systems don't just "see" what is given, but actively adjust their camera parameters or spatial position to improve detection. By categorizing nearly two decades of research, the authors highlight how view planning overcomes challenges like poor lighting and occlusion, transforming object detection from a static task into an intelligent, goal-oriented behavior.

What Problem Does This Paper Address?

Standard object detection algorithms are passive: they process a single input image and hope for the best. However, in real-world scenarios—like a robot navigating a cluttered warehouse or a drone flying in low light—the initial viewpoint is often suboptimal. Objects might be occluded, too far away, or shrouded in shadows, leading to missed detections or false positives.

The authors address the challenge of View Planning, or "Active Vision." Instead of accepting a bad image, an active system asks: "Where should I move, or how should I change my camera settings, to see this object more clearly?" This involves a feedback loop where the system analyzes current detection confidence and plans the "Next Best View" (NBV) to maximize information gain.

Methods

The paper organizes AOD methods into a clear taxonomy based on the "degrees of freedom" the system can manipulate:

  1. 2D Pixel-level & Parameter Planning:
    • Pixel-based: The system identifies "regions of interest" within a high-resolution frame and "moves" its focus (digital zoom) to those areas to save computation while maintaining accuracy.
  2. Parameter-based: The system adjusts simulated or real camera settings like exposure, brightness, and resolution to mitigate environmental interference (e.g., fog or low light).
  3. 3D Spatial Movement Planning:
    • Known Spatial Relationships: Uses prior knowledge (like scene graphs or semantic priors—e.g., "forks are usually near plates") to plan an efficient path to a target.
  4. Unknown Spatial Relationships: The agent relies entirely on current visual observations and Reinforcement Learning (RL) to decide its next move in an unexplored environment.
  5. Hybrid Planning: A nascent field combining both spatial movement and internal parameter adjustment (e.g., moving closer and adjusting zoom simultaneously).

The survey highlights the transition from traditional probabilistic models (like Markov Decision Processes) to modern Deep Reinforcement Learning (DRL) architectures (like Deep Q-Networks) to handle the high-dimensional state spaces of 3D environments.

Datasets

The paper categorizes datasets based on the dimensionality of the task:

  • 2D Datasets: Standard benchmarks like PASCAL VOC and MS COCO are used to test pixel-level planning. Specialized datasets like RGB-T324 (thermal/visible) and SA (Small Airport remote sensing) are used for parameter-based AOD.
  • 3D Simulation Environments: House3D, MINOS, and 3D-FRONT allow agents to train in photorealistic virtual homes where lighting and layouts can be procedurally generated.
  • 3D Real-world Datasets: AVD (Active Vision Dataset) and T-LESS (for texture-less industrial objects) provide real-world images captured from multiple viewpoints, allowing researchers to test movement strategies without needing a physical robot.

Key Insights and Ideas

  • The "State Expression" Problem: In 3D AOD, how you represent the "state" is crucial. The most effective recent methods combine raw images with object bounding boxes and "cropped" views of potential targets to give the agent a better sense of its progress.
  • Greedy vs. Global: Most current AOD algorithms are "greedy"—they look for the next best view. The authors argue that true intelligence requires Global Optimization, where the agent plans a multi-step trajectory rather than just the immediate next step.
  • Semantic Priors: Knowing that a "toaster" is likely in a "kitchen" significantly reduces the search space. Integrating this "common sense" into neural networks via scene graphs is a major trend in improving search efficiency.
  • The Cost of Action: Unlike passive detection, active vision has a "cost" (battery life, time, mechanical wear). Effective view planning must balance the gain in detection accuracy against the distance traveled.

Key Results

  • Accuracy Boost: On the PASCAL VOC dataset, active methods (e.g., using RL to refine region proposals) achieved an AP50 of 81.6%, significantly outperforming standard passive baselines (77.3%–79.1%).
  • Efficiency in 3D: In the AVD dataset, top-tier AOD algorithms achieved Success Rates (SR) as high as 0.78, meaning the agent successfully found and accurately detected the target in nearly 80% of trials, even when starting from a random location.
  • Robustness: Active parameter adjustment (like the AHDet strategy) allows systems to maintain high detection rates in "extreme" conditions (low resolution or heavy fog) where passive detectors typically fail completely.

Limitations and Future Work

The authors identify several "bottlenecks" for the field:

  • Computational Efficiency: Training DRL agents for AOD is extremely resource-intensive and often slow to converge.
  • Local Optima: Agents often get stuck in "dead ends" where the next best view doesn't provide enough new information to trigger further movement.
  • Multi-Agent Collaboration: Most current research focuses on a single camera. Future work should explore how multiple robots or distributed sensor networks can collaborate to "surround" and identify objects faster.
  • Unified Frameworks: There is a lack of a "universal" evaluation metric that accounts for detection accuracy, path efficiency, and energy consumption across both 2D and 3D tasks.

Summary updated

Abstract

目标检测是计算机视觉领域的基础研究方向之一。由于图像采集时物体摆放密集、光照条件差等因素导致图像失去细节,当使用此类图像作为输入时,常规的目标检测算法对目标物的检测结果无法满足任务需求。为了解决这类问题,面向目标检测的视点规划这一智能感知方法应运而生,其可自主分析当前条件下影响检测任务的因素,调整相机的位姿参数规避影响,实现目标物准确检测。面向目标检测的视点规划方法不仅可以辅助计算机视觉的其他领域,也会为未来的智能化生活提供便利。为了反映其研究现状和最新进展,本文梳理了2007年以来的文献,对国内外的研究方法做出概括性总结。首先,以算法应用的场景维度和调整参数作为分类依据,将面向目标检测的视点规划方法分为二维像素调整的规划方法、三维空间移动的规划方法以及两者结合的规划方法3类,本文重点对前两类方法进行分析与总结。其次,解析每类方法的基本思想,并指出各类方法需解决的关键问题,然后对解决问题的主要研究方法进行归纳和分析,并总结各自的优点和局限性。除此之外,本文也对各类场景下可使用的数据集和评价指标进行简要介绍。最后,在目前方法的分析基础上,探讨面向目标检测的视点规划领域所面临的挑战,并对未来研究方法进行展望。
BibTeX
@article{wang2024view,
  author = {Jianyu Wang and Feng Zhu and Yingming Hao and Qun Wang and Pengfei Zhao and Haibo Sun},
  title = {{View planning methods of object detection: a survey}},
  journal = {Journal of Image and Graphics},
  year = {2024},
  doi = {10.11834/jig.240319},
}
Figure