Hierarchical Reinforcement Learning for Viewpoint Planning with Scalable Precision in UAV Inspection

Hua Wu, Hao Li, Junwei Yu, Yanxiong Wu, Xiaojing Bai, Mengyang Pu, Li Sun, Yihuan Li, Juncheng LiuDrones 2025 · DOI

TL;DR

This paper introduces a novel hierarchical reinforcement learning (HRL) approach for UAV viewpoint planning that significantly reduces the number of waypoints needed for inspection while simultaneously ensuring high observation precision. By decomposing the task into high-level waypoint selection and low-level pose/zoom optimization, their method achieves a 70% waypoint reduction and scalable precision up to 1.51 pixels/mm, boosting inspection efficiency and quality.

What Problem Does This Paper Address?

Inspecting complex structures with Unmanned Aerial Vehicles (UAVs) is crucial for maintenance and operational assessment. However, traditional methods often require a vast number of waypoints to ensure complete surface coverage, leading to inefficient flight times, increased energy consumption, and higher operational risks. Furthermore, achieving sufficient observation precision to detect small defects can be challenging with fixed camera settings. This paper tackles the dual challenge of optimizing inspection efficiency (minimizing waypoints) and ensuring high, scalable observation precision in UAV inspection tasks.

Methods

The core of this work is a Hierarchical Reinforcement Learning (HRL) framework called HRL-SPVP. This approach decomposes the complex viewpoint planning problem into two distinct levels:

  • High-Level Waypoint Planning: This strategy focuses on selecting the optimal sequence of waypoint positions to achieve full surface coverage of the inspection target using the minimum number of waypoints. It's modeled as a Semi-Markov Decision Process (SMDP) to handle potentially long-duration actions.
  • Low-Level Pose and Zoom Planning: Once a waypoint is selected, this strategy dynamically adjusts the UAV's camera pose (yaw and pitch) and focal length to capture multiple images (multi-shot per waypoint). This ensures high-precision observation of local details and compensates for varying inspection requirements. This level is modeled as a standard Markov Decision Process (MDP).

A key innovation is the unified reward function that balances coverage reward, a waypoint penalty (to encourage fewer waypoints), and an observation precision reward/penalty. This function encourages collaborative optimization between the high-level and low-level strategies. The method also incorporates a multi-shot per waypoint strategy, where multiple images are taken from a single location with varying camera settings, further reducing the need for numerous waypoints.

Datasets

The paper evaluates its method on several models:

  • Wind turbine models (three different postures: Inverted Y-shaped, Horizontal, Upright Y-shaped)
  • Publicly available models: SolarPlant, Big Ben, and Hoa Hakananai'a.

These models are used for both training and evaluating the reinforcement learning agents.

Key Insights and Ideas

  • Hierarchical Decomposition is Key: Breaking down the viewpoint planning problem into high-level waypoint selection and low-level pose/zoom adjustment effectively decouples complex interdependencies, making the problem more tractable and allowing for more sophisticated strategies.
  • Multi-Shot per Waypoint is a Game Changer: Instead of a single capture per waypoint, taking multiple shots with varying camera parameters from one location drastically reduces the total number of waypoints needed, directly impacting efficiency.
  • Unified Reward Function for Coordinated Optimization: Designing a reward that explicitly penalizes extra waypoints and rewards observation precision, alongside coverage, is crucial for achieving the desired balance between efficiency and quality.
  • Scalable Precision is Achievable: The method demonstrates that observation precision isn't a fixed parameter but can be dynamically controlled and scaled by adjusting the low-level strategy, allowing for tailored inspections.

Key Results

  • Significant Waypoint Reduction: The HRL-SPVP method reduces the number of waypoints by at least 70% compared to classic Coverage Path Planning (CCPP) and non-hierarchical reinforcement learning (DQN) methods across various inspection objects. For example, for the SolarPlant model, it reduced waypoints from 278 (CCPP) and 162 (DQN) to just 6.
  • High Observation Precision: The method achieves an average observation resolution of 1.51 pixels/mm, demonstrating its capability to capture fine details.
  • Improved Inspection Efficiency: The reduction in waypoints translates to a significant reduction in flight time. For a typical UAV, this could mean saving up to 37 minutes per charge, enabling approximately three times more inspection cycles per flight.
  • Qualitative Improvements: Visualizations show more optimized and less redundant flight paths.

Limitations and Future Work

  • Ideal Conditions Assumption: The current method assumes ideal environmental conditions and does not explicitly account for dynamic environments or sensors with a high number of degrees of freedom.
  • Future Work: The authors propose extending the HRL framework to integrate robust optimization techniques to handle uncertainties inherent in real-world UAV inspection scenarios, such as dynamic environments or sensor limitations.

Summary updated

Abstract

Viewpoint planning is crucial to ensure both inspection efficiency and observation precision in UAV inspection tasks. To address the issues of excessive waypoints and inadequate observation precision in traditional methods, this paper proposes a hierarchical reinforcement learning-based viewpoint planning method. The proposed method decomposes the viewpoint planning task into a high-level waypoint planning strategy and a low-level pose and zoom planning strategy. Additionally, a reward function is designed to enhance inspection precision, enabling collaborative optimization of waypoint positions, viewpoint poses, and focal lengths. Experimental results show that, compared with the classic coverage path planning method and non-hierarchical reinforcement learning approaches, the proposed method reduces the number of waypoints by at least 70% across multiple inspection objects. Furthermore, experiments with viewpoint planning at different precision levels demonstrate that the proposed method achieves scalable precision during inspection, with the observation resolution improving to 1.51 pixels/mm. Finally, a qualitative comparison is made between the proposed method in this paper and other representative methods in viewpoint planning. These results effectively demonstrate the validity and superiority of the proposed method in improving both inspection task efficiency and observation precision.
BibTeX
@inproceedings{wu2025hierarchical,
  author = {Hua Wu and Hao Li and Junwei Yu and Yanxiong Wu and Xiaojing Bai and Mengyang Pu and Li Sun and Yihuan Li and Juncheng Liu},
  title = {{Hierarchical Reinforcement Learning for Viewpoint Planning with Scalable Precision in UAV Inspection}},
  booktitle = {Drones},
  year = {2025},
  doi = {10.3390/drones9050352},
}
Figure