Internet of Things Applications: Animal Monitoring with Unmanned Aerial Vehicle

Jun Xu, Gürkan Solmaz, Rouhollah Rahmatizadeh, Damla Turgut, Ladislau BölöniarXiv 2016 · OpenReview

TL;DR

This paper introduces a novel approach for animal monitoring in large wildlife areas using a combination of wireless sensor networks (WSNs) and an Unmanned Aerial Vehicle (UAV). By employing a Markov Decision Process (MDP) and Q-learning, the UAV intelligently plans its flight path to maximize the value of collected animal data, outperforming traditional methods.

What Problem Does This Paper Address?

Monitoring endangered species in vast, remote wildlife areas presents significant challenges. Traditional methods often involve attaching tracking devices to animals, which is difficult and sometimes infeasible in real-world scenarios. Furthermore, relying solely on static sensor networks for data collection can be inefficient due to communication limitations and the time-sensitive nature of animal movement data. The authors aim to address the problem of efficiently detecting animal locations and predicting their movements without direct animal tagging, while also optimizing data collection for a mobile platform like a UAV.

Methods

The core of the proposed solution is an MDP-based path planning approach for the UAV, solved using Q-learning. Here's a breakdown:

  • Network Model: The observation area is divided into virtual grids. Each grid acts as a cluster of sensor nodes, with a designated cluster head responsible for collecting data. A single UAV serves as a mobile sink to collect information from these cluster heads.
  • Value of Information (VoI): The paper quantifies the importance of sensed data using the concept of Value of Information (VoI), where data is more valuable when it's fresh and its value decays over time. This is modeled using an exponential decay function.
  • Rewards: Rewards are assigned to the UAV for visiting grids where animals are detected. These rewards are influenced by factors like sensor credibility, distance to the animal, and event duration. A penalty is applied for visiting grids with no animal activity.
  • Markov Decision Process (MDP): Each grid is treated as a state in the MDP. The UAV's actions are to move to adjacent grids or stay in its current location. The goal is to learn an optimal policy that maximizes the cumulative VoI.
  • Q-Learning: This reinforcement learning algorithm is used to solve the MDP. The UAV learns the expected future rewards (Q-values) for taking specific actions in each state, allowing it to dynamically plan its path. The "e-greedy" strategy is employed to balance exploration (discovering new areas) and exploitation (visiting known "hot spots").

Datasets

The proposed approach was evaluated using two real-world animal movement datasets:

  • ZebraNet dataset: This dataset contains GPS traces of 5 zebras in Kenya, with a sampling time of 10 minutes over 14 days. It's characterized by animals roaming over a broad area and moving frequently.
  • Leopard tortoises dataset: This dataset includes movement traces of 10 leopard tortoises in the Kalahari desert, recorded over three months with a 60-minute sampling time. These animals exhibit more stationary and predictable movement patterns within smaller areas.

The authors converted these datasets into a 10km x 10km grid-based area for their simulations.

Key Insights and Ideas

  • Animal Movement Patterns are Key: The paper leverages the observation that animals often exhibit predictable movement patterns, tending to stay in specific "hot spots" or habitats. This insight is crucial for designing an efficient path planning strategy.
  • VoI as a Guiding Metric: Quantifying the value of information based on its freshness is a powerful way to guide the UAV's data collection efforts, ensuring that the most relevant data is prioritized.
  • Reinforcement Learning for Dynamic Environments: Using MDP and Q-learning allows the UAV to learn and adapt its path planning strategy in real-time based on the observed animal activity, making it more robust than static or purely greedy approaches.
  • Exploration vs. Exploitation is Crucial: The e-greedy strategy is vital for discovering new animal locations and adapting to changing animal behaviors, preventing the UAV from getting stuck in local optima.

Key Results

The proposed MDP-based path planning approach demonstrated superior performance compared to greedy, TSP-based, and random approaches across both datasets.

  • Higher Value of Information (VoI): The MDP approach consistently achieved higher VoI over time, indicating more efficient and timely data collection. For instance, it showed an 80% increase in VoI compared to the greedy approach.
  • Reduced Message Delay: The MDP approach resulted in significantly lower median message delays, meaning animal data was reported and processed much faster. This is crucial for real-time monitoring.
  • Increased Animal Encounters: The UAV using the MDP approach encountered animals more frequently, suggesting a more effective strategy for locating them. This was a 90% increase in encounters compared to the greedy approach.
  • Stable Performance: The MDP approach showed consistent and stable performance across different experimental runs, highlighting its reliability.

Limitations and Future Work

  • Single UAV Assumption: The current model assumes a single UAV. The authors acknowledge that in larger or more complex scenarios, multiple UAVs might be necessary, which would introduce additional coordination challenges.
  • Parameter Tuning: The performance of the MDP approach is influenced by parameters like the exploration rate ($\epsilon$). While the paper empirically found $\epsilon=0.2$ to be optimal, further research could explore more adaptive methods for parameter selection.
  • Grid Size Optimization: The paper suggests that an optimal grid size (between 4x4 and 5x5) exists for balancing exploration and prediction difficulty. Future work could investigate more dynamic grid partitioning strategies.
  • Generalizability: While tested on zebras and tortoises, extending the model to other animal species with different movement behaviors and environmental conditions would be valuable.
  • Sensor Network Complexity: The paper assumes a simplified sensor network model. Future work could incorporate more realistic sensor network dynamics, such as sensor failures or varying communication ranges.

Summary updated

Abstract

In animal monitoring applications, both animal detection and their movement prediction are major tasks. While a variety of animal monitoring strategies exist, most of them rely on mounting devices. However, in real world, it is difficult to find these animals and install mounting devices. In this paper, we propose an animal monitoring application by utilizing wireless sensor networks (WSNs) and unmanned aerial vehicle (UAV). The objective of the application is to detect locations of endangered species in large-scale wildlife areas and monitor movement of animals without any attached devices. In this application, sensors deployed throughout the observation area are responsible for gathering animal information. The UAV flies above the observation area and collects the information from sensors. To achieve the information efficiently, we propose a path planning approach for the UAV based on a Markov decision process (MDP) model. The UAV receives a certain amount of reward from an area if some animals are detected at that location. We solve the MDP using Q-learning such that the UAV prefers going to those areas that animals are detected before. Meanwhile, the UAV explores other areas as well to cover the entire network and detects changes in the animal positions. We first define the mathematical model underlying the animal monitoring problem in terms of the value of information (VoI) and rewards. We propose a network model including clusters of sensor nodes and a single UAV that acts as a mobile sink and visits the clusters. Then, one MDP-based path planning approach is designed to maximize the VoI while reducing message delays. The effectiveness of the proposed approach is evaluated using two real-world movement datasets of zebras and leopard. Simulation results show that our approach outperforms greedy, random heuristics and the path planning based on the traveling salesman problem.
BibTeX
@misc{xu2016internet,
  author = {Jun Xu and Gürkan Solmaz and Rouhollah Rahmatizadeh and Damla Turgut and Ladislau Bölöni},
  title = {{Internet of Things Applications: Animal Monitoring with Unmanned Aerial Vehicle}},
  year = {2016},
  eprint = {1610.05287},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1610.05287},
}
Figure