Automatic geolocation of targets tracked by aerial imaging platforms using satellite imagery

Pradyumn Kumar Shukla, Shubhra Goel, Pravendra Singh, Bharat Lohani2014 · DOI

TL;DR

This paper presents a novel method for automatically geolocating targets tracked by aerial platforms, particularly useful in GPS-denied environments. By registering aerial imagery with georeferenced satellite imagery using the SURF algorithm, the system can determine target coordinates without relying on expensive onboard Position and Orientation Systems (POS). This approach offers a cost-effective and robust solution for surveillance and disaster management applications.

What Problem Does This Paper Address?

Accurately determining the geographical location (geolocation) of targets tracked by aerial platforms like Unmanned Aerial Vehicles (UAVs) is crucial for many applications, including surveillance and search and rescue. Traditional methods often rely on onboard Position and Orientation Systems (POS) which include GPS. However, these systems can be expensive, power-hungry, and are susceptible to signal jamming or denial in certain environments. This limitation restricts the widespread adoption of UAVs for precise target geolocation in such scenarios. The paper aims to bridge this gap by proposing a method that leverages image registration between aerial and satellite imagery, thereby reducing reliance on onboard POS and enabling geolocation even in partially GPS-denied conditions.

Methods

The core of the proposed methodology lies in automatic feature-based image registration. The process begins with a georeferenced satellite image (e.g., from Google Earth) stored in the UAV's database. When an aerial image of a target is captured, it's first enhanced to be compatible with the satellite image. The Speeded Up Robust Features (SURF) algorithm is then employed for registration. SURF is chosen for its computational efficiency and robustness to scale and rotation variations. It identifies key interest points (blobs) and computes descriptor vectors for both images. These features are then matched to establish a transformation. Once the aerial image is registered to the satellite image, the known coordinates of the target within the aerial image can be translated to the satellite image's coordinate system, thus providing its geolocation.

A key innovation is the handling of subsequent frames. Instead of registering every new aerial frame to the satellite image, the system registers subsequent aerial frames to the initially registered aerial frame. This significantly speeds up the process, as the computationally intensive feature extraction for the reference image is done only once. The RANSAC algorithm is used to filter out outliers during matching, ensuring robust registration.

Datasets

The paper does not use traditional large-scale datasets for training. Instead, it utilizes a simulated experimental setup based on a scaled model of the Rajeev Motwani Building at IIT Kanpur.

  • Reference Image: A high-resolution satellite image of the IIT Kanpur area was obtained from Google Earth.
  • Aerial Images: Simulated aerial images were captured using two cameras:
  • A Nikon D5100 SLR camera (simulating a higher-altitude, nadir-viewing satellite image) from 20m altitude.
  • An Apple i4s mobile phone camera (simulating a lower-altitude, oblique-viewing aerial image) from 0.5m to 1m altitude.
  • Target: A toy car was used as the moving target on the model.

The use of a scaled model allowed for controlled experimentation and simulation of different sensor characteristics and viewing conditions.

Key Insights and Ideas

  • Decoupling Geolocation from Onboard POS: The most significant insight is the demonstration that accurate geolocation can be achieved by registering aerial imagery with existing georeferenced satellite data, thereby reducing or eliminating the need for expensive and potentially unreliable onboard POS.
  • Efficiency through Sequential Registration: Registering subsequent aerial frames to an already registered frame (rather than to the satellite image each time) dramatically improves processing speed, making real-time tracking more feasible.
  • SURF's Suitability: The SURF algorithm proves to be an effective choice for this task due to its speed and ability to handle variations in image scale, rotation, and illumination, which are common when comparing satellite and aerial imagery.
  • Value of Oblique Imagery: While nadir views provide a broad overview, oblique aerial imagery, even from low altitudes, offers valuable detail about feature heights and textures, which can aid in more precise feature matching.

Key Results

The experimental validation demonstrated the feasibility and accuracy of the proposed method.

  • Registration Accuracy: The method achieved satisfactory image registration between simulated satellite and aerial images.
  • Geolocation Accuracy: The validation showed a shift of only 4 to 6 pixels in the target's position when comparing the proposed method's results to the original position captured by the Nikon D5100 camera. The Root Mean Square Error (RMSE) for X and Y axes was reported to be around 4.5-6.4 pixels.
  • Speed: The sequential registration of subsequent frames was found to be significantly faster than registering each frame individually to the satellite image, making it suitable for real-time applications.
  • Qualitative Results: Figures in the paper visually demonstrate successful image registration and target geolocation, showing overlaid images and identified target locations.

Limitations and Future Work

  • Limitations:
    • Scale of Experiment: The experiments were conducted on a scaled model, which, while useful for controlled testing, may not fully replicate the complexities of real-world scenarios with varying terrain and atmospheric conditions.
  • Accuracy in GNSS-Denied Environments: While the method is designed for GNSS-denied environments, the paper acknowledges that the accuracy is dependent on the quality of the initial satellite image and the effectiveness of the SURF algorithm in matching features. A "slight shift" in geolocation due to obliqueness was noted.
  • Reliance on Initial Registration: The accuracy of subsequent frame registration relies heavily on the quality of the initial registration between the aerial and satellite images.
  • Future Work:
    • Real-World Testing: The authors suggest further validation with real-world UAV data and in actual GNSS-denied environments.
  • Integration with Advanced Tracking: Integrating the proposed geolocation module with more sophisticated real-time tracking algorithms could enhance its practical utility.
  • Improving Accuracy: Further research could focus on improving the accuracy to sub-meter levels, which would be critical for certain applications.
  • Handling Extreme Conditions: Investigating the method's performance under more challenging conditions, such as significant illumination changes, occlusions, or highly dynamic environments, would be beneficial.

Summary updated

Abstract

Tracking of targets from aerial platforms is an important activity in several applications, especially surveillance. Knowled ge of geolocation of these targets adds additional significant and useful information to the application. This paper determines the geolocation of a target being tracked from an aerial platform using the technique of image registration. Current approaches utilize a POS to determine the location of the aerial platform and then use the same for geolocation of the targets using the principle of photogrammetry. The constraints of cost and low-payload restrict the applicability of this approach using UAV platforms. This paper proposes a methodology for determining the geolocation of a target tracked from an aerial platform in a partially GPS devoid environment. The method utilises automatic feature based registration technique of a georeferenced satellite image with an ae rial image which is already stored in UAV's database to retrieve the geolocation of the target. Since it is easier to register subsequent aerial images due to similar viewing parameters, the subsequent overlapping images are registered together sequentially thus resulting in the registration of each of the images with georeferenced satellite image thus leading to geolocation of the target under interest. Using the proposed approach, the target can be tracked in all the frames in which it is visible. The proposed concept is verified experimentally and the results are found satisfactory. Using the proposed method, a user can obtain location of target of interest as well features on ground without requiring any POS on-board the aerial platform. The proposed approach has applications in surveillance for target tracking, target geolocation as well as in disaster management projects like search and rescue operations.
BibTeX
@inproceedings{shukla2014automatic,
  author = {Pradyumn Kumar Shukla and Shubhra Goel and Pravendra Singh and Bharat Lohani},
  title = {{Automatic geolocation of targets tracked by aerial imaging platforms using satellite imagery}},
  booktitle = {˜The},
  year = {2014},
  doi = {10.5194/isprsarchives-xl-1-381-2014},
}
Figure