DeepLabV3+/Efficientnet Hybrid Network-Based Scene Area Judgment for the Mars Unmanned Vehicle System

Shuang Hu, Jin Liu, Zhiwei KangSensors 2021 · DOI

TL;DR

This paper introduces a novel hybrid DeepLabV3+/Efficientnet neural network for robust scene area judgment on Mars, crucial for the safety and efficiency of unmanned exploration vehicles. By combining DeepLabV3+'s powerful feature extraction with Efficientnet's classification capabilities, the system achieves a remarkable 99.84% accuracy, significantly outperforming standalone Efficientnet and ensuring safer Mars missions.

What Problem Does This Paper Address?

Mars exploration missions, while vital for understanding our solar system, are fraught with peril due to the planet's harsh and unpredictable environment. Traditional Mars unmanned ground vehicles (UGVs) struggle to navigate these complexities, leading to potential mission failures. A critical challenge is the ability of these vehicles to accurately and efficiently assess their surroundings to identify safe paths, potential hazards, and areas requiring further investigation. This "scene area judgment" is paramount for autonomous navigation and decision-making. Existing intelligent technologies, particularly deep learning, offer promise, but achieving high accuracy in the unique Martian context, with its specific terrain features like rocks and quicksand, remains an open problem.

Methods

The core of this paper is the development of a DeepLabV3+/Efficientnet hybrid network. The approach is a two-stage process:

  1. Feature Extraction (DeepLabV3+): The DeepLabV3+ network is employed first to extract rich feature information from Mars images. DeepLabV3+ is chosen for its high accuracy in semantic segmentation and its ability to capture detailed feature information using techniques like atrous convolution (also known as hole convolution) and an encoder-decoder structure. This allows it to enlarge the receptive field without losing resolution, crucial for identifying subtle terrain features.
  2. Scene Area Judgment (Efficientnet): The extracted features from DeepLabV3+ are then fed into an Efficientnet network. Efficientnet, known for its efficient scaling of network depth, width, and resolution, is used here for classification. It categorizes the scene into three types: "safe area," "report area," and "dangerous area."

Based on these classifications, the Mars unmanned vehicle system takes specific actions:

  • Safe Area: The UGV continues exploration.
  • Report Area: The system saves the results, and the UGV decelerates.
  • Dangerous Area: The system launches an unmanned aerial vehicle (UAV) for further exploration.

This hybrid approach aims to leverage the strengths of both networks – DeepLabV3+ for detailed feature extraction and Efficientnet for efficient and accurate classification.

Datasets

The primary dataset used in this research is the Mars32K dataset.

  • This dataset was obtained by NASA's Curiosity rover on the surface of Mars.
  • It contains images with main feature information including rocks and quicksand.
  • The dataset was manually annotated to prevent issues caused by training process variations.
  • It was divided into 480 training sets and 130 validation sets.
  • To address the issue of a small sample size and potential underfitting, data augmentation was performed, employing eight methods including flip transform, random crop, color jittering, translation shift, scale, contrast, noise disturbance, and rotation/reflection. This resulted in an augmented dataset of 4800 training samples and 1300 validation samples, totaling 6100.

Key Insights and Ideas

  • Synergy of Feature Extraction and Classification: The paper demonstrates that combining a powerful semantic segmentation network (DeepLabV3+) for feature extraction with an efficient classification network (Efficientnet) leads to superior performance compared to using either network in isolation for the complex task of Mars scene area judgment.
  • Importance of Detailed Feature Extraction for Mars: The unique and challenging Martian terrain necessitates sophisticated feature extraction. DeepLabV3+'s ability to capture fine-grained details, even in low-level features, is shown to be critical for accurate classification of safe, report, and dangerous areas.
  • Data Augmentation is Crucial for Limited Datasets: The research highlights the effectiveness of data augmentation techniques in overcoming the limitations of smaller datasets, enabling models to generalize better and achieve higher accuracy in specialized domains like Mars exploration.
  • Hierarchical Decision-Making for Safety: The proposed three-tiered classification (safe, report, dangerous) and corresponding actions (pass, report/decelerate, send UAV) provide a robust and layered safety mechanism for Mars exploration vehicles.

Key Results

  • Overall Accuracy: The DeepLabV3+/Efficientnet hybrid network achieved an impressive 99.84% accuracy on the test set for scene area judgment.
  • Comparison with Efficientnet: The hybrid network showed a significant improvement of approximately 18% in accuracy compared to using the Efficientnet network alone.
  • Feature Extraction Performance:
    • The Drn base network showed the best performance for feature extraction, achieving 97.3% accuracy on the test set.
  • For feature extraction of specific types (rocks and quicksand), Drn also performed best with 93.1% accuracy.
  • MIOU (Mean Intersection Over Union) was 87.5%, and FWIOU (Frequency Weighted Intersection Over Union) was 93.4%, indicating high efficiency in feature extraction.
  • Hybrid Network Performance:
    • The hybrid network achieved 99.8% accuracy on the training set and 97.1% accuracy on the test set.
  • The confusion matrix revealed a low error rate, with only 4% of dangerous areas misclassified as report areas and 2% of report areas misclassified as safe areas.
  • Efficientnet Alone: When used solely for scene area judgment, the Efficientnet network achieved 81% accuracy on the test set with a loss of 0.51.

Limitations and Future Work

  • Dataset Size: While data augmentation was employed, the original Mars32K dataset had a relatively small number of samples (610 total), which can still pose challenges for model generalization.
  • Computational Resources: The paper does not explicitly detail the computational resources required for training or inference, which can be a significant factor for onboard systems.
  • Real-world Deployment: The experiments were conducted on a dataset. Further validation and testing in simulated or actual Mars-like environments would be necessary for real-world deployment.

The paper suggests that the DeepLabV3+/Efficientnet hybrid network effectively addresses the problem of insufficient feature extraction capabilities of Efficientnet alone and significantly improves accuracy. Future work could focus on optimizing the model for real-time performance on resource-constrained Mars vehicles and exploring more diverse Martian terrain scenarios.

Summary updated

Abstract

Due to the complexity and danger of Mars's environment, traditional Mars unmanned ground vehicles cannot efficiently perform Mars exploration missions. To solve this problem, the DeepLabV3+/Efficientnet hybrid network is proposed and applied to the scene area judgment for the Mars unmanned vehicle system. Firstly, DeepLabV3+ is used to extract the feature information of the Mars image due to its high accuracy. Then, the feature information is used as the input for Efficientnet, and the categories of scene areas are obtained, including safe area, report area, and dangerous area. Finally, according to three categories, the Mars unmanned vehicle system performs three operations: pass, report, and send. Experimental results show the effectiveness of the DeepLabV3+/Efficientnet hybrid network in the scene area judgment. Compared with the Efficientnet network, the accuracy of the DeepLabV3+/Efficientnet hybrid network is improved by approximately 18% and reaches 99.84%, which ensures the safety of the exploration mission for the Mars unmanned vehicle system.
BibTeX
@inproceedings{hu2021deeplabv,
  author = {Shuang Hu and Jin Liu and Zhiwei Kang},
  title = {{DeepLabV3+/Efficientnet Hybrid Network-Based Scene Area Judgment for the Mars Unmanned Vehicle System}},
  booktitle = {Sensors},
  year = {2021},
  doi = {10.3390/s21238136},
}
Figure