Online, self-supervised terrain classification via discriminatively trained submodular Markov random fields

Paul Vernaza, Ben Taskar, Daniel D. LeeICRA 2008 · OpenReview

TL;DR

This paper introduces a novel approach to autonomous terrain classification for mobile robots using discriminatively trained submodular Markov Random Fields (MRFs). By leveraging stereo vision for self-supervision and employing efficient graph-cut inference and averaged subgradient learning, the method achieves high accuracy in real-time, outperforming traditional independent pixel classifiers.

What Problem Does This Paper Address?

Mobile robots navigating in unknown environments often rely on sensors like stereo cameras to classify terrain as either "obstacle" or "ground" for safe navigation. While stereo provides depth information, it's limited in range. Monocular cameras offer richer visual features at longer distances but lack depth. The challenge is to train a robust monocular terrain classifier that can learn from readily available stereo data (self-supervision) and operate in real-time on the robot. Previous methods often assumed pixel independence, limiting their accuracy. This paper aims to overcome this by modeling spatial correlations between pixels using MRFs and developing an efficient learning framework suitable for online robot operation.

Methods

The core of the paper is the application of submodular Markov Random Fields (MRFs) for terrain classification. Unlike traditional approaches that treat pixels independently, MRFs allow modeling of correlations between neighboring pixels. The authors specifically use submodular MRFs with binary potentials and binary labels, which are amenable to efficient exact inference via graph cuts (specifically, min-cut/max-flow algorithms).

For learning, they employ a discriminative max-margin criterion, inspired by support vector machines and "maximum margin planning." This involves learning a weight vector that maximizes the margin between the desired classification and all other possible classifications. The learning process uses an averaged subgradient method, which is robust to the non-differentiable nature of the objective function arising from the min-cut computation. This method allows for online updates, making it suitable for real-time adaptation on a robot.

Self-supervision is achieved by using dense stereo data to generate ground truth labels for training the monocular classifier. A robust least-squares approach is used to estimate the ground plane from stereo disparity, and pixels deviating from this plane are labeled as obstacles.

Datasets

The experiments were conducted using data collected from a DARPA Learning Applied to Ground Robots (LAGR) standard platform. This involved collecting sequential images from a Bumblebee stereo camera. For evaluation, two datasets were used: an "easy" dataset with good contrast and an "hard" dataset with poorer contrast and exposure. The paper does not introduce new public benchmarks but uses real-world robot log data.

Key Insights and Ideas

  • MRFs for Spatial Correlation: The key insight is that modeling spatial correlations between pixels using MRFs significantly improves terrain classification accuracy over independent pixel classifiers. This is crucial for tasks where local context matters.
  • Graph Cuts for Efficient Inference: The use of submodular MRFs allows for efficient and exact inference using graph cuts, a well-established and computationally efficient technique in computer vision.
  • Discriminative Max-Margin Learning: Learning a discriminative model with a max-margin objective leads to a more robust classifier that generalizes better, especially in challenging conditions.
  • Averaged Subgradient for Online Learning: The averaged subgradient method provides a practical way to train the MRF online, allowing the robot to adapt to new environments and obstacles in real-time.
  • Stereo for Self-Supervision: Leveraging readily available stereo data for ground truth generation is a practical and scalable way to obtain large amounts of training data for monocular classification.

Key Results

  • The MRF classifier significantly outperformed a histogram-based independent classifier, achieving higher accuracy on both training and test sets.
  • On the "hard" dataset, the MRF showed an 11.5% increase in classification accuracy on the held-out set compared to the independent classifier.
  • Qualitative results showed that the MRF classifier produced more uniform and consistent classifications across large image patches, unlike the often inconsistent histogram classifications.
  • In the online training scenario, the mean classification accuracy per image reached 93.6% by the 11th epoch.
  • The computational efficiency was demonstrated with unoptimized MATLAB code achieving real-time performance. A full subgradient calculation step (including MAP inference) took less than 40 milliseconds per 160x120 image.

Limitations and Future Work

  • Feature Selection: The authors used simple RGB color features for simplicity. They acknowledge that better features would likely yield even better performance.
  • Neighborhood Size: The current implementation uses small pixel neighborhoods. Future work could explore increasing neighborhood size, potentially using pyramidal graph structures, to capture larger-scale context.
  • Computational Efficiency: While real-time performance was achieved, further improvements are possible. The authors suggest warm-starting the max-flow computation for faster inference and learning, especially as problem data changes slowly.
  • Online Learning Regularization: The authors found that using a modified regularizer (||w - wi||²) for online learning, which encourages conservative changes to the weight vector, worked better than a simple L2 norm. This suggests that careful regularization is important for online adaptation.

Summary updated

BibTeX
@inproceedings{vernaza2008online,
  author = {Paul Vernaza and Ben Taskar and Daniel D. Lee},
  title = {{Online, self-supervised terrain classification via discriminatively trained submodular Markov random fields}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2008},
  url = {https://openreview.net/forum?id=mNouLa48fOO},
}
Figure