SpCoSLAM 2.0: An Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mapping

Akira Taniguchi, Yoshinobu Hagiwara, Tadahiro Taniguchi, Tetsunari Inamura

TL;DR

This paper introduces SpCoSLAM 2.0, an improved and scalable online learning algorithm for robots to simultaneously learn spatial concepts, language, and maps. It addresses limitations of previous work by enhancing accuracy and reducing computational complexity, enabling more robust long-term human-robot interaction.

What Problem Does This Paper Address?

Robots operating in human environments need to adaptively learn about their surroundings, including recognizing places, understanding associated language, and building maps. Manually designing this knowledge is impractical. Previous online learning methods for this task, like the authors' own SpCOSLAM, suffered from limited accuracy, especially in early learning stages, and increasing computational cost as more data was acquired. This made long-term, real-world deployment challenging. The paper aims to overcome these limitations by developing an algorithm that is both more accurate and computationally scalable for continuous learning.

Methods

The core contribution is SpCoSLAM 2.0, an evolution of their prior SpCOSLAM algorithm. It builds upon a Bayesian probabilistic model integrating multimodal place categorization, lexical acquisition, and Simultaneous Localization and Mapping (SLAM).

Key improvements include:

  • Fixed-Lag Rejuvenation (FLR): This technique, inspired by fixed-lag smoothers, allows the algorithm to revisit and correct past estimations of latent variables (like place identities and word associations) for a limited "lag" period. This helps mitigate the impact of early, potentially inaccurate, learning on later stages.
  • Re-segmentation of Word Sequences (RS): To improve lexical acquisition, the algorithm re-segments word sequences using a more robust approach, addressing issues like under-segmentation that can arise from speech recognition errors.
  • Sequential Bayesian Update (SBU): For scalability, the algorithm adopts a sequential update for posterior hyperparameters, similar to how SLAM algorithms update maps. This avoids recomputing everything from scratch with each new piece of data.
  • Pseudo-Online Learning with FLR for Word Sequences: The algorithm extends FLR to word sequence sampling, allowing it to learn from a fixed window of recent data rather than all past data, further improving scalability.

Datasets

  • Real Environment Dataset: An open dataset named "albert-b-laser-vision" was used, containing odometry, laser range data, and image data. Japanese speech data was prepared to correspond to the robot's movements in this dataset.
  • Simulator Environments (SIGVerse): The experiments were also conducted in various virtual home environments created using SIGVerse and Sweet Home 3D. This allowed for controlled testing across different settings.

Key Insights and Ideas

  • The value of "looking back": The concept of fixed-lag rejuvenation is crucial. It highlights that in online learning, allowing the system to correct past mistakes, even with a limited window, significantly improves overall accuracy and robustness, especially in dynamic environments.
  • Scalability doesn't have to mean sacrificing accuracy: By decoupling the learning of spatial concepts and language from the continuous SLAM process and using techniques like SBU and FLR for word sequences, the algorithm achieves constant computation time per step, independent of the total data collected. This is a significant step towards truly long-term robot learning.
  • Addressing the "chicken and egg" problem of word segmentation: The re-segmentation approach helps break the cycle where early, poor word segmentation can propagate errors, leading to better lexical acquisition over time.

Key Results

  • Improved Accuracy: SpCoSLAM 2.0 (especially methods (C), (D), and (E)) demonstrated higher accuracy in place categorization (ARI) and lexical acquisition (PAR) compared to the original SpCOSLAM (A) and (B).
  • Comparable to Batch Learning: The improved and scalable algorithms achieved accuracy comparable to batch learning methods like SpCoA++ (F).
  • Scalability Achieved: The scalable algorithm (E1-E3) maintained a constant calculation time per step, regardless of the amount of training data, a significant improvement over the original algorithms whose computation time increased with data.
  • Real-world and Simulation Validation: The proposed methods showed consistent improvements in both real-world experiments and simulated environments.

Limitations and Future Work

  • Forgetting Mechanism: The authors acknowledge that for long-term operation, a forgetting mechanism would be beneficial to adapt to environmental changes (e.g., new place names or altered layouts).
  • Topological Structure Learning: While the current method constructs spatial concepts on a metric map, extending it to learn topological structures is a future direction.
  • Integration with Other Models: The authors propose integrating SpCOSLAM with other probabilistic models, such as those in the SERKET architecture, for more comprehensive cognitive modeling.
  • Parallel Processing: While the algorithm is designed for scalability, further speedups could be achieved through parallel processing, especially using GPUs.
  • Trade-off in Scalable Algorithm: The scalable algorithm with FLR of St involves a trade-off between computation time and accuracy, requiring careful parameter tuning based on the robot's capabilities and operational needs.

Summary updated

Abstract

We propose a novel online learning algorithm, called SpCoSLAM 2.0, for spatial concepts and lexical acquisition with high accuracy and scalability. Previously, we proposed SpCoSLAM as an online learning algorithm based on unsupervised Bayesian probabilistic model that integrates multimodal place categorization, lexical acquisition, and SLAM. However, our original algorithm had limited estimation accuracy owing to the influence of the early stages of learning, and increased computational complexity with added training data. Therefore, we introduce techniques such as fixed-lag rejuvenation to reduce the calculation time while maintaining an accuracy higher than that of the original algorithm. The results show that, in terms of estimation accuracy, the proposed algorithm exceeds the original algorithm and is comparable to batch learning. In addition, the calculation time of the proposed algorithm does not depend on the amount of training data and becomes constant for each step of the scalable algorithm. Our approach will contribute to the realization of long-term spatial language interactions between humans and robots.
BibTeX
@article{taniguchi2020spcoslam,
  author = {Akira Taniguchi and Yoshinobu Hagiwara and Tadahiro Taniguchi and Tetsunari Inamura},
  title = {{SpCoSLAM 2.0: An Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mapping}},
  journal = {Auton. Robots},
  year = {2020},
  doi = {10.1007/s10514-020-09905-0},
}
Figure