Symbol Emergence in Cognitive Developmental Systems: A Survey.

Tadahiro Taniguchi, Justus H. Piater, Florentin Wörgötter, Emre Ugur, Matej Hoffmann, Lorenzo Jamone, Takayuki Nagai, Benjamin Rosman, Toshihiko Matsuka, Naoto Iwahashi, Erhan Öztop

TL;DR

This survey argues that the traditional "symbol grounding problem" in AI is too narrow, focusing on connecting symbolic AI to sensorimotor data. Instead, it proposes "symbol emergence" as a more encompassing concept, emphasizing how symbols arise and evolve dynamically within social and cognitive systems, much like human language. The paper reviews this problem across biological and artificial systems, offering an integrative viewpoint and outlining key challenges for creating truly cognitive systems.

What Problem Does This Paper Address?

The paper tackles the challenge of creating artificial intelligence (AI) and cognitive systems that can learn and use symbols in a way that mirrors human development and social interaction. Traditionally, AI has focused on the "symbol grounding problem," which aims to connect abstract symbols (like those in logic) to real-world sensory data. However, the authors argue this approach is limited because it often assumes pre-existing symbolic systems and doesn't account for how symbols are dynamically created, evolve, and are shared within a community. They propose shifting the focus to "symbol emergence," which considers the developmental and social aspects of how symbols come into being and gain meaning, drawing inspiration from semiotics and developmental psychology.

Methods

This paper is a survey, so it doesn't present novel algorithms or experiments. Instead, its "method" is a comprehensive review and synthesis of existing research across various fields:

  • Semiotics: It introduces the Peircean and Saussurian definitions of symbols, emphasizing the triadic relationship (sign, object, interpretant) and the dynamic nature of meaning.
  • Evolutionary Biology: It traces the development of actions and action sequences as precursors to symbolic thought.
  • Neuroscience: It discusses mirror neurons and brain imaging studies as potential neural underpinnings for concept formation.
  • Cognitive Science: It differentiates between "concepts" and "categories," highlighting the role of perceptual symbol systems (PSS) over amodal symbol systems (ASS).
  • Developmental Psychology: It reviews theories on how infants develop symbolic understanding through interaction and play.
  • Artificial Intelligence and Robotics: It surveys approaches in pattern recognition, unsupervised learning, and reinforcement learning that attempt to address symbol grounding and emergence.

The paper advocates for a Perceptual Symbol System (PSS) perspective, which posits that internal representations are grounded in multimodal sensory experiences, rather than abstract, amodal tokens.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Symbol Grounding vs. Symbol Emergence: The core argument is that "symbol emergence" is a more fruitful framework than "symbol grounding" for developing cognitive systems. Symbol emergence acknowledges the dynamic, social, and developmental nature of symbols, moving beyond the static, token-based view of traditional AI.
  • Semiotics as a Foundation: Understanding symbols through the lens of semiotics (Peirce, Saussure) provides a richer, more dynamic definition that includes the "interpretant" – the process of relating a sign to an object – which is crucial for meaning.
  • Bottom-Up Development is Key: Cognitive systems, like human infants, should learn symbols through interaction with their environment and social partners, rather than having pre-programmed symbolic systems. This bottom-up approach is essential for developing truly adaptive and intelligent agents.
  • PSS Over ASS: Perceptual Symbol Systems (PSS), which ground symbols in sensory experiences, are better suited for cognitive developmental systems and robots operating in the real world than Amodal Symbol Systems (ASS), which rely on abstract, context-independent tokens.
  • The Importance of Social Interaction: Symbol negotiation and sharing within a community are critical for establishing shared meaning and enabling communication. This highlights the need for robots to engage in social interaction to learn and use symbols effectively.
  • Concepts as Dynamic Structures: Concepts are not static entities but dynamic, multimodal representations that are constantly updated through experience and interaction.

Key Results

As this is a survey paper, it doesn't present experimental results. Instead, its "results" are the synthesis of existing research and the proposed integrative viewpoint. The key takeaway is the conceptual shift from "symbol grounding" to "symbol emergence" as a more comprehensive framework for AI and cognitive systems.

Limitations and Future Work

The paper acknowledges several challenges and areas for future work:

  • Computational Models: Developing computational models that can truly realize symbol emergence, integrating multimodal sensorimotor information, social interaction, and dynamic symbol negotiation, remains a significant challenge.
  • Cognitive Architectures: Creating cognitive architectures that enable robots to learn behaviors and language from sensorimotor systems in an unsupervised manner is crucial.
  • Unsupervised Learning: While progress has been made, robust unsupervised learning methods for syntax, meaning of function words, and nonlinguistic symbol systems (gestures, gaze) are still needed.
  • Human-Robot Communication: Developing methods for robots to share symbols and beliefs with humans based on shared experiences, incorporating mechanisms for inferring each other's beliefs (theory of mind), is essential for natural interaction.
  • Bridging the Gap: Integrating probabilistic generative models with neural networks and neuro-dynamic models is seen as a promising avenue for building more capable cognitive architectures.
  • Defining Terminology: The paper emphasizes the need for clearer definitions of terms like "concept," "category," "symbol," and "feature representation" to avoid confusion in interdisciplinary research.

Summary updated

Abstract

Humans use signs, e.g., sentences in a spoken language, for communication and thought. Hence, symbol systems like language are crucial for our communication with other agents and adaptation to our real-world environment. The symbol systems we use in our human society adaptively and dynamically change over time. In the context of artificial intelligence (AI) and cognitive systems, the symbol grounding problem has been regarded as one of the central problems related to symbols. However, the symbol grounding problem was originally posed to connect symbolic AI and sensorimotor information and did not consider many interdisciplinary phenomena in human communication and dynamic symbol systems in our society, which semiotics considered. In this paper, we focus on the symbol emergence problem, addressing not only cognitive dynamics but also the dynamics of symbol systems in society, rather than the symbol grounding problem. We first introduce the notion of a symbol in semiotics from the humanities, to leave the very narrow idea of symbols in symbolic AI. Furthermore, over the years, it became more and more clear that symbol emergence has to be regarded as a multifaceted problem. Therefore, second, we review the history of the symbol emergence problem in different fields, including both biological and artificial systems, showing their mutual relations. We summarize the discussion and provide an integrative viewpoint and comprehensive overview of symbol emergence in cognitive systems. Additionally, we describe the challenges facing the creation of cognitive systems that can be part of symbol emergence systems.
BibTeX
@article{taniguchi2019symbol,
  author = {Tadahiro Taniguchi and Justus H. Piater and Florentin Wörgötter and Emre Ugur and Matej Hoffmann and Lorenzo Jamone and Takayuki Nagai and Benjamin Rosman and Toshihiko Matsuka and Naoto Iwahashi and Erhan Öztop},
  title = {{Symbol Emergence in Cognitive Developmental Systems: A Survey.}},
  journal = {IEEE Trans. Cogn. Dev. Syst.},
  year = {2019},
  doi = {10.1109/tcds.2018.2867772},
}
Figure