Developmental Dynamics of RNNPB: New Insight about Infant Action Development

Jun-Cheol Park, Dae-Shik Kim, Yukie NagaiSAB 2014 · DOI

TL;DR

This paper proposes a Recurrent Neural Network with Parametric Bias (RNNPB) model to explain how infants learn goal-directed actions. The model demonstrates a developmental progression where it first learns to achieve the goal of an action and then refines the specific movements (means) to reach that goal, mirroring observed infant behavior.

What Problem Does This Paper Address?

Developmental psychology has observed that infants' ability to imitate actions evolves over time. Specifically, younger infants tend to focus on replicating the goal of an action, while older infants can also accurately reproduce the means (the specific movements or trajectory) used to achieve that goal. This paper aims to computationally model this developmental shift in action learning, seeking to understand the underlying mechanisms that allow for this progression from goal-focused to means-inclusive imitation.

Methods

The authors employ a Recurrent Neural Network with Parametric Bias (RNNPB) architecture, a variation of Jordan-type recurrent neural networks. The key innovation lies in the "Parametric Bias" (PB) units, which are self-organized during the learning process. These PB units are hypothesized to encode different aspects of actions. The RNNPB is trained using Backpropagation Through Time (BPTT) to minimize errors between desired and generated motor behaviors. The learning process is analyzed by observing how the PB units' "space" (a 2D representation) evolves over training iterations, correlating this evolution with the agent's ability to reproduce both the goal and the means of various actions.

Datasets

The paper does not use external datasets. Instead, it defines a simulated environment with a virtual robot arm. Six reference "goal-directed actions" are created within this environment, each characterized by one of two goal positions (A or B) and one of three movement styles (simple or two types of "hopping" movements). These six actions serve as the training and evaluation data for the RNNPB model.

Key Insights and Ideas

  • Phased Learning: The RNNPB model exhibits a clear phased learning process: it first learns to achieve the goal of an action and only later learns to accurately reproduce the means (movement style). This directly mirrors observed developmental stages in infant imitation.
  • PB Space Dynamics: The self-organization of the PB units' space is a crucial indicator of learning progress. Initially undifferentiated, the PB space gradually separates, allowing the network to distinguish between different actions.
  • Salience of Features: The model suggests that the developmental shift is driven by the relative salience of the goal versus the means. The learning error function implicitly prioritizes goal achievement, making the means a more salient feature for learning only after the goal is sufficiently mastered.
  • Internal Maturation: The model's development occurs through internal parameter updates without external signals, suggesting that internal "maturation" can drive the acquisition of complex skills.

Key Results

  • After 10,000 training iterations, the RNNPB model could accurately reproduce simple actions (goal and basic trajectory) but struggled with more complex "hopping" movements, indicating that the goal was learned but the means were not fully acquired.
  • After 200,000 training iterations, the model successfully reproduced all six actions, accurately achieving both the goal and the specific movement styles.
  • The error for the goal (Egoal) became smaller than the error for the means (Emeans) after sufficient training, supporting the idea that goal achievement is mastered before precise movement control.

Limitations and Future Work

  • Limited Modality: The current model relies solely on motor commands and does not incorporate other sensory inputs like vision, which are crucial for infant learning.
  • Simplified Actions: The simulated actions, while varied, are still a simplification of real-world infant interactions.
  • Future Work: The authors intend to extend the model by incorporating multimodal representations (e.g., visual information) to better simulate infant experiments and bridge the gap between their model and actual infant mechanisms. They also aim to explore how the internal maturing procedure of the network can explain the development of goal-directed actions without external signals.

Summary updated

BibTeX
@inproceedings{park2014developmental,
  author = {Jun-Cheol Park and Dae-Shik Kim and Yukie Nagai},
  title = {{Developmental Dynamics of RNNPB: New Insight about Infant Action Development}},
  booktitle = {SAB},
  year = {2014},
  doi = {10.1007/978-3-319-08864-8\_14},
}
Figure