Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks

Giovanni Sutanto, Zhe Su, Stefan Schaal, Franziska Meier

Abstract

In order to robustly execute a task under environmental uncertainty, a robot needs to be able to reactively adapt to changes arising in its environment. The environment changes are usually reflected in deviation from expected sensory traces. These deviations in sensory traces can be used to drive the motion adaptation, and for this purpose, a feedback model is required. The feedback model maps the deviations in sensory traces to the motion plan adaptation. In this paper, we develop a general data-driven framework for learning a feedback model from demonstrations. We utilize a variant of a radial basis function network structure --with movement phases as kernel centers-- which can generally be applied to represent any feedback models for movement primitives. To demonstrate the effectiveness of our framework, we test it on the task of scraping on a tilt board. In this task, we are learning a reactive policy in the form of orientation adaptation, based on deviations of tactile sensor traces. As a proof of concept of our method, we provide evaluations on an anthropomorphic robot.

Introduction

While robots excel in structured environments following pre-programmed trajectories, real-world manipulation presents significant challenges due to environmental uncertainty. For instance, in a surface scraping task, an unknown tilt in the board can cause a fixed trajectory to either lose contact or jam the tool. To achieve robustness, robots require a feedback model capable of mapping deviations in sensory data—such as tactile pressure—to real-time corrective actions.

Traditional feedback models are often hand-designed, which is impractical for high-dimensional sensors like tactile skins. Alternatively, standard neural networks can be used, but they are typically "phase-blind." They fail to account for the fact that a specific sensory error might necessitate different corrections depending on whether the robot is at the beginning, middle, or end of a movement primitive.

Methodology: Phase-Modulated Neural Networks

The proposed framework builds upon Dynamic Movement Primitives (DMPs), which represent robot motions through differential equations governed by an internal "clock" or phase variable ($p$). To address the limitations of standard networks, the authors introduce the Phase-Modulated Neural Network (PMNN).

The Phase-Modulated Neural Network (PMNN) architecture, showing the integration of phase kernels ($G_i$) to gate the network's output based on task progress.

The PMNN architecture consists of several key components:

  1. Input Layer: Processes the error signal ($\Delta S$), defined as the difference between actual and expected sensory traces.
  2. Feature Extraction: Regular hidden layers extract high-level features from high-dimensional tactile data.
  3. Phase Modulation: A specialized hidden layer is modulated by phase kernels ($G_i$). This ensures the network's output is gated by the current progress of the task.
  4. Convergence Stability: Since the phase velocity approaches zero at the end of a movement, the PMNN naturally attenuates corrections, ensuring the robot reaches its target state stably.

Experimental Setup and Dataset

The framework was evaluated using an anthropomorphic Barrett WAM arm equipped with BioTac biomimetic tactile sensors. The chosen task was scraping paint off a dry-erase board mounted on a tilt stage.

Experimental setup for the scraping task, featuring a Barrett WAM arm, BioTac tactile sensors, and a tiltable dry-erase board.

To train the model, a dataset was collected across five board angles (0°, 2.5°, 5°, 7.5°, and 10°). For non-zero tilts, a human provided corrective demonstrations by manually adjusting the robot's orientation while it executed its nominal path. This created a supervised dataset mapping sensory errors to corrective actions.

Key Insights and Results

The study yielded several critical insights into learning reactive policies:

  • Temporal Context is Essential: Standard Feedforward Neural Networks (FFNNs) struggle without temporal context. By making the feedback policy a function of both sensory error and task phase, PMNNs achieve significantly higher accuracy.
  • End-to-End Learning: PMNNs outperformed modular systems (e.g., PCA-based feature extraction followed by a phase-dependent model) because they learn features specifically optimized for phase-modulated feedback.
  • Tactile-Driven Adaptation: The results demonstrate that tactile feedback can effectively replace or augment vision for fine-grained control, especially when the tool obscures the workspace.

Quantitatively, the PMNN achieved a low Normalized Mean Squared Error (NMSE) between 0.15 and 0.22 across various test scenarios. It demonstrated superior generalization to unseen demonstrations compared to standard FFNN architectures. In physical deployments, the robot successfully adapted its tool orientation in real-time to maintain flat contact with the board, despite having no prior knowledge of the surface tilt.

Conclusion and Future Work

This work presents a robust method for learning time-dependent feedback strategies directly from human demonstrations. While effective, the approach currently requires significant human effort to provide corrections across different environmental settings. Future research could explore multi-stage manipulation tasks or use this framework to bootstrap Reinforcement Learning, allowing robots to refine their initial learned policies through autonomous practice.

Summary updated

Abstract

In order to robustly execute a task under environmental uncertainty, a robot needs to be able to reactively adapt to changes arising in its environment. The environment changes are usually reflected in deviation from expected sensory traces. These deviations in sensory traces can be used to drive the motion adaptation, and for this purpose, a feedback model is required. The feedback model maps the deviations in sensory traces to the motion plan adaptation. In this paper, we develop a general data-driven framework for learning a feedback model from demonstrations. We utilize a variant of a radial basis function network structure --with movement phases as kernel centers-- which can generally be applied to represent any feedback models for movement primitives. To demonstrate the effectiveness of our framework, we test it on the task of scraping on a tilt board. In this task, we are learning a reactive policy in the form of orientation adaptation, based on deviations of tactile sensor traces. As a proof of concept of our method, we provide evaluations on an anthropomorphic robot. A video demonstrating our approach and its results can be seen in https://youtu.be/7Dx5imy1Kcw
BibTeX
@inproceedings{sutanto2018learning,
  author = {Giovanni Sutanto and Zhe Su and Stefan Schaal and Franziska Meier},
  title = {{Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2018},
  url = {https://openreview.net/forum?id=vH0dt-Q10Qh},
}
Figure