Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks

Giovanni Sutanto, Zhe Su, Stefan Schaal, Franziska Meier

TL;DR

This paper introduces Phase-Modulated Neural Networks (PMNNs), a novel architecture that allows robots to learn complex, time-dependent feedback strategies directly from human demonstrations. By embedding the "phase" (or progress) of a movement into a neural network, the authors enable an anthropomorphic robot to use tactile "feel" to adapt its orientation in real-time—successfully scraping a tilted board even when the exact tilt is unknown.

What Problem Does This Paper Address?

In structured environments, robots can follow pre-programmed trajectories with ease. However, real-world manipulation is messy. If a robot is scraping a surface and the surface tilts, a fixed trajectory will fail: the tool will either lose contact or jam into the board.

To handle this, robots need a feedback model that maps deviations in sensory data (like tactile pressure) to corrections in action. Traditionally, these models are either hand-designed (which is nearly impossible for high-dimensional sensors like tactile skins) or learned using standard neural networks. The problem with standard networks is that they are "phase-blind"—they don't realize that the same sensory "bump" might require a different correction at the beginning of a task versus the end.

Methods

The researchers built their framework on Dynamic Movement Primitives (DMPs), a mathematical way to represent robot motions as a set of differential equations. DMPs have an internal "clock" called a phase variable ($p$).

The core innovation is the Phase-Modulated Neural Network (PMNN):

  1. Input Layer: Receives the error signal ($\Delta S$), which is the difference between what the robot expects to feel and what it actually feels.
  2. Feature Extraction: Regular hidden layers process this high-dimensional tactile data into useful features.
  3. Phase Modulation: The final hidden layer is multiplied by "phase kernels." This ensures the network's output is gated by the current progress of the task.
  4. Convergence Guarantee: Because the phase velocity goes to zero at the end of a movement, the PMNN naturally "shuts off" its corrections, ensuring the robot reaches its final goal stably.

Datasets

The authors collected a custom dataset using a Barrett WAM arm equipped with BioTac biomimetic tactile sensors.

  • Task: Scraping paint off a dry-erase board.
  • Variations: The board was tilted at five different angles (0°, 2.5°, 5°, 7.5°, and 10°).
  • Demonstrations: 15 demonstrations were recorded for each setting. In the non-zero tilt settings, a human provided "corrections" by manually adjusting the robot's orientation while it attempted its nominal path, creating a supervised dataset of "Sensory Error $\to$ Corrective Action."

Key Insights and Ideas

  • The "When" Matters: Standard Feedforward Neural Networks (FFNNs) struggle because they lack temporal context. PMNNs solve this by making the feedback policy a function of both the sensory error and the task phase.
  • End-to-End beats Modular: The authors compared PMNNs to a system where features were first extracted via PCA (Principal Component Analysis) and then fed into a phase-dependent model. The PMNN performed better because it learns to extract features that are specifically useful for phase-dependent feedback.
  • Tactile-Driven Adaptation: The paper demonstrates that tactile sensors can replace vision for fine-grained orientation control, which is crucial for tasks where the tool obscures the line of sight.

Key Results

  • Superior Accuracy: In regression tests (Table II), the PMNN achieved low Normalized Mean Squared Error (NMSE) across training, validation, and testing (ranging from 0.15 to 0.22).
  • Generalization: The PMNN significantly outperformed standard FFNNs in generalizing to unseen demonstrations.
  • Real-World Success: The model was deployed on a physical robot. Without any prior knowledge of the board's tilt, the robot successfully used its "sense of touch" to adjust its tool orientation in real-time, maintaining flat contact with the board throughout the scraping motion.

Limitations and Future Work

  • Demonstration Effort: The approach requires a human to manually provide corrections for various environmental settings, which can be time-consuming.
  • Task Specificity: While the framework is general, it was only evaluated on a single scraping task. Future work could explore more complex, multi-stage manipulation.
  • Bootstrap for RL: The authors suggest that this "Learning from Demonstration" approach could be used to "bootstrap" (initialize) Reinforcement Learning, allowing a robot to start with a decent policy and then refine it through self-practice.

Summary updated

Abstract

In order to robustly execute a task under environmental uncertainty, a robot needs to be able to reactively adapt to changes arising in its environment. The environment changes are usually reflected in deviation from expected sensory traces. These deviations in sensory traces can be used to drive the motion adaptation, and for this purpose, a feedback model is required. The feedback model maps the deviations in sensory traces to the motion plan adaptation. In this paper, we develop a general data-driven framework for learning a feedback model from demonstrations. We utilize a variant of a radial basis function network structure -with movement phases as kernel centers- which can generally be applied to represent any feedback models for movement primitives. To demonstrate the effectiveness of our framework, we test it on the task of scraping on a tilt board. In this task, we are learning a reactive policy in the form of orientation adaptation, based on deviations of tactile sensor traces. As a proof of concept of our method, we provide evaluations on an anthropomorphic robot.
BibTeX
@inproceedings{sutanto2018learning,
  author = {Giovanni Sutanto and Zhe Su and Stefan Schaal and Franziska Meier},
  title = {{Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2018},
  url = {https://openreview.net/forum?id=vH0dt-Q10Qh},
}

Loading related content...

Figure