Deformation Capture via Self-Sensing Capacitive Arrays (Video)

Oliver Glauser, Daniele Panozzo, Otmar Hilliges, Olga Sorkine-HornungIROS 2018 · DOI

Introduction

Traditional motion capture (mocap) systems typically rely on optical camera arrays and markers, which are constrained by line-of-sight requirements. These systems often fail when subjects move behind obstacles or wear loose-fitting clothing. While wearable inertial measurement units (IMUs) offer a portable alternative, they are generally limited to tracking skeletal kinematics and cannot capture soft-tissue deformations such as muscle bulging or skin stretching.

To address these limitations, researchers have developed a low-cost, wearable sensor array capable of reconstructing dense 3D surface deformations in real-time. This system utilizes a soft, stretchable silicone-based capacitive sensor array combined with a deep learning-based reconstruction pipeline.

Overview of the soft sensor array hardware and its application in real-time 3D arm reconstruction.

Methodology

Fabrication and Sensing

The sensors are constructed from multiple layers of soft silicone, with conductive regions created by mixing silicone with carbon black. A key innovation in the fabrication process is the use of a standard laser cutter to etch electrode patterns directly into the layers. This digital fabrication approach ensures high-precision alignment and significantly reduces costs compared to traditional molding or manual assembly.

The sensor operates as a capacitive array. As the silicone stretches, the overlapping area of the conductive plates changes, resulting in a measurable change in capacitance. To minimize wiring complexity, a time-multiplexing scanning scheme is employed, allowing the system to resolve area changes at 92 distinct sensor cells using only 24 leads. Additionally, wearing a thin nylon sock under the sensor acts as an electrical shield against the body's natural field, stabilizing capacitive readings.

Real-time visualization of capacitive sensing data mapped to a 2D heatmap during manual manipulation.

3D Reconstruction Pipeline

Because area changes alone do not uniquely determine a 3D shape, a Deep Neural Network (DNN) is used to map capacitance readings to 3D vertex positions. The network is trained on a custom dataset where ground-truth 3D positions were captured using an 8-camera OptiTrack system. To ensure physical plausibility, the DNN's output is further refined using a nonlinear elastic deformation model.

Experimental Results

The system's performance was evaluated across several scenarios, including human joint articulation (wrist and elbow) and muscle bulging (biceps). The reconstruction achieved a mean marker error of approximately 3.5 mm for the wrist and elbow. In tests on freeform objects, such as an inflating balloon, the error was reduced to 2.75 mm.

The pipeline operates at approximately 8 Hz, enabling interactive real-time applications. Durability tests demonstrated that the sensor could withstand over 550 cycles of stretching to twice its original length without significant loss in accuracy or physical degradation.

Discussion and Future Work

While the current system provides a robust solution for local deformation capture, it currently requires per-user calibration and is limited to sensing deformations directly beneath the array. Future work will focus on developing universal models through larger, multi-user datasets and integrating additional sensor types, such as bend sensors, to capture non-stretching deformations like folding. This technology paves the way for infrastructure-free, high-fidelity motion capture in diverse environments.

Summary updated

BibTeX
@inproceedings{glauser2018deformation,
  author = {Oliver Glauser and Daniele Panozzo and Otmar Hilliges and Olga Sorkine-Hornung},
  title = {{Deformation Capture via Self-Sensing Capacitive Arrays (Video)}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  doi = {10.1109/iros.2018.8594203},
}

Loading related content...

Figure