LuxTac: A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning

Md Rakibul Islam Prince, John Pelan, Zhiyuan Zhang, Adyaa Khera, Katherine Pesetski, Adeesh Desai, Akimizu Sugiyama, Minwoo Ki, Benjamin Alan Capelle, Temitope Ibrahim Amosa, Zijie Xu, Yihan Huang, Taehun Chung, Long Wang, Wei Li, Shaoting Lin, Yu ShearXiv 2026 · arXiv

TL;DR

LuxTac is a thin (3.5 mm), flexible, and open-source tactile sensor array that uses off-the-shelf digital color sensors to resolve three-axis force at every "taxel" (tactile pixel). By employing interchangeable gel cartridges with colored internal walls, it achieves high-fidelity shear sensing without the bulk of camera-based systems, significantly improving robot performance in precision tasks like peg-in-hole insertion.

What Problem Does This Paper Address?

For robots to perform complex manipulation, they need a sense of touch that goes beyond simple pressure. Ideally, a tactile sensor should be thin enough to tile over a gripper, flexible enough to cover curved surfaces, and capable of sensing shear (lateral force), which is critical for detecting slips.

Current solutions fall into two camps, both with major drawbacks:

  1. Piezoresistive arrays: These are thin and cheap but typically only sense pressure (normal force), suffer from significant electrical drift, and are prone to hysteresis.
  2. Camera-based sensors (e.g., GelSight): These provide incredible high-resolution 3D geometry and shear data but are bulky due to the required optical path for the camera, making them difficult to integrate into compact robot fingers.

LuxTac bridges this gap by creating a "digital" optical taxel array that is as slim as a resistive skin but as capable as a camera-based sensor.

Methods

The core of LuxTac is a grid of TCS34725 RGBC light-to-digital converters paired with LEDs on a flexible four-layer PCB. Each taxel is an independent digital device on its own I2C bus segment, managed by multiplexers. This architecture makes the sensor highly scalable; you can tile these cells into arrays of any size or aspect ratio without redesigning the readout electronics.

The sensing happens in a 2 mm translucent silicone layer capped with an opaque black film. As an object indents the silicone, the black film moves closer to the sensor, absorbing more light and decreasing the intensity of the back-scattered LED signal.

The "aha" moment comes from the interchangeable gel cartridges:

  • Normal Cartridge: A plain white silicone layer for standard pressure mapping.
  • Shear Cartridge: The silicone is divided into cubes by colored walls (red/green for the x-axis, yellow/blue for the y-axis). When a shear force tilts these walls, the color balance of the light reaching the sensor shifts. This allows the system to resolve 3-axis force at every single taxel using only color data—no cameras, magnets, or complex bump overlays required.
Fig. 1. LuxTac overview. (a) The normal and shear gel cartridges, which mount on the same taxel board. (b) Real-world interactions (white arrows indicate the direction of motion) and the corresponding LuxTac readings. With the normal cartridge, pressing yields a per-taxel pressure map, and tapping produces repeatable 2–4% intensity drops above the 3σ noise floor. With the shear cartridge, rolling and twisting yield per-taxel pressure (blue) and shear (orange arrows). (c) Simulated grasp in IsaacGym with TacSL, with the simulated tactile reading (inset).

Datasets

The authors evaluated LuxTac through several lenses:

  • Characterization Dataset: 240 normal presses and extensive shear tests performed using an automated CNC indenter and a high-precision six-axis ATI Gamma force/torque sensor for ground truth.
  • Sim-to-Real Benchmark: 12 distinct grasps (600 frames) of 3D-printed shapes (cuboid, cylinder, ellipse, peg) were compared against simulated versions in IsaacGym using the TacSL library.
  • Robot Learning Tasks: 50 human demonstrations each for two real-world tasks: peg-in-hole insertion and cube sorting (distinguishing smooth vs. dotted surfaces).

Key Insights and Ideas

  • Color as a Shear Encoder: Using colored internal walls to turn a simple light intensity sensor into a 3-axis force vector sensor is a brilliant bit of "mechanical intelligence." It moves the complexity from the electronics/software into the material fabrication.
  • Digital at the Source: By using light-to-digital converters, the analog signal never leaves the sensor package. This eliminates the noise and crosstalk issues common in large-scale analog tactile skins.
  • Sim-to-Real Fidelity: Because the sensor responds to local indentation, it is remarkably easy to simulate. The authors achieved a 0.942 cosine similarity between real and simulated contact patterns, which is a strong green light for training robot policies in simulation.
Sensing principle under normal and shear loading.

Key Results

  • Precision: The normal cartridge shows a highly linear response ($R^2 = 0.999$) with a detection threshold of ~104 mN and excellent repeatability (2.0%).
  • Shear Accuracy: The shear cartridge models reached an $R^2$ of 0.96/0.96/0.92 for summed x/y/z forces. The median error in shear direction was only 4.8°.
  • Robot Performance: In the peg-in-hole task, adding LuxTac data to an Action Chunking Transformer (ACT) policy increased the success rate from 40% (vision-only) to 87%.
  • Cost: The prototype costs roughly $6.30 per taxel, with 97% of the cost coming from the off-the-shelf sensors, suggesting significant room for price drops with next-gen components.

Limitations and Future Work

While impressive, LuxTac has a few hurdles to clear:

  • Shape Dependency: Like most deformation-based sensors, the force estimate depends on the shape of the object (a sharp point indents deeper than a flat surface for the same force).
  • Material Creep: The silicone exhibits viscoelastic behavior, meaning the signal "creeps" under sustained load, which the current quasi-static models don't fully account for.
  • Thermal Drift: LED intensity drifts with temperature, requiring the sensor to be re-baselined at the start of each robot episode.

Future work aims to refine the wall geometry of the shear cartridge, test with more complex (deformable/sharp) objects, and use the per-taxel shear data directly as an input for more reactive robot control policies.

For those interested in building their own, the authors have open-sourced the design files, code, and fabrication guides at https://luxtac.github.io.

Summary updated

Abstract

Low-cost tactile arrays have made touch a practical input for robot learning, but piezoresistive designs sense only pressure, while camera-based sensors that also resolve shear are too bulky to tile over a gripper. To solve this problem, we present LuxTac, a flexible, scalable optical taxel array, about 3.5 mm thick and built from off-the-shelf parts. Each taxel is a digital RGBC light-to-digital converter under a translucent white silicone layer capped by an opaque black film, so the back-scattered LED light falls monotonically with local indentation depth. The same board accepts two interchangeable gel cartridges: a plain one for normal sensing, and one with colored walls that encodes the local shear in the color of the returned light, with no camera, magnets, or bump overlay, making LuxTac, to our knowledge, the first optical taxel array to resolve shear at individual taxels. Characterized with an automated CNC indenter and a six-axis force/torque sensor, the normal cartridge responds linearly to force ($R^2$ = 0.999) from 104 mN to at least 5.8 N, with 2.0% repeatability, 5.9% hysteresis, and a taxel-to-taxel variation of 6-10% after normalization. With the shear cartridge, per-taxel models estimate the three-axis force at every taxel; across three indenter shapes and nine held-out sessions, the summed estimates reach $R^2$ = 0.96/0.96/0.92 in x/y/z, each taxel's estimate 0.92/0.93/0.94, and the median shear-direction error is 4.8 degrees. Adding LuxTac pressure readings to an Action Chunking Transformer policy substantially improves peg-in-hole success over a vision-only baseline. Design files, code, and videos are available at https://luxtac.github.io.
BibTeX
@misc{prince2026luxtac,
  author = {Md Rakibul Islam Prince and John Pelan and Zhiyuan Zhang and Adyaa Khera and Katherine Pesetski and Adeesh Desai and Akimizu Sugiyama and Minwoo Ki and Benjamin Alan Capelle and Temitope Ibrahim Amosa and Zijie Xu and Yihan Huang and Taehun Chung and Long Wang and Wei Li and Shaoting Lin and Yu She},
  title = {{LuxTac: A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning}},
  year = {2026},
  eprint = {2611.23575},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23575},
}

Loading related content...

Figure