A robot pressing a camera-based touch sensor into a part learns a lot about the surface, such as scratches, gouges, and height. It learns almost nothing about what lies a few millimetres below. We propose to make the robot's touch vibrate at a sweep of frequencies while it presses. In soft materials, a surface vibration at high frequency stays shallow and a vibration at low frequency reaches deeper. This is the same principle acoustic microscopes use to measure damaged layers in silicon, and satellite radars use to measure snow on sea ice. The robot can therefore read the part layer by layer from a single contact that lasts a fraction of a second.
The practical target is inspection during manufacture of compliant parts: cast silicone, elastomer overmoldings, printed soft polymers, and bonded soft laminates. In these parts, voids, inclusions, and delaminations are invisible from outside and are usually found only after the part is finished and sectioned or scrapped. The learning contribution is a neural network whose levels are indexed by excitation frequency, and therefore by probing depth, rather than by image resolution. Information flows from shallow levels to deep levels, and each deeper level is asked to explain only what the shallower levels could not. We call this differencing. It is a learned version of the two-frequency subtraction that makes radar snow-depth retrieval work.
Core Claim
Our claim is that a frequency sweep can stand in for waiting. Existing dynamic touch methods get their subsurface signal from slow loading and unloading. Using Fiber Optic Bundles to Miniaturize Vision-Based Tactile Sensors classifies hardness at 98% and detects phantom nodules at 100% from 16-frame indentation–relaxation videos. Establishing direct relationships between soft material perception and rheology shows that the firmness a contact reports is still changing over the first two seconds. That timescale conflicts with inspection during production. A harmonic sweep instead encodes depth in frequency. It can be run in a few hundred milliseconds per contact point, and it yields a continuous depth estimate rather than a present/absent label.
The scale argument comes from Rayleigh-wave physics, the same physics behind the half-wavelength rule in Measurement of a damaged layer thickness with SAM.. The probed depth is roughly $d(f) \approx c_R/(2f)$, where $c_R$ is the surface-wave speed and $f$ the excitation frequency. For a silicone with shear modulus near 100 kPa, $c_R$ is on the order of 10 m/s. A band from about 200 Hz to 5 kHz then spans probing depths from roughly centimetres down to about a millimetre, which matches the defect depths that matter in soft parts.
The same estimate shows where the idea fails. In stiff polymers and metals, wavelengths at these frequencies are far longer than the part is thick, so there is no depth selectivity. We scope the proposal to compliant materials and treat the stiffness at which selectivity disappears as something to measure, not assume.
Research Program
The probe. We will build it on a camera-based touch sensor with printed markers on its gel skin. A voice-coil actuator superimposes a small harmonic oscillation on a static preload. The camera is triggered in sync with the excitation at stepped phase offsets. This stroboscopic sampling reconstructs amplitude and phase maps of the vibrating deformation field across the contact patch at each frequency, even though the camera itself runs at ordinary frame rates. An accelerometer and force sensor on the actuator record mechanical impedance versus frequency. One contact therefore produces a stack of spatial maps indexed by frequency, plus a one-dimensional impedance spectrum.
The testbed. No public tactile dataset has subsurface defects at known depths, so we will build one. It consists of cast silicone and printed soft-polymer specimens at several stiffnesses, with embedded voids, rigid inclusions, and bonded-layer delaminations at depths from 0.5 to 10 mm. Ground truth comes from destructive sectioning. We will also build a set of PVC plates with back-drilled holes at 0 to 2.5 mm depth, replicating the geometry of the Université Laval IRT-PVC thermography specimens used by Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography. These PVC plates act as a stiffer boundary case and give us a point of contact with published numbers. Following the simulation-pretraining result of Deep learning for melt pool depth contour prediction from surface thermal images via vision transformers, which reached high accuracy with half the experimental data after simulator pretraining, we will pretrain on finite-element simulations of layered viscoelastic blocks.
The model. A feature pyramid network, introduced in Feature Pyramid Networks for Object Detection, merges coarse deep features into fine shallow ones by adding them through lateral connections. We keep that lateral structure and its spatial resolution levels, but add a second axis: frequency band $k$, ordered from highest (shallowest) to lowest (deepest). A shared encoder turns each frequency's amplitude–phase map into spatial features $F_k$. The depth-slice representation is then built shallow to deep:
$$h_k = \phi\big(F_k - W_k\, h_{k-1}\big).$$
Here $h_{k-1}$ is the representation of the next-shallower slice, $W_k$ is a learned $1\times1$ projection predicting how that shallow structure shows up at the lower frequency, and $\phi$ is a small convolutional block. Each $h_k$ decodes to a defect map for its depth slice. From these we regress a continuous depth estimate and a defect segmentation. The subtraction is the key ingredient. Low-frequency responses always mix in shallow structure, just as Ku-band radar echoes contain the snow surface. Forcing deeper levels to model the residual should keep a shallow void from being reported as a deep one.
The pipeline. The probe is placed inside a two-stage inspection loop modelled on Robotic Defect Inspection with Visual and Tactile Perception for Large-scale Components. There, a camera flags uncertain regions and a touch sensor confirms them, cutting inspection time 70-fold against exhaustive touch while reaching 100% recall on surface defects. In our version the camera screens for surface cues, and the vibrating probe visits a planned set of points. Most subsurface defects in soft parts have no visual cue at all, so visual gating alone cannot suffice. We will therefore also test a sparse-grid sampling policy and report inspection time per unit area.
Baselines
The naive reference is a single static contact with the same sensor and the same preload, fed through the same encoder. This is the static heightfield use of touch in Robotic Defect Inspection with Visual and Tactile Perception for Large-scale Components, upgraded to a learned model so the comparison isolates the signal rather than the classifier. Beating it only shows that dynamics help. Any dynamic method should clear it by a wide margin on defects deeper than about 1 mm.
The strongest competitor on our question is quasi-static indentation–relaxation video classified by a video masked autoencoder, as in Using Fiber Optic Bundles to Miniaturize Vision-Based Tactile Sensors. It reports 98% hardness accuracy and 100% nodule detection on its phantoms. For continuous tactile depth regression, we found no reported result on Lacuna. We will rerun this method on our specimens with a regression head, on the same train/test split, from the same pretrained initialisation, with the same hyperparameter search budget. We give it contact durations from 0.3 s up to 3 s so that it is not handicapped by a short window. Beating it means lower depth error and higher detection at matched contact time. It also means matching its accuracy at a small fraction of its contact time.
The physics-only baseline fits phase velocity against frequency from the phase maps and applies the half-wavelength inflection rule of Measurement of a damaged layer thickness with SAM., using bulk moduli measured on defect-free coupons. It needs no training data. The learned model must beat it clearly on non-planar defects such as inclusions and voids, not only on planar delaminations.
On the PVC replicas, the non-contact reference point is Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography. It reports a depth mean absolute error of 0.0082 cm and a segmentation mean intersection-over-union of 0.869 on IRT-PVC. We will compare against these published figures with the caveat that our specimens replicate but are not the same set.
Three ablations isolate the key ingredient. The first replaces the subtraction with the standard additive lateral connection. The second stacks all frequencies as input channels into a single-level network with no frequency axis. The third drops the phase maps and keeps only the impedance spectrum. All ablations share parameter count within 10% and the same training budget.
Why not just…?
Why not just press slowly and let a video model learn depth, as Di et al. do? This is the real threat. Relaxation dynamics also carry depth information, and Action Augmentation of Tactile Perception for Soft-Body Palpation shows that richer multi-axis trajectories raise subsurface inclusion classification by up to 35% over simple pressing. Our bet is about time, not ceiling accuracy. The two-second transients reported by Establishing direct relationships between soft material perception and rheology imply that slow palpation needs seconds per point. We will plot accuracy against contact time for both methods. If slow palpation matches us at equal time, the frequency premise fails, and we will report it as a negative result.
Why not just fit the dispersion curve and apply the half-wavelength rule, with no network? For planar layers this may be nearly as good, and it is interpretable. However, the rule assumes laterally uniform layers and known bulk constants, which the SAM work itself names as a requirement. Voids and inclusions scatter energy laterally and violate that assumption. If the physics-only baseline matches the network on inclusions too, we will ship the physics method.
Why not just use pulse thermography, which needs no contact? It is faster over large areas and well validated on PVC. But thermal contrast depends on a difference in conductivity. A delamination whose faces are still in contact, or an inclusion with thermal properties close to the matrix, may show little thermal signal while still changing stiffness. We state this as a hypothesis to test, not a finding. The PVC-replica comparison, and a set of silicone specimens with thermally matched inclusions, will show whether touch adds anything thermography misses.
Why not just use a standard feature pyramid or a flat transformer over the frequency stack? Multi-scale fusion clearly helps when defect sizes vary widely: DDNet: Deformable Convolution and Dense FPN for Surface Defect Detection in Recycled Books moves mean average precision from 32.5% to 46.7%. That result concerns spatial scale, not the mixing of shallow and deep responses across frequency. The additive and flat-stack ablations answer this question directly. If they match the subtraction variant, the architectural claim is withdrawn and the contribution reduces to the probe and dataset.
Proposed Experiments
The first check is physical. Before any learning, we measure how the phase-map response to a single embedded void changes with frequency, for void depths of 1, 3, and 6 mm, in three silicone stiffnesses. Depth selectivity should appear as a shift in the frequency at which the void's signature fades, following $c_R/(2f)$. If no such shift exists within our actuator's band, the project stops or narrows to the softest materials.
The main experiment trains on silicone and soft-polymer specimens and tests on held-out specimens. We report depth mean absolute error, detection recall and precision, and segmentation intersection-over-union per depth slice, all as functions of contact time. Recall is the metric that matters for scrap prevention.
Two transfer tests follow. The first moves from simulation-only pretraining to real specimens, measuring how much real data is needed with and without pretraining. The second transfers across stiffness, locating where accuracy collapses as materials stiffen.
Finally, a full-pipeline run on larger moulded panels measures inspection time per square decimetre and recall of subsurface defects. We compare three modes: the two-stage visual gate, sparse-grid probing, and exhaustive probing.
The decisive outcomes are two. The first is whether the subtraction pyramid beats the slow-palpation video model at matched short contact time. The second is whether it beats the physics-only fit on inclusions. Both must hold for the proposal's central claim to stand.
References
- Using Fiber Optic Bundles to Miniaturize Vision-Based Tactile Sensors
- Establishing direct relationships between soft material perception and rheology
- Measurement of a damaged layer thickness with SAM.
- Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography
- Deep Learning for Melt Pool Depth Contour Prediction From Surface Thermal Images via Vision Transformers
- Feature Pyramid Networks for Object Detection
- Robotic Defect Inspection with Visual and Tactile Perception for Large-Scale Components
- Action Augmentation of Tactile Perception for Soft-Body Palpation
- DDNet: Deformable Convolution and Dense FPN for Surface Defect Detection in Recycled Books