TL;DR
This paper introduces PerceptNet, a novel deep learning approach for learning how humans perceive the similarity between haptic textures. Unlike previous methods, PerceptNet effectively uses "unorderable" human judgments (where people are unsure which of two textures is more similar to a reference) to build a more robust perceptual model, outperforming existing techniques.
What Problem Does This Paper Address?
Designing intuitive haptic interfaces, like those used for accessibility or enhanced user experiences, requires a way to map haptic signals (like vibrations or textures) to meaningful concepts. This mapping relies on understanding how humans perceive the similarity between different haptic signals. However, accurately capturing this human perception is challenging because:
- Humans are poor at quantifying similarity: People struggle to assign consistent numerical values to how similar two textures feel. They are better at making relative comparisons (e.g., "texture A is more similar to texture B than to texture C").
- Ambiguity in perception: Sometimes, people are genuinely uncertain about which of two textures is more similar to a reference. Previous methods often ignored these "unorderable" or ambiguous judgments, potentially losing valuable information.
- Data collection is tedious: Gathering large amounts of precise haptic similarity data from humans is expensive and time-consuming.
This paper aims to address these challenges by developing a method that can learn a perceptual similarity metric from readily available, non-numerical, and even ambiguous human comparisons.
Methods
The core of PerceptNet is a deep neural network designed to learn a mapping from raw haptic signal features to an embedding space. In this space, the standard Euclidean distance between two embedded signals directly corresponds to their perceived dissimilarity by humans.
Key methodological contributions include:
- Parametric Deep Learning Approach: Instead of non-parametric methods like Multidimensional Scaling (MDS) that require extensive pairwise comparisons, PerceptNet uses a parametric deep neural network. This allows it to generalize better, learn from partial data, and even embed new, unseen haptic signals without retraining.
- Novel Triplet Loss Function: The network is trained using triplets of haptic signals (a reference, a similar signal, and a dissimilar signal). Crucially, PerceptNet's loss function handles two types of triplets:
- High-margin triplets: Where humans clearly indicate one signal is more similar than another to the reference.
- Low-margin triplets: Where humans are uncertain about the ordering (i.e., the signals are perceived as similarly different from the reference). By incorporating these ambiguous triplets, PerceptNet captures more nuanced human perception.
- CQFB Spectral Features: The authors found that pre-processing the raw haptic acceleration data into Constant Q-Factor Bank (CQFB) spectral features improved the network's performance. These features compactly summarize periodic patterns in the haptic signals, making them easier for the network to learn from.
Datasets
- TUM Texture Dataset: This dataset, containing acceleration signals from 108 different surface materials, was used for training and evaluation. The authors leveraged existing perceptual similarity judgments from this dataset to construct their training and testing triplets. They defined ground-truth perceptual distance as the fraction of subjects who could distinguish between two signals.
Key Insights and Ideas
- Unorderable triplets are valuable: The most significant insight is that explicitly modeling human uncertainty (low-margin triplets) leads to a more accurate perceptual similarity metric than ignoring these ambiguous judgments. This allows the model to better capture the "nuances and limitations of human perception."
- Deep learning for haptic perception: PerceptNet demonstrates the power of deep neural networks for learning complex, non-linear perceptual relationships in haptic data, outperforming traditional linear methods.
- Spectral features aid learning: Using CQFB spectral features as input to the network, rather than raw acceleration data, leads to better performance and a more efficient model architecture.
Key Results
- Superior performance: PerceptNet significantly outperformed baseline methods (Euclidean distance and learned Mahalanobis distance) across various experimental setups, including "held-out triplets," "held-out samples," and "held-out classes."
- High generalization accuracy: In the "held-out triplets" experiment, PerceptNet achieved 84% triplet generalization accuracy (TGA), compared to ~50% for baselines. Even in the challenging "held-out classes" scenario, PerceptNet achieved 67% TGA.
- Accurate pairwise classification: PerceptNet was highly accurate (AUC = 0.97) in predicting whether pairs of signals were distinguishable or not, significantly outperforming baselines (AUC = 0.69, 0.66).
- Scalability: The model's performance increased with more training data, showing diminishing returns for larger datasets, indicating good scalability.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the CQFB features are a "compact initial encoding" and that a more complex, hand-tuned network might achieve comparable results with raw input.
- The current model's performance drops when generalizing to entirely unseen classes, as expected.
- Future Work:
- Modeling more complex nuances of human perception, such as just-noticeable differences (JND).
- Generalizing to a wider range of novel signals.
- Developing an online learning framework to incrementally train the model.
Summary updated