Acoustic scattering AI for non-invasive object classifications: A case study on hair assessment

Interspeech 2025

TL;DR

This paper introduces a novel non-invasive object classification method using acoustic scattering, demonstrating its effectiveness in classifying hair type and moisture. By analyzing the scattered sound waves generated when acoustic stimuli interact with hair, the researchers achieved nearly 90% accuracy using fine-tuned self-supervised learning models, offering a privacy-preserving alternative to visual methods.

What Problem Does This Paper Address?

Traditional object classification heavily relies on visual data, which can be limited by lighting conditions, occlusion, and privacy concerns. Furthermore, visual methods primarily capture surface-level information, making it difficult to assess internal structures or material properties. The paper addresses this gap by proposing an acoustic scattering-based approach. When sound waves interact with an object, they generate a scattered acoustic field that encodes its structural and material characteristics. This method aims to provide a non-contact, privacy-preserving way to classify objects, with a specific focus on the challenging task of hair assessment, which is relevant to various industries.

Methods

The core of the paper lies in applying advanced AI techniques to analyze acoustic scattering signals. The authors explored four main classification strategies:

  • Fully Supervised Training: Using a ResNet-50 model adapted for single-channel mel-scaled spectrograms.
  • Embedding-Based Classification: Employing the VGGish model to extract embeddings, which were then fed into an XGBoost classifier.
  • Foundation Model Supervised Fine-tuning: Adapting the Audio Spectrogram Transformer (AST) model, pre-trained on ImageNet and AudioSet, for acoustic scattering data.
  • Self-Supervised Learning (SSL) Model Fine-tuning: This was the most promising approach, utilizing the Wav2Vec2-Conformer large model. They experimented with two fine-tuning strategies: partial fine-tuning (updating only the Conformer Encoder) and complete fine-tuning (updating all parameters).

The acoustic stimulus used was an Exponential Sine Sweep (ESS), chosen for its high signal-to-noise ratio in impulse response measurements. Experiments were conducted in a soundproof room to ensure controlled conditions.

Datasets

The study created a custom dataset for hair assessment. This dataset was constructed from multiple recording rounds, with each session involving untangled and combed hair samples attached to dummy mannequin heads. An ESS impulse was played, and the scattered acoustic signal was recorded by dual microphones. The data was resampled to 48kHz. The dataset comprises:

  • 4 hair classes: Represented by different dummy heads (A, B, MAMI, MINAYO).
  • 3 hair condition patterns: Dry hair, hair with shampoo applied, and hair with cream applied.

The dataset was split using round-robin cross-validation for evaluation.

Key Insights and Ideas

  • Acoustic Scattering as a Rich Information Source: The research demonstrates that scattered acoustic fields contain significant information about an object's physical properties, extending beyond surface-level visual cues.
  • SSL Models Excel in Acoustic Classification: Fine-tuning self-supervised learning models, particularly Wav2Vec2-Conformer, proved to be the most effective strategy for acoustic scattering classification, outperforming traditional supervised and embedding-based methods.
  • Importance of Convolutional Layers: The results suggest that convolutional neural networks (CNNs), even when integrated within larger architectures like Wav2Vec2-Conformer, play a crucial role in capturing local features essential for acoustic classification.
  • Privacy-Preserving Alternative: The acoustic scattering approach offers a compelling alternative to visual methods, especially in scenarios where privacy is a concern, as it doesn't require capturing identifiable visual data.
  • Hair Assessment Feasibility: The study successfully shows that acoustic scattering can differentiate between hair types and moisture levels, opening doors for applications in the beauty and personal care industries.

Key Results

The paper reports impressive classification accuracies:

  • Task 1 (Hair Type Classification): The best performing model, Wav2Vec2-Conformer with complete fine-tuning, achieved an accuracy of 87.4% ± 13.9% and an average AUC of 0.97.
  • Task 2 (Hair Condition Classification): Again, Wav2Vec2-Conformer with complete fine-tuning led the pack, achieving an accuracy of 89.8% ± 10.6% and an average AUC of 0.96.

These results highlight the superiority of fine-tuned SSL models for this specific task.

Limitations and Future Work

  • Dataset Size and Diversity: While the dataset was carefully constructed, the authors acknowledge the need for larger and more diverse datasets to further validate the robustness of the models across a wider range of hair types and conditions.
  • Environmental Factors: Although experiments were conducted in a controlled soundproof room, real-world applications might face challenges with ambient noise and reverberation. Future work could explore more robust noise-cancellation techniques.
  • Generalizability: The study focused on hair assessment. Extending this acoustic scattering approach to classify other complex objects and materials would be a valuable next step.
  • Real-time Implementation: Investigating the feasibility of real-time acoustic classification for practical applications is also a promising avenue for future research.

The authors have made their code and analysis open-sourced, encouraging further exploration and development in this area.

Summary updated

BibTeX
@inproceedings{2025acoustic,
  title = {{Acoustic scattering AI for non-invasive object classifications: A case study on hair assessment}},
  booktitle = {Interspeech},
  year = {2025},
}
Figure