Filamentary Convolution for SLI: A Brain-Inspired Approach with High Efficiency

Boyuan Zhang, Xibang Yang, Tong Xie, Shuyuan Zhu, Bing ZengSensors 2025 · DOI

TL;DR

This paper introduces "filamentary convolution," a novel approach for spoken language identification (SLI) that mimics human auditory processing. By focusing on frequency patterns within individual speech frames rather than across frames, it reduces computational complexity and improves feature extraction efficiency, leading to better language recognition performance.

What Problem Does This Paper Address?

Spoken Language Identification (SLI) aims to determine the language of a spoken utterance. Current deep learning approaches often use Time-Frequency Acoustic Features (TFAFs) derived from the Short-Time Fourier Transform (STFT). However, standard rectangular convolution kernels used in these systems tend to mix information across time frames, leading to aliasing and degrading the extraction of crucial frequency-based features like pitch and rhythm. This approach also deviates from how the human auditory system processes sound, which is more focused on filtering and analyzing frequency components within short time windows. The paper addresses this by proposing a convolution method that better aligns with this brain-inspired, frequency-focused processing.

Methods

The core innovation is filamentary convolution. Unlike traditional rectangular convolutions that span multiple time frames, filamentary convolution operates within a single time frame, focusing on the frequency domain. This "filtering-like" operation is designed to preserve inter-frame features and avoid frequency mixing and aliasing. The authors propose replacing standard convolution kernels with these filamentary kernels. They integrate this into a CNN-LSTM architecture, termed FCK-NN, which includes an encoding module, a multi-level information extraction (MIE) module, a splicing module, and an LSTM module for temporal processing. The paper also introduces an "incremental efficiency" (IE) metric to quantify the trade-off between accuracy gains and parameter increases when using filamentary convolution.

Datasets

The authors utilized several datasets for their experiments:

  • OpenSLR and Common Voice: These were used to construct a new, refined multilingual dataset.
  • Custom-Refined Dataset: This dataset comprises 100,717 recordings across 44 languages, created using a novel "examiner-referee" refinement strategy to ensure data quality, clarity, and sufficient duration.
  • Public Corpora: For cross-validation, they used subsets of the OpenSLR, Voxlingua107, and Common Voice 6.1 datasets.

Key Insights and Ideas

  • Brain-Inspired Processing: The filamentary convolution's focus on intra-frame frequency analysis directly mimics how the human auditory system is thought to process sound, prioritizing frequency filtering.
  • Avoiding Aliasing: By operating within a single frame, filamentary convolution inherently avoids the frequency mixing and aliasing issues associated with cross-frame convolutions.
  • Parameter Efficiency: Filamentary convolution requires fewer parameters than traditional methods while achieving comparable or better accuracy, leading to improved computational efficiency.
  • Importance of Intra-Frame Features: The results suggest that preserving detailed temporal relationships within frames is crucial for effective SLI, and premature integration of inter-frame information can be detrimental.
  • Data Quality Matters: The creation of a high-quality, carefully curated dataset significantly impacts model performance and generalization.

Key Results

  • Filamentary convolution demonstrated superior performance in capturing critical frequency variations (intonation, rhythm) compared to rectangular and 1-D temporal convolutions, as shown by CAM activation maps.
  • In experiments, filamentary convolution led to higher accuracy and significantly improved incremental efficiency (IE) compared to baseline methods. For instance, one configuration achieved an IE of 7.501 with Fbank features.
  • The FCK-NN model, incorporating filamentary convolution, achieved an accuracy of 92.38% on their curated dataset using Fbank features, outperforming other methods.
  • Cross-validation results showed that their curated dataset led to better generalization capabilities compared to other publicly available datasets.
  • Ablation studies confirmed the importance of each component in the FCK-NN architecture, with the MIE module being particularly vital.

Limitations and Future Work

  • Limited Kernel Size Exploration: While the paper explores some kernel sizes, a more exhaustive search could potentially yield further improvements.
  • Generalizability to Other Tasks: While the paper focuses on SLI, the authors suggest that filamentary convolution could be beneficial for other acoustic-based tasks, which warrants further investigation.
  • Computational Cost of Refinement: The examiner-referee strategy for data curation, while effective, is labor-intensive and may not be scalable for extremely large datasets.
  • Future Work: The authors suggest exploring filamentary convolution in other frameworks lacking temporal extraction modules and investigating its potential for more direct retrieval of original speech positions corresponding to key information. They also propose further validation on other acoustic-based models.

Summary updated

Abstract

Spoken language identification (SLI) relies on detecting key frequency characteristics like pitch, tone, and rhythm. While the short-time Fourier transform (STFT) generates time-frequency acoustic features (TFAF) for deep learning networks (DLNs), rectangular convolution kernels cause frequency mixing and aliasing, degrading feature extraction. We propose filamentary convolution to replace rectangular kernels, reducing the parameters while preserving inter-frame features by focusing solely on frequency patterns. Visualization confirms its enhanced sensitivity to critical frequency variations (e.g., intonation, rhythm) for language recognition. Evaluated via self-built datasets and cross-validated with public corpora, filamentary convolution improves the low-level feature extraction efficiency and synergizes with temporal models (LSTM/TDNN) to boost recognition. This method addresses aliasing limitations while maintaining computational efficiency in SLI systems.
BibTeX
@inproceedings{zhang2025filamentary,
  author = {Boyuan Zhang and Xibang Yang and Tong Xie and Shuyuan Zhu and Bing Zeng},
  title = {{Filamentary Convolution for SLI: A Brain-Inspired Approach with High Efficiency}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25103085},
}
Figure