Abstract
We investigate the problem of direct waveform modelling using parametric kernel-based filters in a convolutional neural network (CNN) framework, building on SincNet, a CNN employing the cardinal sine (sinc) function to implement learnable bandpass filters. To this end, the general problem of learning a filterbank consisting of modulated kernel-based baseband filters is studied. Compared to standard CNNs, such models have fewer parameters, learn faster, and require less training data. They are also more amenable to human interpretation, paving the way to embedding some perceptual prior knowledge in the architecture. We have investigated the replacement of the rectangular filters of SincNet with triangular, gammatone and Gaussian filters, resulting in higher model flexibility and a reduction to the phone error rate. We also explore the properties of the learned filters learned for TIMIT phone recognition from both perceptual and statistical standpoints. We find that the filters in the first layer, which directly operate on the waveform, are in accord with the prior knowledge utilised in designing and engineering standard filters such as mel-scale triangular filters. That is, the networks learn to pay more attention to perceptually significant spectral neighbourhoods where the data centroid is located, and the variance and Shannon entropy are highest.
Introduction
Deep learning has revolutionized speech recognition, yet standard Convolutional Neural Networks (CNNs) often function as "black boxes." When processing raw audio, the filters learned in the first layer of a standard CNN are frequently chaotic and difficult to interpret in either the time or frequency domains. Furthermore, these models require a high number of parameters—one for every tap in the filter—which necessitates massive datasets for effective training.
This research addresses these challenges by introducing a generalized framework for Kernelised CNNs. By replacing standard filters with parametric kernels inspired by biology and physics, the authors create models that are more accurate, faster to train, and remarkably aligned with the human auditory system.
Methodology
The core innovation lies in a general formulation for modulated kernels. Instead of learning every weight in a filter independently, the network learns a few parameters for a modulated kernel:
$$h(t) = K(t; \theta) \cdot \cos(2\pi f_c t)$$
Where $K(t)$ is a baseband kernel and the cosine term acts as a "carrier" to shift the filter to a specific center frequency ($f_c$). The study evaluates four specific kernel types:
- SincNet (Baseline): Uses a sinc function (rectangular in frequency).
- Sinc²Net: Uses a squared-sinc function, resulting in triangular filters in the frequency domain, similar to the classic Mel-filterbank.
- GammaNet: Employs Gammatone filters, which mimic the human cochlea.
- GaussNet: Uses Gaussian filters for a smooth bell-shaped response.
Key Insights and Perceptual Alignment
Evaluation on the TIMIT dataset revealed that these parametric models naturally align with human auditory perception and data statistics:
- Perceptual Frequency Allocation: Without explicit constraints, the networks learned to place more filters in the lower frequency range (below 2 kHz). This distribution closely matches perceptual scales like Mel and Bark, where human hearing has higher resolution.
- Statistical Efficiency: The network focused its "attention" on spectral regions where the data exhibited the highest variance and Shannon entropy (information content).
- Biological Verification: For GammaNet, the network learned an average "filter order" of 4.3, strikingly close to the value of 4 typically used in auditory research to model the human cochlea.
- Temporal Context: Optimal performance was achieved using a 200ms window, suggesting the network captures medium-term speech properties like syllables that standard 25ms windows miss.
Experimental Results
The parametric models outperformed both standard CNNs and the original SincNet on the TIMIT phone recognition task. Beyond accuracy, these models are significantly more efficient; while a standard CNN filter might require 129 parameters, these kernel-based filters require only 2 or 3.
| Model | Phone Error Rate (PER) |
|---|---|
| Standard MLP (MFCCs) | 18.5% |
| Standard CNN (Raw) | 18.2% |
| SincNet | 17.6% |
| Sinc²Net (Triangular) | 16.9% |
| GaussNet | 17.0% |
| GammaNet | 17.2% |
Conclusion
By constraining the first layer of a CNN to learn mathematically defined kernels, we achieve a model that is both high-performing and highly interpretable. While the results on the clean TIMIT dataset are promising, future work will explore the robustness of these kernels in noisy, real-world environments and investigate extending this parametric approach to deeper layers of the network.
Summary updated