Abstract
We propose a novel neural network model for music signal processing using vector product neurons and dimensionality transformations. Here, the inputs are first mapped from real values into three-dimensional vectors then fed into a three-dimensional vector product neural network where the inputs, outputs, and weights are all three-dimensional values. Next, the final outputs are mapped back to the reals. Two methods for dimensionality transformation are proposed, one via context windows and the other via spectral coloring. Experimental results on the iKala dataset for blind singing voice separation confirm the efficacy of our model.
Introduction
In the field of Music Information Retrieval (MIR), tasks such as blind singing voice separation—isolating a vocalist from background instrumentation—traditionally rely on processing spectrograms. Standard deep learning approaches treat each time-frequency unit in a spectrogram as a scalar real number. However, audio signals are inherently contextual; a single point in time is deeply influenced by its temporal neighbors. While traditional Deep Neural Networks (DNNs) can attempt to capture this by stacking frames, they often struggle to model the complex, multi-dimensional interactions between these frames effectively. This research explores whether a specialized architecture designed for multi-dimensional data, specifically Vector Product Neural Networks (VPNNs), can better capture these relationships.
Methodology
This work modernizes the Vector Product Neural Network (VPNN), an architecture where neurons operate on 3D vectors using the mathematical cross product rather than scalar weighted sums.
The Vector Product Operation
The core innovation lies in the cross-product operation ($x \times y$). Because the cross product of two 3D vectors results in a third vector where each component is a combination of the others (e.g., $x_2y_3 - x_3y_2$), the network inherently captures inter-dimensional dependencies. To ensure computational efficiency on modern hardware, the authors reformulated these vector products into a matrix multiplication format suitable for GPU acceleration.
Dimensionality Transformation
Since raw audio spectrograms consist of scalar values, the authors proposed two "lifting" techniques to project data into 3D vector space:
- Context-Windowed (WVPNN): Each scalar is mapped to a vector containing the previous, current, and next frames $[t-1, t, t+1]$.
- Spectral Coloring (CVPNN): Each scalar is mapped to a 3D RGB vector using a "hot" colormap logic, treating audio intensity as a color value.
Experimental Setup
The models were evaluated using the iKala dataset, a benchmark for singing voice separation containing 252 song clips. Due to hardware constraints, 63 clips were used for training and 189 for testing. Audio was downsampled to 16,000 Hz, and Short-Time Fourier Transform (STFT) was applied to generate magnitude spectra for processing.
Results and Discussion
The experimental results demonstrate a clear structural advantage for VPNN-based architectures. As shown in the performance comparison below, VPNNs consistently outperformed standard DNNs, even when the DNNs were given significantly more parameters to ensure a fair comparison.
Key findings include:
- CVPNN Performance: The Spectral Coloring approach (CVPNN) achieved a Global Normalized Source-to-Distortion Ratio (GNSDR) of 8.87 dB. This outperformed a standard DNN with the same number of neurons (8.16 dB) and even a much larger DNN ($DNN_2$) with three times the parameters (8.37 dB).
- WVPNN Superiority: The Context-Windowed VPNN (WVPNN) achieved the highest overall performance with a GNSDR of 9.01 dB and a Global Source-to-Interference Ratio (GSIR) of 13.82 dB.
- Efficiency: The fact that VPNNs outperform larger DNNs suggests that the mathematical structure of the vector product is fundamentally better suited for modeling signal relationships than simple weighted sums.
Conclusion
Vector Product Neural Networks offer a promising alternative to traditional real-valued networks for music signal processing. By leveraging the cross-product to capture inter-dimensional dependencies, these models achieve superior source separation with higher parameter efficiency. Future research will focus on extending these vector operations to Convolutional (CNN) and Recurrent (RNN) architectures and exploring N-dimensional generalizations beyond the 3D constraint of the standard vector product.
Summary updated