A FULLY COMPLEX-VALUED RADIAL BASIS FUNCTION NETWORK AND ITS LEARNING ALGORITHM

Savitha Ramasamy, Suresh Sundaram, N. SundararajanInternational Journal of Neural Systems 2009 · DOI

TL;DR

This paper introduces a novel "Fully Complex-Valued Radial Basis Function" (FC-RBF) network that utilizes a complex-valued activation function, allowing it to process and output complex numbers directly. This advancement leads to improved performance in tasks requiring accurate phase handling, such as signal equalization and beamforming, outperforming existing complex-valued RBF networks.

What Problem Does This Paper Address?

Many real-world signals, particularly in fields like communications engineering, signal processing, and medical imaging, are inherently complex-valued. Traditional neural networks often struggle to effectively process these signals, either by treating them as separate real and imaginary components (split-complex) or by mapping complex inputs to real outputs. This limitation hinders their ability to accurately capture and manipulate the phase information crucial for many applications. Existing complex-valued RBF networks, while using complex weights, often employ activation functions that map complex inputs to real outputs, resulting in a loss of phase information during the learning process. This paper aims to bridge this gap by proposing a truly complex-valued RBF network with a complex-valued activation function.

Methods

The core of this work is the introduction of a Fully Complex-Valued Radial Basis Function (FC-RBF) network. The key innovation lies in its fully complex-valued activation function, specifically the sech(.) function. Unlike previous approaches where activation functions mapped complex inputs to real outputs (Cⁿ → R), this new function maps complex inputs to complex outputs (Cⁿ → C). This allows the network to maintain and process complex-valued information throughout its layers, including the gradient calculations.

To address the challenge of determining the optimal network architecture (number of hidden neurons) and parameter initialization (neuron centers), the authors propose a K-means clustering (KMC) based initialization scheme. This method helps in selecting an efficient network size and initializing the neuron centers, which are critical for the performance of RBF networks.

The learning algorithm is a complex-valued gradient descent method tailored for the FC-RBF network. This algorithm leverages the properties of the complex-valued activation function to compute complex-valued gradients, leading to more accurate phase approximation.

Datasets

The paper evaluates the proposed FC-RBF network on several benchmark and real-world problems:

  • Complex XOR problem: Used to study the convergence characteristics of the network.
  • Synthetic function approximation problem: Defined by the equation Z = Z₁² + Z₂³, used to assess the network's approximation capabilities.
  • Two-spiral classification problem: A challenging non-linear classification task to evaluate the network's ability to handle complex data patterns.
  • Non-minimum phase equalization problem: A real-world application in communications to test the network's performance in signal restoration.
  • Adaptive beam-forming problem: Another real-world application in array signal processing to evaluate the network's ability to direct signals and suppress interference.

The paper does not introduce new datasets but uses established benchmark problems and standard real-world scenarios.

Key Insights and Ideas

  • The power of complex-valued activation functions: The paper strongly advocates for the use of activation functions that operate entirely in the complex domain (Cⁿ → C). This is crucial for applications where phase information is as important as magnitude.
  • Sech(.) as a suitable complex activation: The authors demonstrate that the sech(.) function possesses desirable properties for a complex-valued activation function, including analyticity and Gaussian-like characteristics, making it a good choice for RBF networks.
  • K-means clustering for efficient initialization: The KMC algorithm proves to be an effective method for determining the network's structure and initializing its parameters, leading to better performance and potentially smaller network sizes.
  • Improved phase approximation: By maintaining complex values throughout the network, the FC-RBF network significantly improves its ability to approximate phase, which is a limitation in previous complex-valued RBF architectures.

Key Results

The FC-RBF network, especially when initialized with KMC, demonstrated superior performance across various tasks:

  • Convergence: Achieved convergence for the complex XOR problem with fewer epochs compared to the CRBF network.
  • Function Approximation: Showed at least a 10% reduction in both magnitude and phase errors compared to other existing complex-valued RBF algorithms.
  • Classification: Achieved 100% accuracy on the two-spiral problem with approximately 20 fewer neurons than other methods.
  • Non-minimum Phase Equalization: Outperformed other networks by at least 0.4 dB SER at 16 dB SNR, with KMC further improving this by another 0.2 dB SER.
  • Adaptive Beam-Forming: Generated beam patterns that closely matched the optimum matrix method, outperforming other complex-valued RBF networks in isolating nulls and achieving desired gains.

Limitations and Future Work

  • Initialization Heuristics: While KMC is proposed, the initial selection of the number of hidden neurons and learning rates still involves some heuristic procedures or extensive search, which could be further optimized.
  • Computational Cost: Although the paper highlights faster convergence, the computational complexity of complex-valued operations might still be a consideration for very large-scale problems.

The paper implicitly suggests that further research could focus on exploring other complex-valued activation functions and refining the initialization strategies for even greater efficiency and performance. The success in various applications also opens doors for exploring FC-RBF networks in other complex-valued signal processing domains.

Summary updated

Abstract

In this paper, a fully complex-valued radial basis function (FC-RBF) network with a fully complex-valued activation function has been proposed, and its complex-valued gradient descent learning algorithm has been developed. The fully complex activation function, sech(.) of the proposed network, satisfies all the properties needed for a complex-valued activation function and has Gaussian-like characteristics. It maps C(n) --> C, unlike the existing activation functions of complex-valued RBF network that maps C(n) --> R. Since the performance of the complex-RBF network depends on the number of neurons and initialization of network parameters, we propose a K-means clustering based neuron selection and center initialization scheme. First, we present a study on convergence using complex XOR problem. Next, we present a synthetic function approximation problem and the two-spiral classification problem. Finally, we present the results for two practical applications, viz., a non-minimum phase equalization and an adaptive beam-forming problem. The performance of the network was compared with other well-known complex-valued RBF networks available in literature, viz., split-complex CRBF, CMRAN and the CELM. The results indicate that the proposed fully complex-valued network has better convergence, approximation and classification ability.
BibTeX
@article{ramasamy2009fully,
  author = {Savitha Ramasamy and Suresh Sundaram and N. Sundararajan},
  title = {{A FULLY COMPLEX-VALUED RADIAL BASIS FUNCTION NETWORK AND ITS LEARNING ALGORITHM}},
  journal = {International Journal of Neural Systems},
  year = {2009},
  doi = {10.1142/s0129065709002026},
}
Figure